Website Crawler

A web crawler is an automated program, or script, that scans or "crawls" through web pages and extracts data from it.

Web Crawling is a method by which you can extract data from another website and use it for your own purpose (save it, display it etc.). For example, you can crawl a weather website, extract today's temparature and display it on your own site. Another example can be that you crawl a currency website and extract real time exchange rates from it and use it for calculations. Similarly, there can be numerous applications web crawling.

Our crawling scripts are written in PHP. cURL is used to crawl sites and Regular Expression are used to parse the content and extract the required information. A simple demo implementations is posted below, however most of the crawling projects are confidential and cannot be disclosed (NDA).

Demo: http://dev.cerontek.com/crawler_sample/