Custom Web Crawling
With custom web crawls, it only takes a few minutes to start collecting data from the web. All you need to do is fill out a simple form.
The form lets you get very specific about your crawling parameters, which tell 80legs what web pages you want to crawl and what data to collect from those web pages.
General Web Crawling
You can use 80legs to run large, general web crawls. Such crawls can collect data like web page content, outgoing links and other data. Large web crawls take advantage of 80legs' ability to run massively parallel crawls.
| With a sufficiently parallel web crawl... |
| 80legs can crawl over 5,000,000 web pages in 1 hour |
Watch this video to learn more about setting up a web crawler to collect keyword data.
Crawling Specific Websites
Another great use of 80legs is to extract specific data from a website. For example, you may want to extract user data from social networks or extract product data from a retail website. You can use 80legs to extract the data you want using custom or pre-built extractors, also known as 80apps.
To continue the tour, click on Use a Crawl Package below.