What is 008?
If you've come to this page, then you're probably interested in learning more about our web crawler, identified as user-agent "008".
008 runs on a grid computing platform that consists of several thousand computers, which is why you may see our web crawler access your site from many different IP addresses.
Why is 008 crawling my website?
008 is the user-agent used by 80legs, a web-crawling service provider. 80legs allows its users to design and run custom web crawls. So, if 008 is crawling your website, it means that one or more 80legs users created a crawl that went (eventually) to your website.
People use 80legs for a variety of reasons, including providing data to their own search engines, monitoring trends in online opinions, and other interesting applications.
Help us crawl your site properly
If you feel that 008 is crawling your site too quickly, please let us know what an appropriate crawl rate is. If you'd like us to stop crawling your website, the best thing to do is to block us using the robots.txt specification. If you block 008 using robots.txt, you will see crawl requests die down gradually, rather than immediately.
Learn more
To read more about the inner workings of 008, please visit our wiki.
To learn more about 80legs, please check out the rest of the site. If you'd like to ask us any questions, please contact us.