Crawler
WebDynamic™ Web Glossary
A program that is sent out by search engines or directories that returns information about a site content for inclusion into search engine databases. Also referred to as spiders and scrubbers.