+A **crawler** is an automated program designed to systematically browse and explore the [Internet](/wiki/internet). It follows links from page to page, acting as a tireless [Bot](/wiki/bot) to discover and index vast amounts of content.
+## See also
+- [Search Engine](/wiki/search_engine)
+- [Web Scraping](/wiki/web_scraping)
+- [Indexing](/wiki/indexing)
... 1 more lines