See also: Search Engine
A program that scans the web looking for URLs. It is started at a particular web page, and then accesses all the links from it. A spider can record information about servers for the creation of an index or search facility. Spiders provide one good way to access distributed files such as those on the web.
TakeDown.NET -> “Spider”