The Web Robots Pages

Wget

Name Wget
Cover Page ftp://gnjilux.cc.fer.hr/pub/unix/util/wget/
Details Page
Operational Status development
Description Wget is a utility for retrieving files using HTTP and FTP protocols. It works non-interactively, and can retrieve HTML pages and FTP trees recursively. It can be used for mirroring Web pages and FTP sites, or for traversing the Web gathering data. It is run by the end user or archive maintainer.
Robot Purpose mirroring, maintenance
Software Type standalone
Software Platform unix
Software Language C
Availability source
Owner's Name Hrvoje Niksic
Owner's Home Page
Owner's Email Address hniksic@srce.hr
Exclusion Protocol yes
Exclusion Tag wget
Supports NOINDEX no
Robot Host *
HTTP From yes
HTTP User-Agent Wget/1.4.0
History
Environment hobby, research
Identifier wget
Updated Mon, 11 Nov 1996 06:00:44 MET
Update By Hrvoje Niksic

The Web Robots Database