Re: Robot to collect web pages per site

Jeremy.Ellman (jeremy@mari.co.uk)
Thu, 6 Jun 1996 16:00:50 +0100 (BST)

Messages sorted by: [ date ][ thread ][ subject ][ author ]
Next message: John D. Pritchard: "www.kollar.com/robots.html"
Previous message: Mannina Bruno: "Recherche de documentation sur les agents intelligents ou Robots."
In reply to: Henrik Fagrell: "Robot to collect web pages per site"

>
>
> Dear robot developpers
>
>
> Does anyone know if the is robot availble that collects all the web pages
> (raw HTML), from a given site, in a file.
>
> example
>
> bash$ collectpagesfromsite http:/info.metacrawler.com/ > outputfile &
>
> Thanks for your help
>

HTMLGobble does this.

" HTMLGOBBLE GRABS HTML PAGES FROM REMOTE WEB SITES

This is not for the faint of heart. You get source code and a
makefile, but no documentation. The program apparently sucks in the
HTML from any site you specify, along with any linked files in the
same directory. Strictly for hackers. "ftp://ftp.rz.uni-
karlsruhe.de/pub/net/www/tools/htmlgobble.tar.gz"
"

Next message: John D. Pritchard: "www.kollar.com/robots.html"
Previous message: Mannina Bruno: "Recherche de documentation sur les agents intelligents ou Robots."
In reply to: Henrik Fagrell: "Robot to collect web pages per site"