Re: Info on authoring a Web Robot

Detlev Kalb (kalb@uzd6)
Tue, 26 Mar 1996 10:09:42 +0000

Messages sorted by: [ date ][ thread ][ subject ][ author ]
Next message: Jaakko Hyvatti: "Re: Limiting robots to top-level page only (via robots.txt)?"
Previous message: Chuck Doucette: "Limiting robots to top-level page only (via robots.txt)?"
Maybe in reply to: Jared Williams: "Info on authoring a Web Robot"
Next in thread: Keith D. Fischer: "Re: Info on authoring a Web Robot"

Jared,

try the WWW Robot and Search Engine FAQ at
http://science.smsu.edu/robot/faq/robot.html

You'll find links to MOMspider and to a patch that
will modify MOMspider into a full fledged web
crawler. I haven't tried it myself yet, but maybe it
helps.

Yours
Detlev Kalb
email: kalb.detlev@guj.de
Fon: 0049 40 3703 2021
Fax: 0049 40 3703 5652

Next message: Jaakko Hyvatti: "Re: Limiting robots to top-level page only (via robots.txt)?"
Previous message: Chuck Doucette: "Limiting robots to top-level page only (via robots.txt)?"
Maybe in reply to: Jared Williams: "Info on authoring a Web Robot"
Next in thread: Keith D. Fischer: "Re: Info on authoring a Web Robot"