Re: Broadness of Robots.txt (Re: Washington again !!!)

Martijn Koster (m.koster@webcrawler.com)
Thu, 21 Nov 1996 12:54:06 -0800


At 11:54 AM 11/21/96, Art Matheny wrote:

>I totally agree with these statements. I would suggest a slightly
>different implementation. The standard should include a list of general
>behavior classifications and assign a fictitious "User-Agent" name to each
>class.

Whhops, you beat me to it :-)

-- Martijn

Email: m.koster@webcrawler.com
WWW: http://info.webcrawler.com/mak/mak.html

_________________________________________________
This messages was sent by the robots mailing list. To unsubscribe, send mail
to robots-request@webcrawler.com with the word "unsubscribe" in the body.
For more info see http://info.webcrawler.com/mak/projects/robots/robots.html