Re: robots.txt

Martijn Koster (m.koster@webcrawler.com)
Thu, 21 Nov 1996 12:54:03 -0800


At 12:53 PM 11/21/96, David M Banes wrote:

>On the other hand, it does fall within the general description of a
>robot, so maybe after all one standard will suffice. Perhaps call the
>file 'access.txt', which any type of software should obey.

Software including Netscape Navigator? :-)

That's what HTTP 4Xx 5xx status codes are for :-)

-- Martijn

Email: m.koster@webcrawler.com
WWW: http://info.webcrawler.com/mak/mak.html

_________________________________________________
This messages was sent by the robots mailing list. To unsubscribe, send mail
to robots-request@webcrawler.com with the word "unsubscribe" in the body.
For more info see http://info.webcrawler.com/mak/projects/robots/robots.html