Re: who/what uses robots.txt

Martijn Koster (m.koster@webcrawler.com)
Thu, 21 Nov 1996 12:53:56 -0800


At 11:35 PM 11/20/96, HipCrime wrote:

>Failing to be able
>to agree on the meaning of the word "robot" is to admit that
>you can't define a "robot" standard. Period.

True. However, if you have a restrictive definition of what
constitutes a "robot", and create a spec for that, there is nothing
to stop anyone to define a "foo", and decide that a "foo" should
adopt and follow the "robots" standard.

-- Martijn

Email: m.koster@webcrawler.com
WWW: http://info.webcrawler.com/mak/mak.html

_________________________________________________
This messages was sent by the robots mailing list. To unsubscribe, send mail
to robots-request@webcrawler.com with the word "unsubscribe" in the body.
For more info see http://info.webcrawler.com/mak/projects/robots/robots.html