Re: who/what uses robots.txt

Martijn Koster (m.koster@webcrawler.com)
Fri, 22 Nov 1996 08:14:29 -0800


At 2:40 PM 11/21/96, Erik Selberg wrote:

>Of course that leaves things that an author considers a "bar" an out
>not to follow a standard, even if other folks consider it a "foo."

But at the end of the day we cannot force authors to follow the
standard even if they themselves would agree it is in fact a foo.

So you haven't really lost anything.

You can only get so far with a BNF -- the rest has to be gentle (?)
persuasion :-)

-- Martijn

Email: m.koster@webcrawler.com
WWW: http://info.webcrawler.com/mak/mak.html

_________________________________________________
This messages was sent by the robots mailing list. To unsubscribe, send mail
to robots-request@webcrawler.com with the word "unsubscribe" in the body.
For more info see http://info.webcrawler.com/mak/projects/robots/robots.html