The spec requires User-agent precedes all other lines,
so the above is invalid, and it's semantics not defined.
># 3. two agents with some lines inbetween
>User-agent: webcrawler
"lines inbetween" are not allowed, as they separate records;
The above record matches webcrawler, has no Allow/Disallow,
so WebCralwer is allowed to go anywhere.
># comments and blank lines
>
>User-agent: infoseek
>Disallow: /tmp
>
>be equivalent?
Removing this kind of ambiguity is exactly what I'm hoping a BNF and
better text will fix.
-- Martijn
Email: m.koster@webcrawler.com
WWW: http://info.webcrawler.com/mak/mak.html
_________________________________________________
This messages was sent by the robots mailing list. To unsubscribe, send mail
to robots-request@webcrawler.com with the word "unsubscribe" in the body.
For more info see http://info.webcrawler.com/mak/projects/robots/robots.html