I don't think there was any sense of consensus that a robot is
something not 'controlled by a human'. It seems clear that a more
specific defninition (see my previous mail) is needed. Additionally,
I think there is consensus that AA is a User-Agent that should follow
robots.txt. Other than Robert, does anyone feel that User-Agents like
AA should _not_ follow the REP in some form?
As in my suggested definition, it seems robot 'categories' makes a lot
of sense. Rob would likely want to exclude all 'robot's, while many
sites may only want to restrict accesses from 'indexers' or 'user
directed agents' or whatever.
...Matthew
_________________________________________________
This messages was sent by the robots mailing list. To unsubscribe, send mail
to robots-request@webcrawler.com with the word "unsubscribe" in the body.
For more info see http://info.webcrawler.com/mak/projects/robots/robots.html