> Therefore, it is unclear to me if robots.txt is appropriate;
And this is a very important point. For anything to ever be
considered as a standard (much less actually BE one), it must
define clearly what is and is not included. Many client-agent
programs are appearing, and it's my opinion that most of them
are mis-classified as "robots".
Maybe you should call this the "agents.txt" file, develop a
standard for client-driven, user-controlled agents. This is
distinctly different from the concept of a "robot" (i.e. NOT
human-controlled).
... Robert
_________________________________________________
This messages was sent by the robots mailing list. To unsubscribe, send mail
to robots-request@webcrawler.com with the word "unsubscribe" in the body.
For more info see http://info.webcrawler.com/mak/projects/robots/robots.html