This attempt to define "robots" has proved futile.
The deep issue (at least my impression of it), was
stated by "Thaddeus O. Cooper" <tcooper@mitre.org>:
> 5. How do we make people use robots.txt?
Seems to me, the crux of your biscuit: MAKING people
observe the robots exclusion "standard".
That being the case, consider the motives of "robot"
writers, and *make* an attempt to appeal to those.
Let's take HipCrime's ActiveAgent as an example.
Q: What is its goal?
A: To locate as many Email addresses as possible.
Would it be correct for ActiveAgent's author to implement
something that would inhibit completion of this assignment?
Programmers, be truthful to yourselves: didn't you learn
(early on), that efficiency is tantamount to success?
This is the ONLY reason that ActiveAgent does not observe the
robot-exclusion "standard": interference with its goal.
The robots-writers list is stuck on the assumption that "robots"
exist for particular purposes. That is completely NOT the case.
Each software designer (in his/her/its heart) hopes that the idea
being implemented is a NEW one. Given that, any attempt to divine
the reasons for a program's existence will surely fail.
We're talking about the WORLD wide web here. Who on this list
(really) believes that he/she can define the motives of "the world"?
Please, please, let me know, if you can, because this is the real
information that I am looking for.
.... Robert
_________________________________________________
This messages was sent by the robots mailing list. To unsubscribe, send mail
to robots-request@webcrawler.com with the word "unsubscribe" in the body.
For more info see http://info.webcrawler.com/mak/projects/robots/robots.html