Considering these two extremes I'm "located" at the end which says
that robots should always obey to the /robots.txt file.
> Maybe it isn't the current definition, but it would seem, to
> accommodate sites such as xxx.lanl.gov, that any automation
> that might be in a position to make a decision about how its
> behavior might be socially (netwise) acceptable, whether it be
> a robot, browser, or anything else that doesn't have a human
> brain, should consult robots.txt. Robots.txt really is the
> answer for anything that isn't capable of making a truly informed
> decision, such as a human might make.
Fine! Concerning something like a definition of ROBOTS
what you write seems to me as the BEST description of what should
be a robot in the way it is discussed here.
The main point I see in your text is:
"anything that isn't capable of making a truly informed
decision, such as a human might make"
With this in mind there is no longer the question of how many pages
a program accesses as characterisation of a robot.
Would you agree that a program which checks a page for changes AFTER
a human which (hopefully) is able to make "truly informed decisions"
can be considered as "anything else" rather than as a robot?
I hoped to get an answer to this question within the previous thread,
but it got under the wheels of this hot discussion...
Mike