Paul Francis writes...
> Well, though I'm curious about your answer, I think
> even this question is besides the point. I think
> anything that hits a site automatically and frequently
> enough to be noticed can legitimately be considered
> a robot.
>
> Perhaps you should cc your message to the list. I
> personally would consider your process a robot (in the reading
> of your original message, it never occured to me that what
> you ran was not a robot), and am curious as to whether
> or not other people feel the same way.
I guess the only thing I can say is that every definition of robot
always reads something like (from the 'standard for robot exclusion')
WWW Robots (also called wanderers or spiders) are programs that
traverse many pages in the World Wide Web by recursively retrieving
linked pages.
which isn't what I was doing (taking a fixed list of URL's and testing
their reachability). You seem to be advocating not only expanding the
definition of robot to "any automated access to a site" but "any access
that gets noticed by the admin." My test, again, didn't "learn" new
URLs, engage in any recursion, or even index anything.
Maybe I should have changed my User-agent to pretend to be a cache or
proxy. :-)
-- nabil@i.net http://www.i.net/aaron/