Re: Does this count as a robot?
YUWONO BUDI (yuwono@uxmail.ust.hk)
Mon, 8 Jan 1996 19:34:51 -0500 (EST)
According to your message:
>
> >One trick that I use to get around symbolic-link loops is to
> >detect any recurring path segment (a /x/) in a URL. Hopefully,
> >no web author creates a subdirectory with the same name as
> >its parent or grand* parent directory (in which case my robot would
> >think there is a loop and stop there). So far (half a thousand
> >sites we visited), I haven't seen such a case.
>
> er, yes, but if there was such a naming convention then you wouldn't be able
> to tell the difference between that and a recurring path segment.
> Would you?
I wouldn't. Then again I need not, because my philosophy on robot
behavior is "be as non-aggressive as possible," so my robot would
simply give it up. If I were a web site admin, I would appreciate
that in a robot.
Anyway, this trick was actually a 5-minute-hacking solution.
-Budi.