Re: Does this count as a robot?

M.Levy@cs.ucl.ac.uk
Mon, 08 Jan 96 18:41:18 +0000

>One trick that I use to get around symbolic-link loops is to
>detect any recurring path segment (a /x/) in a URL. Hopefully,
>no web author creates a subdirectory with the same name as
>its parent or grand* parent directory (in which case my robot would
>think there is a loop and stop there). So far (half a thousand
>sites we visited), I haven't seen such a case.
>
>-Budi.

er, yes, but if there was such a naming convention then you wouldn't be able
to tell the difference between that and a recurring path segment.
Would you?

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
||Micah Levy Department of Computer Science ||
|| University College London ||
||Web Page: http://www.cs.ucl.ac.uk/students/M.Levy/ ||
||Email: M.Levy@cs.ucl.ac.uk Cestor@delphi.com ||
|| zcacma0@cs.ucl.ac.uk Micah@delphi.com ||
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||