Re: word spam

(monier@pa.dec.com)
Wed, 17 Apr 96 09:02:46 -0700


> What do you mean by "Someone published the URL"? Do you mean that someone
> has to explictly link to these pages, or that these files are simply located
> in the data directory of the Web Server?

Unfortunately, same thing. If someone publishes the directory, AND the
directory browsing feature on the server is not disabled, THEN scooter will get
a nicely formatted html page containing pointers to every file in the directory,
and there is no easy way for me (short of really ugly heuristics) to detect the
situation. But I wish I could. Plea to most webmasters: please please please,
disable the directory indexing feature, it is rarely needed, and causes robots
to pick a lot of junk.

--Louis