Re: servers that don't return a 404 for "not found"
Aaron Nabil (nabil@teleport.com)
Mon, 14 Oct 1996 06:02:48 -0700 (PDT)
On Mon, 14 Oct 1996, Aaron Nabil wrote:
> For each of the 200,000 hosts in my database, try sending them a
> bogus URL, and if they don't return a 404, label them as "unindexable"
> and stick them in my "site stop list" file. You know, the one that
> contains xxx.lanl.gov. :)
Let me restate that as "if they DO return a 200" as opposed to "if
they DON'T return a 404."
-a