Re: Alta Vista searches WHAT?!?
Edward Stangler (mred@neosoft.com)
Thu, 18 Jan 1996 20:59:44 CST
** Reply to note from Erik Selberg <speed@cs.washington.edu> 01/17/96 10:47am PST
> Now, the /robots.txt won't do a bit of good here. Why? Because (a)
> robots don't have to support the robots.txt file, and (b) because the
> goal is to keep said data _private_ from everyone, not just
> robots. The problem is that users feel that hiding data is a good
> solution to security. Robots just publicly announce that security of
> that form is bogus. The issue people have with robots I think is
> bogus; what they should be addressing is that there needs to be a
> better form of protection on the Web, or at least a more intuitive
> method of setting access control lists than the funky .htaccess file
> stuff (or at least a better UI!).
What if you're using ROBOTS.TXT to exclude CGI's which don't appear in /cgi-bin?
What if the CGI's--or any data types unknown to the robot--are indistinguishable from
directory pathnames or acceptable data types except if (a) it is excluded with
something like ROBOTS.TXT or (b) the robot spends considerable time and resources to
analyze it?
-Ed-
mred@neosoft.com
http://www.neosoft.com/~mred
1:106/1076 - 30:30/0 - 85:842/105
74620,2333