RE: Alta Vista searches WHAT?!?

Ted Sullivan (tsullivan@blizzard.snowymtn.com)
Tue, 16 Jan 1996 17:30:00 -0800

If you put your files in a file system area that the Web server has access
to then they will get picked up. Robots cannot find things that they cannot
see. It comes down to site security, publish what you desire the world to
see and hide the rest.

Ted Sullivan

----------
From: robots
To: robots
Subject: Alta Vista searches WHAT?!?
Date: Wednesday, January 17, 1996 2:53PM

There has been a concern raised on another list that I belong to, about the
privacy implications of robots and such. The specific example was that the
Alta Vista web crawler didn't only index linked documents, but any and all
documents that it could find at a site! Is this true, and if so, how is it
doing it? How does one keep documents private? I sure don't want my
personal
correspondence sitting out on someone's database just because my home
directory
happens to be readable!