Re: Alta Vista searches WHAT?!?

Martijn Koster (m.koster@webcrawler.com)
Tue, 16 Jan 1996 13:25:45 -0700


At 1:05 PM 1/16/96, Ed Carp @ TSSUN5 wrote:
>There has been a concern raised on another list that I belong to, about the
>privacy implications of robots and such. The specific example was that the
>Alta Vista web crawler didn't only index linked documents, but any and all
>documents that it could find at a site! Is this true, and if so, how is it
>doing it?

What is this Alta-Vista vicious rumour mill stuff on the list recently? :-)

If you have a question, ask the robot author, his email address is on
the robots page...

It would also help if you included the complete referenced article --
I wouldn't be in the least surprised if the person's files in question
were in fact reacheable from the web, and therefore findable by any browser
or robot.

>How does one keep documents private? I sure don't want my personal
>correspondence sitting out on someone's database just because my home directory
>happens to be readable!

To protect pages from other people, configure your server to return
"access denied" for them...

-- Martijn

Email: m.koster@webcrawler.com
WWW: http://info.webcrawler.com/mak/mak.html