Re: Question about Robot.txt

Klaus Johannes Rusch (e8726057@student.tuwien.ac.at)
Thu, 30 Jan 1997 21:28:35 CET

In <199701302006.MAA21092@fct.fidelca.es>, =?ISO-8859-1?Q?Alvaro_Mu=F1oz-Aycuens_Martinez?= <alvarom@fct.fidelca.es> writes:
> I always see messages in the list that they talk about robot.txt,I have
> some ideas but i donīt know very well what is robot.txt and in my server, I
> noticed that dosenīt exist and i think that robot.txt is very important for
> indexing a site.

robots.txt controls where well-behaving robots shouldn't go to. For more
information see the URL shown in the footer of any message from this list:

> For more info see http://info.webcrawler.com/mak/projects/robots/robots.html

Klaus Johannes Rusch

--
e8726057@student.tuwien.ac.at, KlausRusch@atmedia.net
http://www.atmedia.net/KlausRusch/
_________________________________________________
This messages was sent by the robots mailing list. To unsubscribe, send mail
to robots-request@webcrawler.com with the word "unsubscribe" in the body.
For more info see http://info.webcrawler.com/mak/projects/robots/robots.html