Spiders will usually take either one or both of these methods:
1 robots.txt for the whole site
2 individual meta tags for individual pages
You will need to add meta information to all pages for all type 2 robots, as
someone may link to one of your pages, and a nofollow specification in your
/index.html would have no effect on indexing of other pages.
Klaus Johannes Rusch
-- e8726057@student.tuwien.ac.at, KlausRusch@atmedia.net http://www.atmedia.net/KlausRusch/ _________________________________________________ This messages was sent by the robots mailing list. To unsubscribe, send mail to robots-request@webcrawler.com with the word "unsubscribe" in the body. For more info see http://info.webcrawler.com/mak/projects/robots/robots.html