Sorry for this one-time-in-a-life newbie question that has been discussed
thousands of times on this list, but do robots have to follow links ? I
mean, will they browse/scan a page that has no link pointing to it ?
I guess it would be hell if those kinds of pages could be referenced
because everybody would publish more and more pages full of keywords and
descriptions, etc.
Thanks for any help.
Tom.
_________________________________________________
This messages was sent by the robots mailing list. To unsubscribe, send mail
to robots-request@webcrawler.com with the word "unsubscribe" in the body.
For more info see http://info.webcrawler.com/mak/projects/robots/robots.html