> Sorry for this one-time-in-a-life newbie question that has been discussed
> thousands of times on this list, but do robots have to follow links ? I
> mean, will they browse/scan a page that has no link pointing to it ?
The easy answer question I would have is ... Without a link, how would the
spider know that the file existed? (assuming it's not a 'standardly'
named file (like /robots.txt))
-- ----------------------------------------------------------------------------- Theo Van Dinter www: http://www.kluge.net/~felicity/ Vice-President WPI Lens and Lights Active Member in SocComm Films Member of WPI ACM AME for the Masque C-Term Show"2 wrongs don't make right but 3 rights make a left." - TVD -----------------------------------------------------------------------------
_________________________________________________ This messages was sent by the robots mailing list. To unsubscribe, send mail to robots-request@webcrawler.com with the word "unsubscribe" in the body. For more info see http://info.webcrawler.com/mak/projects/robots/robots.html