That's just one more reason to make robots that play by the rules.
Now if instead of following relative links your robot is trying "*.*" at
every directory branch, then you could be accused of rumaging through
files. Whether that would get you sued, I don't know.
-- ================================================================== Mitch Allen WebDroid http://www.webdroid.com P.O. Box 6569 mailto:mitch@webdroid.com Boston, MA. 02114 _________________________________________________ This messages was sent by the robots mailing list. To unsubscribe, send mail to robots-request@webcrawler.com with the word "unsubscribe" in the body. For more info see http://info.webcrawler.com/mak/projects/robots/robots.html