I write because I realize there are issues regarding its behavior toward
other sites, some of which have come up on this list in recent months. As
a minimum, the Metacrawler will religiously abide by the standard for
robot exclusion. However, if you have concerns, questions, or suggestions
regarding the Metacrawler's past, present, or future behavior, please
bring them up here if you feel they're appropriate for the list, or email
me privately if not.
Thanks.
-- Paul Phillips | why would you want to own /dev/null? "ooo! ooo! look! Jocular Cynic | i stole nothing! i'm the thief of nihilism! i'm the new <paulp@go2net.com> | god of zen monks." +1 206 447 1595 | -- Kevin Lyda, alt.sysadmin.recovery_________________________________________________ This messages was sent by the robots mailing list. To unsubscribe, send mail to robots-request@webcrawler.com with the word "unsubscribe" in the body. For more info see http://info.webcrawler.com/mak/projects/robots/robots.html