A spider from magellan.mckinley.com hit us hard today and did a
deep recursive search of our web tree. Not very friendly, but their
spider did check /robots.txt which indicates that they may have
successfully implemented the robot exclusion protocol.
Christopher Penrose
penrose@ucsd.edu
http://www-crca.ucsd.edu/TajMahal/after.html
here is their internic info if anyone else wants to complain to them:
The McKinley Group (MCKINLEY-DOM)
85 Liberty Ship Way Suite 201
Sausalito, CA 94965
Domain Name: MCKINLEY.COM
Administrative Contact, Technical Contact, Zone Contact:
Cohen, Alexander J. (ASC2) xcohen@MCKINLEY.COM
415-331-1884 FAX
Record last updated on 21-Sep-95.
Record created on 14-Jul-94.
Domain servers in listed order:
NS1.NOC.NETCOM.NET 204.31.1.1
NS2.NOC.NETCOM.NET 204.31.1.2