we are going to run a web-indexing-bot with the following
specifications:
Name: Motor
Version: 0.2
supports robots exclusion standard
fetches max 1 document per site during 5 minutes
higher priority for "*.de", "*.ch" and "*.at" Domains
supports frames, redirection and if-modified-since commands
The results will be made available without charge on our
http://www.webindex.de site. Start is scheduled for Aug-1-96.
The bot has been tested locally.