There are lots of other web middleware facilities in the works,
and a "deny permission" element or quality of service element is
in some them. If we want smarter robots we need the middleware too.
- there are docs on the w3.org site for all manner of proposals,
watch out .. some of them may happen!
- without these middleware facilities, the number of dumb robots and
site slurpers will *proliferate* . Is robots.txt the solution of
choice to the 'personalised' robots that will come in the box with
Windows 96 say?
mj