The security on our site is not meant to be high: it is there primarily so
that the forms CGI scripts have a unique user name to figure out who is doing
what. Thus for our site we would probably be happy to just place a user name
and password in robots.txt, or some similar low-security solution. However, I
can see that for other sites this might not be an acceptable, so spider
maintainers might want to consider adding fields for the username and password
to use to their 'Please index this URL' submission forms. Then, ideally, it
should be possible to submit these forms securely.
--Roger Dearnaley (roger_dearnaley@intouchgroup.com)