Re: Suggestion to help robots and sites coexist a little better

Rob Hartill (robh@imdb.com)
Thu, 18 Jul 1996 17:23:23 -0600 (MDT)


Nick Arnett wrote:

>>before anyone focuses on this, please will robot owners agree a string
>>to add to the USER_AGENT so that server admins can do something with it.
>>This should be a quick and painless task; it just need doing.
>
>Um, you're suggesting that before anyone consider the proposal developed at
>a W3C workshop by people from a number of the largest search services and
>search engine developers, they should consider your proposal...?

Sorry if I'm tredding on toes. I thought we had been discussing the
robot user agent header first and that we'd reached a stage where
the idea had been accepted and that all that was needed was to choose
a name. I just didn't want to see the idea fade away if people got
distracted with this other proposal (which I don't object to).

>Could I suggest that anyone with such comments and criticism subscribe to
>the W3C discussion list for distributed search and indexing?

okay, I've done that now.

rob