> Browsers only have to implement an authentication scheme that is
> difficult enough to spoof that it's easier for robots to either obtain
> their own authentication or follow some other RES.
>
> Here's a zoneball one:
>
> The authentication is:
>
> User-CRC: <some-funky-number>
>
> Where <some-funky-number> is an encoding of the User-Agent and IP
> address.
>
> All the big boys know the encoding / decoding function, and people who
> do some research will find out about it. What this does is elevate the
> level of expertise required to create a robot --- no longer can folks
> just whip one out. Granted, people who want to abuse the system
> _still_ can, what we've done is make it difficult for naive users to
> accidentally botch it.
play ball
if the function was parameterized uniquely for each user agent and the
parameters changed periodically in secured messaging from browser-agent
producers to wwweb servers.... then it would be more interesting.
the trick would be to make the parameter space large enough so that
cracking one could take as long as the reset period. each additional
parameter exponentially expands the function space. the number of
paramters should be variable. i think i've seen a standard for specifying
function parameters.?. maybe X.509 certificates.
the function spec would be distributed among servers in a hierarchical
trust delegation to distribute secure communication overhead.
then some User-Agents would be authenticated for some servers subject to
communication overhead. at least within the update period the server would
just have to do the decoding once and then recognize the correct encode.
trusted servers could be relied on to tell others' servers which
User-Agent/IPs were rejected with false encodings.
-john
_________________________________________________
This messages was sent by the robots mailing list. To unsubscribe, send mail
to robots-request@webcrawler.com with the word "unsubscribe" in the body.
For more info see http://info.webcrawler.com/mak/projects/robots/robots.html