Frank Wales replies:
>
>I think that it is: the internet is built on a set of evolving standards
>that deal with the majority of cases well enough to give a stable,
>workable system that is still flexible enough to adapt quickly to
>change. I see no reason why creating a taxonomy of robots
>should be any different, nor a system of protocols for permitting
>them to access the resources of the net in an organised and
>generally-acceptable manner. If you have reasons for your
>apparent disdain, I'd be interested in hearing them (probably
>off the list).
>
I have no disdain at all. I have the greatest respect for the creators
of the REP. I have some respect for my own abilities.
Yet I cannot imagine all the infinite variety of applications
that might be seriously hindered by the REP.
Frank said:
>>So it's sometimes good practice to ignore the request, then?
>
>Steve replied:
>>Yes, I believe that it may be.
>
Frank re-replied:
>I think this definitely needs enlarging upon. Can you
>suggest where I can find some examples?
>
Easily. Suppose I have a very slow site and I cannot afford to
browse on-line. And I am a programmer. So I write an automatic
program that will do this:
It will read a file with no more than 50 URL's;
It will retrieve them one by one, including
all associated graphics, applets, etc., just like
Netscape Navigator would, issuing requests no
more frequently then once every 120 seconds;
and it will store them for my future off line browsing on my system.
Now here is a perfectly reasonable application (for me, it seems so at least)
which I believe will harm no site if executed bug-free,
and which cannot do what it is supposed to do if it followed the REP.
>Steve:
>>There is already ample jurisprudence in the United States that the
>>First Amendment DOES apply to creating programs. So the administrator
>>of xxx.lanl.gov, an American Governmental site, CANNOT revoke that
>>right or restrict it to the creation of only certain kinds of
>>programs, by creating a robots.txt file on his site.
>
>You seem to be arguing that a robots.txt file should inhibit the
>creation of software, which is certainly not what I am debating,
>nor something I believe, even if it were possible. Of course the
>Bill of Rights protects the creation of software, but it
>definitely does not extend to protecting all uses of such software,
>for example where such uses tangibly affect the freedoms or
>property of others.
>
No. I'm arguing something stronger:
that You and I and everybody else that agrees that
the REP is a very good idea CANNOT impose it on others without
their consent legally in the United States.
>They may conceivably be able to argue in court that the REP,
>plus their robots.txt file, constituted certain warnings
They may. But until they do, and until they actually have this recognized
by a court, it is not law, so they cannot act as if it were.
--Steve Simon