>At 2 minute intervals xxx.lanl.gov can become overloaded.
>
>YOU DON'T KNOW WHAT THESE URLs DO. DON'T MAKE IGNORANT ASSUMPTIONS.
>
I should not have to make assumptions about what the URL does.
I should be able to access a few hundred URL's on your site in
a few hours, if you make those URL's available on the Web, simply
because I can do that by sitting in front of my Netscape
Browser, and it is unwise for YOU to make ANY assumptions
about what pages, in which sequence, and within reason how quickly,
I am going to attempt to access them.
>The HEAD requests at 2 minute intervals to xxx.lanl.gov cause scripts
>to run which take more than 2 minutes to finish. Pile them up and the
>thing melts down.
>
I dont care. If that's the case, do not make them accessible
on the WWW, or defend your site against meltdown by other means,
(e.g. embed in your CGI scripts increasing
amounts of sleep, if they come repeatedly from the same IP address,
to make it impossible that an errant robot or a persistent
human being accessing too many of your pages should cause your site
to melt down.) But do not transfer the responsibility for the
unintended meltdown on whoever is accessing your site.
--Steve Simon