Re: Defenses against bad robots

mred@neosoft.com
Fri, 17 May 1996 23:03:19 CST


** Reply to note from <kathy@accessone.com> 05/17/96 1:04pm PDT

> (E) Trap the robot into retrieving "a gigabyte-size HTML
> document generated on-the-fly" (1). Please reply with
> examples of this technique.

void main()
{
unsigned long i;
printf( "Content-type: text/html\n\n"
"<HTML><HEAD><TITLE>Bye-bye</TITLE></HEAD><BODY>\n" );

for( i=0; i < (-1UL); i++ )
printf( "Bye" );

printf( "</BODY></HTML>\n" );
}
> (2) Suspicious error status codes : 204 no content, 300
> multiple choices, 301 moved permanently, 302 moved
> temporarily, 303 method, 304 not modified, 400 bad request,
> 401 unauthorized, 402 payment method, 403 forbidden, 404 not
> found, 405 method not allowed, 406 none acceptable, 407 proxy
> authentication required, 408 request timeout, 409 conflict,
> 410 gone, 500 internal server error, 501 not implemented, 502
> bad gateway, 504 gateway timeout.

People very frequently generate 404 and 302 status codes.
In fact, 302 would be
returned when you set up a perfectly intentional alias in Netscape Commerce
server. Those, along with 301, cannot be deemed suspicious.

-Ed-
mred@neosoft.com
http://www.neosoft.com/~mred/