do robots send HTTP_HOST?

Joe Pruett (joey@q7.com)
Thu, 3 Oct 1996 09:04:56 -0700 (PDT)

Messages sorted by: [ date ][ thread ][ subject ][ author ]
Next message: Z Smith: "Re: The Internet Archive robot"
Previous message: Michael De La Rue: "Bug in LibWWW perl + Data::Dumper (libwwwperl refs are strange)"
Next in thread: Aaron Nabil: "Re: do robots send HTTP_HOST?"
Reply: Aaron Nabil: "Re: do robots send HTTP_HOST?"

i want to keep robots out if they aren't using the canonical name for my
web server (www.foo.com). but i can't find anywhere that documents if
robots send the HTTP_HOST header defined in http 1.0.

i'm not a subscriber to this list, so please respond to me directly with
any info you may have.

Next message: Z Smith: "Re: The Internet Archive robot"
Previous message: Michael De La Rue: "Bug in LibWWW perl + Data::Dumper (libwwwperl refs are strange)"
Next in thread: Aaron Nabil: "Re: do robots send HTTP_HOST?"
Reply: Aaron Nabil: "Re: do robots send HTTP_HOST?"