Lack of support for "If-Modified-Since"

Howard, Dan: CIO (howard.dan@ic.gc.ca)
Thu, 13 Feb 1997 15:25:02 -0500

I'm finding that a lot of servers don't support 'If-Modified-Since'
conditional GET's, even ones such as Netscape.com and Microsoft.com. The
application that I'm writing will be re-checking selected URLs every few
weeks for updates, so in the interest of preserving bandwidth, I'd like this
to work.

Using LWP (for perl), I use the following line to ask remote servers for any
documents that are less then half a day old:
$request->header('If-Modified-Since' => HTTP::Date::time2str(time-40000));

Many responses come back '304', but about one third of them are *still*
'200'. NCSA, Netscape and Apache servers all seem to be guilty and
not-guilty of this, depending on whose site you visit.

Is there something I'm doing wrong?

Dan Howard
Industry Canada
(613) 596-0201

_________________________________________________
This messages was sent by the robots mailing list. To unsubscribe, send mail
to robots-request@webcrawler.com with the word "unsubscribe" in the body.
For more info see http://info.webcrawler.com/mak/projects/robots/robots.html