>
> I'm finding that a lot of servers don't support 'If-Modified-Since'
> conditional GET's, even ones such as Netscape.com and Microsoft.com. The
> application that I'm writing will be re-checking selected URLs every few
> weeks for updates, so in the interest of preserving bandwidth, I'd like this
> to work.
>
> Using LWP (for perl), I use the following line to ask remote servers for any
> documents that are less then half a day old:
> $request->header('If-Modified-Since' => HTTP::Date::time2str(time-40000));
>
>
> Many responses come back '304', but about one third of them are *still*
> '200'. NCSA, Netscape and Apache servers all seem to be guilty and
> not-guilty of this, depending on whose site you visit.
>
> Is there something I'm doing wrong?
'If-Modified-Since' relies on responses coming back with a "Last-Modified"
header. Dynamic HTML (e.g. with embedded Server Side Includes) will not
(usually) come with a "Last-Modified" header, thus rendering
'If-Modified-Since' useless.
_______________________________________________________________________
Rob Hartill. Internet Movie Database Ltd. http://www.imdb.com/
CGI ? how quaint.
_________________________________________________
This messages was sent by the robots mailing list. To unsubscribe, send mail
to robots-request@webcrawler.com with the word "unsubscribe" in the body.
For more info see http://info.webcrawler.com/mak/projects/robots/robots.html