As per usual, it depends on the robot. WebCrawler's current behaviour
is to ignore the page if the refresh time is below some threshold.
The thinking is that if the site doesn't want the page on the user's
retina for any length of time we don't want it in our index. Also means
that some spammers are out of luck :)
-- Martijn
Email: m.koster@webcrawler.com
WWW: http://info.webcrawler.com/mak/mak.html
_________________________________________________
This messages was sent by the robots mailing list. To unsubscribe, send mail
to robots-request@webcrawler.com with the word "unsubscribe" in the body.
For more info see http://info.webcrawler.com/mak/projects/robots/robots.html