There are scenarios where it's interesting. Some sites offer "up to the hour"
currency for specialized information (e.g., news). Polling doesn't seem
to be the best model for this.
Seems that when the source and sink (or producer/consumer) can
agree on what's important to know at the site, and there's a reasonable
degree of uncertainty about change rates, then notification is a grand
strategy.
Notification may open up lots of other possibly useful scenarios, which
have unanticipated consequences, as well. For instance, if a site
makes available a notification scheme to which "anyone" can hook
themselves as a dependent (receive notifications), that site may find
that lots of non-crawlers are hooking themselves. A good thing for
the non-crawler site wanting change notification, but it does raise the
notification load, to the point where it may exceed the original crawler-
load. (sort of like highway maintenance around here -- build a road,
and it encourages more houses, ergo traffic gets worse than it was
before the road).
This creates a tension between sending an "i'm changed" note,
when the administrator knows that it will create a flood of download
requests, and wanting to keep constituents apprised of the current
state of the site. If you don't know your constituents, where's the
value proposition?
> Excite's latest version of EWS (version 1.1) contains a feature that allows
> your copy of EWS to send notification to the Excite Search service when things
> that you index locally change.
A private protocol is not the same thing as a generally
available one until there exists a monopoly ;-)
I think that Mike S's comment has some merit (that it may be more
utility to the Intranet). Clearly, in the intranet, fallout concerns
can be managed. In the net-at-large...
Cheers
lef
Larry Fitzpatrick Bytes: http://www.opentext.com
Analog: 301-961-1994 Bits: 301-657-9776
Matter: Open Text, 3 Bethesda Metro Ctr, Bethesda MD 20814
_________________________________________________
This messages was sent by the robots mailing list. To unsubscribe, send mail
to robots-request@webcrawler.com with the word "unsubscribe" in the body.
For more info see http://info.webcrawler.com/mak/projects/robots/robots.html