for some of the related TRs see..
http://harvest.cs.colorado.edu/harvest/user-manual-1.1/node73.html
-john
> Has anyone thought about applying for a TCP port number dedicated for =
> intercommunications between various robots, as well as the additional =
> protocol for exchange of info. As many of you will have seen, and some =
> I have spoken to, I have developed a web crawler =
> (http://funnelweb.net.au) which performs searchs/indexing for the South =
> Pacific countries (based in Australia), selectable by individual =
> country. I have received a lot of queries in regard to others using the =
> code for various projects in other countries (and even internal =
> corporate networks). As a result, I'm currently implementing a =
> distributed searching/indexing facility. The best approach I can think =
> of is to have a dedicated port which can be used by remote agents to =
> either conduct searchs of another country's data or to register a URL =
> for processing and indexing by that agent for the remote database (hope =
> that makes sense).
>
> Any comments on this would be GREATLY appreciated.