RE: How can IR Agents be evaluate ?

Dan Quigley (DanQ@quigleys.com)
Sat, 10 Aug 1996 14:56:33 -0700


I have not heard or read of any evaluation "standard" for retrieval
systems other than those typically used to benchmark database engines.
I believe that any fair comparison of the current genre of self-indexed
IR engines could only occur at fundamental data processing levels.

In approx. 1991(?) Bell Labs published an interesting paper (benchmarks
and other research data) on a self-adaptive IR system. It was built
around their proprietary network protocol which never gained market
acceptance. I heard they still use that search engine in house.
Searching their web site or making a call to their New Jersey office may
prove useful to you.

The "agents" word is being tossed around a lot these days so its getting
tough to really know what people mean by that. A quantified definition
of agents would be a good place to start.

Dan Quigley

>----------
>From: fiorenti@cirano.cpr.it[SMTP:fiorenti@cirano.cpr.it]
>Sent: Saturday, August 10, 1996 8:56 AM
>To: robots@webcrawler.com
>Subject: How can IR Agents be evaluate ?
>
>Hi,
> I'm developing a IR system based on multi-agents paradigm, in my
>thesis.
> My questions are :
>
>How can IR agents be evaluate ? Does exist any standard evaluation
>methods ? Where can I find useful information ? Is it correct to
>compare Internet Search Engine like Lycos, Altavista.... with IR agent
>systems ? I think that they are complementary, dont'you ?
>
> Any suggestions are welcome.
>
> Thanks. Bye.
>
> -G-
>PS
>IR = Information Retrieval
>