Re: Freely available robot code in C available?

Edwin Carp (ecarp@tssun5.dsccc.com)
Wed, 13 Dec 1995 15:15:17 +0000


> Date: Wed, 13 Dec 1995 07:48:15 -0800
> To: robots@webcrawler.com
> From: narnett@Verity.COM (Nick Arnett)
> Subject: Re: Freely available robot code in C available?
> Reply-to: robots@webcrawler.com

> At 10:25 PM 12/12/95, ecarp@tssun5.dsccc.com wrote:
> >... then build an Oracle database
> >composed of URLs and text which could be searchable via an SQL
> >query.
>
> Aside from the question of my you want to build your own, rather
> than buying an off-the-shelf solution (we have one, too) -- why
> Oracle? A text search engine will give much better performance and
> have many more text-oriented features than Oracle or another RDBMS.
> Search engines are a kind of database, of course, but one that is
> oriented toward text, rather than fielded data (which some of them
> also support).

The problem with an off-the-shelf solution is that most of them are
not flexiabel enough for our needs. Also, we are tied to a product
that does not allow us to make any changes unless we go back to the
vendor. Customizations are likely to be expensive, and this project
is being done on a literal shoestring, using existing hardware and
home-grown software.

Oracle, because taht's what we have in-house, and we have lots and
lots of reporting and search tools for it.