______________________________ Reply Separator _________________________________
Subject: Re: databases for spiders
Author: Non-HP-owner-robots (owner-robots@webcrawler.com) at HP-Taiwan,mimegw34
Date: 11/15/96 4:19 AM
The Verity full-text indexes have a 255-field flat field database in
addition to the full word index, so we we store the URLs with the search
engine. This means that for every document indexed by the spider, there is
also a searchable field that contains every URL in that document.
Nick Arnett
Evangelist
Verity Inc.
_________________________________________________
This messages was sent by the robots mailing list. To unsubscribe, send mail
to robots-request@webcrawler.com with the word "unsubscribe" in the body.
For more info see http://info.webcrawler.com/mak/projects/robots/robots.html
_________________________________________________
This messages was sent by the robots mailing list. To unsubscribe, send mail
to robots-request@webcrawler.com with the word "unsubscribe" in the body.
For more info see http://info.webcrawler.com/mak/projects/robots/robots.html