>Alta Vista uses a fast robot. I ran this robot for a week and got 16M pages.
>If I had run it for two weeks I would no doubt have 25-30M pages today. Once I
Are you saying that your entire database was obtained in a week?!
What do the dates mean in the listings that are returned?--the date the
file was created, changed, or documented by AV? I've been using AV to trace
links to our page, and I can use the advanced query and divide things up by
time period. When I do this the distribution of pages is over a matter of
months, not a week. (By the way, there seems to be a bug in the date
feature, since it returns data even when you set it to the future, and
there are a couple other odd things.)
Are you able to update the database in real time, or do you have to rebuild
it every time you add/revise data?
--Mark