------=_NextPart_000_01BC0645.BB44F560
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Yes, You can ftp it from
ftp://rs.internic.net/netinfo/
There is the file is about 50 MEGS!
It appears to be all listed domain names at the Internic. Does anybody
know of a Java application that will create a database of just the index
page on each of these servers?
Thanks,
Tom
----------
> From: DA <k3is@UnbSJ.CA>
> To: Yossi Cohen <ifaa1@ibm.net>
> Cc: robots@webcrawler.com
> Subject: Re: Q: size of the web in bytes, comprehensive list
> Date: Saturday, January 18, 1997 3:35 PM
>
>
>
> On Sat, 18 Jan 1997, Yossi Cohen wrote:
>
> > Did anyone measure the size of the whole web in bytes ?
> >
>
> Your are NUTTTTTTTTTTTS.
>
> > Is there a comprehensive list of all http servers ?
> >
>
> Even if there is...how good will that be? Everyday there are new servers
> configured!
>
> Are you bored?
>
> Later,
> Q.
>
> _________________________________________________
> This messages was sent by the robots mailing list. To unsubscribe, send
mail
> to robots-request@webcrawler.com with the word "unsubscribe" in the body.
> For more info see
http://info.webcrawler.com/mak/projects/robots/robots.html
------=_NextPart_000_01BC0645.BB44F560
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Yes, You can ftp it from =
ftp://rs.internic.net/netinfo/
There is the file is about 50 MEGS!
It =
appears to be all listed domain names at the Internic. Does =
anybody know of a Java application that will create a database of just =
the index page on each of these =
servers?
Thanks,
Tom
----------
> From: DA =
<k3is@UnbSJ.CA>
> To: Yossi Cohen <ifaa1@ibm.net>
> Cc: robots@webcrawler.com
> Subject: Re: Q: size of the web in bytes, =
comprehensive list
> Date: Saturday, January 18, 1997 3:35 =
PM
>
>
>
> On Sat, 18 Jan 1997, Yossi Cohen =
wrote:
>
> > Did anyone measure the size of the whole =
web in bytes ?
> >
>
> Your are =
NUTTTTTTTTTTTS.
>
> > Is there a comprehensive list of =
all http servers ?
> >
>
> Even if there is...how =
good will that be? Everyday there are new servers
> =
configured!
>
> Are you bored?
>
> =
Later,
> Q.
>
> =
_________________________________________________
> This messages =
was sent by the robots mailing list. To unsubscribe, send mail
> =
to robots-request@webcrawler.com with the word "unsubscribe" in the =
body.
> For more info see http://info.webcrawler.com/mak/projects/robots/robot=
s.html