--Louis
>----------
>From: John D. Pritchard[SMTP:jdp@cs.columbia.edu]
>Sent: Monday, June 03, 1996 9:04 AM
>To: robots@webcrawler.com
>Subject: Re: alta vista and virtualvin.com
>
>
>i have always thought that one builds a site with a couple of fairly
>static
>pages under which the dynamic stuff is accessable. this provides a way
>to
>index the site. the static pages are pointed to by robots.txt. i
>don't
>see an unsolved problem here, but i'm not a heavy shtml, et al, user --
>rather a java fan.
>
>the w3 has an html3.2 instead-of-cookies, btw. admittedly if robots
>ignore
>robots.txt and become mired in informational quicksand then the server
>would either need to internally maintain somekind of cookies scheme
>(which
>is anti statlessness) or use cookies or OBJECTs to keep anyone,
>including
>dumb machines, out of quicksand.
>
>-john
>
>