Re: Web pages being served from an SQL database

Randy Fischer (fischer@ucet.ufl.edu)
Tue, 17 Dec 1996 16:27:12 -0500


> ...Lycos mentions that it doesn't look at any URLs that contain "?".
> Is there any way to convince the robots that it is OK to go ahead and
> traverse these URLs? Does anyone have any suggestions as to what would
> be the right solution?

You could encode everything as extra path information I suppose,
and forget explicit parameters. Any reason that wouldn't work?

Randy
_________________________________________________
This messages was sent by the robots mailing list. To unsubscribe, send mail
to robots-request@webcrawler.com with the word "unsubscribe" in the body.
For more info see http://info.webcrawler.com/mak/projects/robots/robots.html