Who sets standards (was Server name in /robots.txt)
Nick Arnett (narnett@Verity.COM)
Sat, 20 Jan 1996 16:20:38 -0800
>The canonical-name issue could be settled, I suppose, if we and Infoseek and
>Lycos and Excite got together and said "do it THIS way". I think
><META> is probably the way to go, since <BASE> is overloaded for other
>functions.
Not to be too parochial, but I think it's at least as important to get
buy-in from companies who are shipping commercial spiders, since there are
a lot more copies of them out there than there are of the services. A
standard endorsed by the services doesn't matter to someone who's indexing
his or her intranet.
Nick