RE: Server name in /robots.txt

David Eagles (eaglesd@planets.com.au)
Sat, 20 Jan 1996 09:13:25 +-1100


------ =_NextPart_000_01BAE717.8ECEEDE0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

> A good solution would be the inclusion of a new optional html tag:
> <host=3D"www.anchovies.com">

True, but if we're in the process of adding new tags, etc, wouldn't it =
be better to add an HTTP response field generated by the server as in =
the URL line below:

HTTP/9.5 200 Document follows
Date: ...
Server: ...
URL: http://www.anchovies.com/pizza

Since the server has absolute knowledge of what it's hostname is =
(hopefully :-), this would even solve the www.mcom.com and =
www.netscape.com problem. This can also resolve the other problems we =
all have trying to guess whether the URL we requested was actually a =
file or if it was a directory and the server automatically supplied the =
default (index.html) for it

Regards,
David

------ =_NextPart_000_01BAE717.8ECEEDE0
Content-Type: application/ms-tnef
Content-Transfer-Encoding: base64

eJ8+IhsWAQaQCAAEAAAAAAABAAEAAQeQBgAIAAAA5AQAAAAAAADoAAENgAQAAgAAAAIAAgABBJAG
ACQBAAABAAAADAAAAAMAADADAAAACwAPDgAAAAACAf8PAQAAAEkAAAAAAAAAgSsfpL6jEBmdbgDd
AQ9UAgAAAAByb2JvdHNAd2ViY3Jhd2xlci5jb20AU01UUAByb2JvdHNAd2ViY3Jhd2xlci5jb20A
AAAAHgACMAEAAAAFAAAAU01UUAAAAAAeAAMwAQAAABYAAAByb2JvdHNAd2ViY3Jhd2xlci5jb20A
AAADABUMAQAAAAMA/g8GAAAAHgABMAEAAAAYAAAAJ3JvYm90c0B3ZWJjcmF3bGVyLmNvbScAAgEL
MAEAAAAbAAAAU01UUDpST0JPVFNAV0VCQ1JBV0xFUi5DT00AAAMAADkAAAAACwBAOgEAAAACAfYP
AQAAAAQAAAAAAAAD0jcBCIAHABgAAABJUE0uTWljcm9zb2Z0IE1haWwuTm90ZQAxCAEEgAEAHwAA
AFJFOiBTZXJ2ZXIgbmFtZSBpbiAvcm9ib3RzLnR4dACWCgEFgAMADgAAAMwHAQAUAAkADQAZAAYA
HQEBIIADAA4AAADMBwEAFAAJAAIAIwAGABwBAQmAAQAhAAAAQTc1OTlCQ0I5RTUyQ0YxMTk4NkEw
MDAwQzA4QzAzNEUAKAcBA5AGAJwEAAASAAAACwAjAAAAAAADACYAAAAAAAsAKQAAAAAAAwA2AAAA
AABAADkAIMmFWrvmugEeAHAAAQAAAB8AAABSRTogU2VydmVyIG5hbWUgaW4gL3JvYm90cy50eHQA
AAIBcQABAAAAFgAAAAG65rtWwcubWahSnhHPmGoAAMCMA04AAB4AHgwBAAAABQAAAFNNVFAAAAAA
HgAfDAEAAAASAAAAZWFnbGVzZEBwYy5jb20uYXUAAAADAAYQ/US0tAMABxBGAgAAHgAIEAEAAABl
AAAAQUdPT0RTT0xVVElPTldPVUxEQkVUSEVJTkNMVVNJT05PRkFORVdPUFRJT05BTEhUTUxUQUc6
PEhPU1Q9IldXV0FOQ0hPVklFU0NPTSJUUlVFLEJVVElGV0VSRUlOVEhFUFJPQwAAAAACAQkQAQAA
ABsDAAAXAwAAzwQAAExaRnVJzfrV/wAKAQ8CFQKoBesCgwBQAvIJAgBjaArAc2V0MjcGAAbDAoMy
A8UCAHByQnER4nN0ZW0CgzM3AuQHEwKDNARGEzMxIGhGaXgJgHMTsAKAfRcKgAjPCdk7F98yNTUP
AoAKgQ2xC2BuZzEwvjMUUAsKFFEL8hNQbxPQhmMFQAqLbGkzNhxhYwtkFWFzMTgXQABAIA4+CuEb
vxzCQSBnb4kEcCBzBvB1dGkCIBQgdwhgbCEwYmUg7HRoIlALgGMKQACQIbFAb2YgYSBuB9FvhwUw
IaEHQCBodG0DIHkBkGc6CoceTx9fHIY8RGhvE8A9IncooC6TAHARcG92CJBzLgWgOG0iPgqPJcwp
1VRyPQpQLCIwIYAioCNgd2XeJxfgIqEiYxyBYweQBCBpI1JkZAuAZyOTJMFzmyxwEcBjLHAh424n
LLG/BUAiQSJAAkAEkCJgby5ygyNwA6BIVFRQIBfgzHNwAiARsCBmCJAiEbZnCfAEkGET0CIheSJj
aRGwcnYxEWEEIC1lVXhSTCAd0COgIjEXcHcHJPYp1THiLzkuNSDRAdAwIEQt8HUHgAIwVzKwBvA1
8XMp1UQzcTr8IC45sCnVBmE0YTmaNUGDOZAkcHRwOi8vKK9AL3BpenphNjxT/yKxIlQ0NRGABCAB
oCFTIlB0a242AGwJgDMgI0J3+xGAMFInBCAoQiQwB4AioLUEICgoQHANwCIAbDPQeDotKSxwInBC
YSHkZb80YAOgIVE0YCJjPDJtKYFfKXIxoSEwPDIjoHQE8GGfQsBFsxyBAmAT4C4gLCD/Q6JGwAOg
B0AhUDIiRKccoH8igAXAR1VDwSJQB0AkUWHZRMJyeS6yMUFnClAEEb9BMBHASbI1BkqBF+BxS/Ht
M4J3P6Ic0HVKsTPQI4B/MsBAoCNABcAs0TBxThMg/y6gF+Ac0AWwTsFGETP6IYCPA3EhkEbAQwJz
dXALUK8IkFEEDbFR0GwFQCgLgGkNsHguJHIpOGFPYXTxNjxSZWcLES9gOOcpMF5kJQ8mExyNFwEA
WeAAAwAQEAEAAAADABEQAAAAAEAABzDAUavWuea6AUAACDDAUavWuea6AR4APQABAAAABQAAAFJF
OiAAAAAAclM=

------ =_NextPart_000_01BAE717.8ECEEDE0--