Re: Possible robots.txt addition
Ian Graham (ianweb@smaug.java.utoronto.ca)
Wed, 6 Nov 1996 12:23:11 -0500 (EST)
>
> > Not knowing a more appropriate forum, I thought I would bring up this
> > issue here.
> >
> > A common problem (at least within our organization) is the expiry and/or
> > change of domain names: internal departmental or divisional reorganizations
> > lead to changes in domain names (in our case, *.utirc.utoronto.ca to
> > *.hprc.utoronto.ca) and the eventual elimination of 'obsolete' domains,
> > after some period of coexistence.
> >
......
> >
> > I therefore propose adding two new field to robots.txt, to indicate both
> > preferred and obsolete domain names for a given server. For example:
> >
> > use-domain: preferred.domain.name
> > obsolete-domain: obsolete.domain.name
>
> This seems like a really useful idea. It certainly isn't robot exclusion, but
> robots.txt seems as appropriate a place as any for it. (It could go in another
> file as well) I'd like to see something like this though:
>
> use-name: myhost.mydomain.edu
>
> Which would allow a robot to know what hostname to return for the URLs on that
> site when that host goes by many different names.
>
> Anyone?
>
> Issac
This is essentally what I mean by use-domain (I just shortened it from
"use-domain-name"). Use-domain-name: might be clearer.
Ian
--
Ian Graham ........................................ ian.graham@utoronto.ca
Information Commons Tel: 416-978-4548
University of Toronto Fax: 416-978-0440