Re: in-document directive to discourage indexing ?
Kevin Hoogheem (khooghee@marys.smumn.edu)
Mon, 24 Jun 96 13:12:18 -0500
I was wondering if anyone has thought that a mandatory
section of a New Robot Standard or even if anyone has talked
to anyone about making a Admin-Email entry a standard
in webservers.
Why should a Admin-Email entry be mandatory, well I can think of
one reason and that is Since most people that
have written robots have the ablity to tell if a link is bad or doesnt
work anymore with there robot they could easily send a message to the
administrator of that website with the url and description of the problem
they have on that page, It would be a way that some Big search engines
can also give back to the community and also would open up a new
door for other robots that could better perform checking of websites.