Re: in-document directive to discourage indexing ?

Denis McKeon (dmckeon@swcp.com)
Mon, 24 Jun 1996 10:16:48 -0600

In <199606190320.VAA13791@kitsune.swcp.com>,
Denis McKeon <dmckeon@swcp.com> wrote:
>Q: Is there a way to discourage a search engine or web robot from
> indexing a particular html document *within the document itself*?

Briefly, and tentatively (and see the reports below):

<META NAME="ROBOTS" CONTENT="NOINDEX">

Thanks to Nick Arnett's and Martijn Koster's pointers,
the interim answers to this and other questions may be found at:

<A HREF="http://www.w3.org/pub/WWW/Search/9605-Indexing-Workshop/ReportOutcomes/S6Group2.html">S6Group2</A> and <br>
<A HREF="http://www.w3.org/pub/WWW/Search/9605-Indexing-Workshop/ReportOutcomes/Spidering.txt">Spidering.txt</A>

and readers of this list might also want to look at

<A HREF="http://www.kollar.com/robots.html">
Robot Exclusion Standard Revisited</A>

> to encourage users to enter a page hierarchy through a top page
> although that suggests a different directive:
> "index this page, but point users at the top page first"

<META NAME="URL" CONTENT="absolute url">

and again, see the reports, above.

-- 
Denis McKeon 
dmckeon@swcp.com