Re: in-document directive to discourage indexing ?

Martijn Koster (m.koster@webcrawler.com)
Wed, 19 Jun 1996 20:02:20 -0700

At 9:20 PM 6/18/96, Denis McKeon wrote:
>Q: Is there a way to discourage a search engine or web robot from
> indexing a particular html document *within the document itself*?

Well actually, yes, there is a proposal, to be published as part of
the Boston W3C workshop on resource discovery, RSN.

The idea is to do (from memory):

<meta name="robots" content="noindex">

A more complete spec will be available of course :-)

Of the big services at least Lycos, WebCrawler, Infoseek and Excite
have indicated they want to support this.

At WebCrawler we will support this RSN; I put the code in last week,
but it is not yet deployed.

-- Martijn

Email: m.koster@webcrawler.com
WWW: http://info.webcrawler.com/mak/mak.html