Re: we should help spiders and not say NO!

• Messages sorted by: [ date ][ thread ][ subject ][ author ]
• Next message: Scott 'Webster' Wood: "HTML query to .ps?"
• Previous message: Ann Cantelow: "RE: alta vista and virtualvin.com"
• Maybe in reply to: Daniel Lo: "we should help spiders and not say NO!"
• Next in thread: John D. Pritchard: "robot clusion; was Re: we should help spiders and not say NO!"
• Reply: John D. Pritchard: "robot clusion; was Re: we should help spiders and not say NO!"

we want so and so bot, and don't go here or there!

why don't we put in the standard a line
that says.

index-request: 02/20/1996 2 http://www.servonet.com/new-link.html
index-request: 06/31/1996 3 http://www.servonet.com/

index-request: last_time_updated go_2_levels_deep start_indexing_here!

which tells bots where we would like them to index.

- daniel

• Next message: Scott 'Webster' Wood: "HTML query to .ps?"
• Previous message: Ann Cantelow: "RE: alta vista and virtualvin.com"
• Maybe in reply to: Daniel Lo: "we should help spiders and not say NO!"
• Next in thread: John D. Pritchard: "robot clusion; was Re: we should help spiders and not say NO!"
• Reply: John D. Pritchard: "robot clusion; was Re: we should help spiders and not say NO!"