Robots Mailing List Archive by author

• About this archive
• Messages sorted by: [ date ][ thread ][ subject ]
• Other mail archives

Starting: Wed 00 Jan 1970 - 16:31:48 PDT
Ending: Thu 18 Dec 1997 - 14:33:60 PDT
Messages: 2106

• Re[3]: SpamBots Thu, 16 Jan 97 14:59:21 -0000
• Re: Game tonight Thu, 26 Dec 96 12:23:52 -0000
• Re: Standard? Mon, 2 Dec 96 09:03:47 -0000
• Re: Standard? Mon, 2 Dec 96 09:03:44 -0000
• Re: make people use ROBOTS.txt? Sat, 23 Nov 96 22:08:05 -0000
• Re: nastygram from xxx.lanl.gov Thu, 11 Jul 96 13:12:29 -0400
• That wacky Wobot Mon, 20 May 1996 17:13:26 -0400
• Re: Defenses against bad robots Sun, 19 May 1996 16:12:41 -0700 (PDT)
• cc:Mail SMTPLINK Undeliverable Message Thu, 02 May 96 14:27:04 JST
• Re: word spam Wed, 17 Apr 96 09:02:46 -0700
• Re: word spam Tue, 16 Apr 96 21:26:45 -0700
• Re: VB and robot development Mon, 8 Apr 96 13:33:36 +0100
• Re: Image Maps Tue, 26 Mar 1996 20:44:05 -0800 (PST)
• Re: Altavista indexing password files Wed, 28 Feb 96 22:36:35 -0800
• Re: Robot Databases Sun, 18 Feb 96 15:47:28 -0800
• Re: Alta Vista searches WHAT?!? Tue, 16 Jan 96 14:47:51 -0800
• Horror story Sun, 14 Jan 96 10:46:32 -0800
• Re: Does anyone else consider... Thu, 11 Jan 96 07:57:21 -0800
• Re: Does anyone else consider... Wed, 10 Jan 96 19:53:28 -0800
• Re: Does anyone else consider this irresponsible? Wed, 10 Jan 96 13:21:06 -0800
• Re: Does anyone else consider this irresponsible? Wed, 10 Jan 96 09:32:44 -0800
• Re: unknown robot Fri, 29 Dec 95 09:43:17 -0800
• BOUNCE robots: Admin request Wed, 8 Nov 95 10:08:51 -0800
• Dinesh
  • robots: lycos's t-rex: strange behaviour Wed, 12 Feb 1997 11:34:48 +0530 (IST)
• (no name)
  • Robot Mirror with Username/Password feature 2 Jul 96 12:04:12
  • libww and robot source for Sequent Dynix/Ptx 4.1.3 13 Mar 96 10:40:35 EDT
  • unknown robot 29 Dec 95 9:04:46 EDT
• =?ISO-8859-1?Q?Alvaro_Mu=F1oz-Aycuens_Martinez?=
  • Question about Robot.txt Wed, 29 Jan 1997 13:05:33 +0100
  • Re: robot source code Tue, 21 Jan 1997 16:25:26 +0100
  • Meta Tags Tue, 21 Jan 1997 16:42:18 +0100
  • What Is wwweb Tue, 3 Dec 1996 22:59:57 +0100
• =?ISO-8859-1?Q?Jaakko_Hyv=E4tti?=
  • Single tar (Re: Inktomi & large scale spidering) Mon, 27 Jan 1997 10:30:49 +0200 (EET)
  • Re: escaped vs unescaped urls Sat, 18 Jan 1997 14:11:22 +0200 (EET)
  • Re: escaped vs unescaped urls Sat, 18 Jan 1997 14:11:22 +0200 (EET)
  • Re: More Robot Talk Fri, 17 Jan 1997 15:54:45 +0200 (EET)
• A. Shiraz Siddiqui
  • Offline Agents for UNIX Mon, 19 Aug 1996 15:16:18 -0500 (CDT)
• Aaron Nabil
  • Re: ActiveAgent Thu, 7 Nov 1996 14:56:46 -0800 (PST)
  • Tim Freeman Sun, 3 Nov 1996 22:28:23 -0800 (PST)
  • Re: Is a robot visiting? Sun, 3 Nov 1996 01:14:05 -0800 (PST)
  • Re: Is a robot visiting? Fri, 1 Nov 1996 13:39:10 -0800 (PST)
  • What to rate limit/lock on, name or IP address? Tue, 22 Oct 1996 20:00:45 -0700 (PDT)
  • Re: do robots send HTTP_HOST? Mon, 14 Oct 1996 06:33:57 -0700 (PDT)
  • Re: servers that don't return a 404 for "not found" Mon, 14 Oct 1996 06:02:48 -0700 (PDT)
  • Re: servers that don't return a 404 for "not found" Mon, 14 Oct 1996 05:59:43 -0700 (PDT)
  • servers that don't return a 404 for "not found" Mon, 14 Oct 1996 03:46:06 -0700 (PDT)
  • Re: Another rating scam! (And a proposal on how to fix it) Sun, 13 Oct 1996 22:17:19 -0700 (PDT)
  • non 2nn repsonses on robots.txt Sun, 13 Oct 1996 20:12:44 -0700 (PDT)
  • Another rating scam! (And a proposal on how to fix it) Sat, 12 Oct 1996 04:33:31 -0700 (PDT)
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 15:19:28 -0700 (PDT)
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 09:37:37 -0700 (PDT)
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 08:53:57 -0700 (PDT)
  • Re: nastygram from xxx.lanl.gov Tue, 9 Jul 1996 23:17:42 -0700 (PDT)
  • How long to cache robots.txt for? Tue, 09 Jul 1996 13:51:58 -0700
  • Re: nastygram from xxx.lanl.gov Tue, 9 Jul 1996 11:08:52 -0700 (PDT)
  • Re: nastygram from xxx.lanl.gov Tue, 9 Jul 1996 00:24:55 -0700 (PDT)
  • Re: nastygram from xxx.lanl.gov Mon, 8 Jul 1996 23:12:38 -0700 (PDT)
  • nastygram from xxx.lanl.gov Mon, 08 Jul 1996 20:09:25 -0700
• Aaron Stayton
  • New Site Fri, 14 Feb 1997 13:22:31 -0800
  • New Site Fri, 14 Feb 1997 13:22:31 -0800
• Abderrezak Kamel
  • Re: robots on an intranet Tue, 06 Aug 1996 11:53:25 -0700
• ACHAKS
  • Request for Source code in C for Robots Tue, 26 Mar 96 18:13:21 EST
• ACHAKS@inf.com
  • The Internet Archive robot Fri, 06 Sep 96 13:17:42 IST
• Adam Gaffin
  • Belated notice of spider article Thu, 7 Nov 1996 09:29:06 -0400
  • robots on an intranet Mon, 5 Aug 1996 16:44:55 -0400
• Adam Jack
  • Re: Anyone doing a Java-based robot yet? Mon, 19 Feb 1996 17:21:02 -0500
  • Re: Anyone doing a Java-based robot yet? Mon, 19 Feb 1996 16:28:36 -0500
  • Re: Anyone doing a Java-based robot yet? Sun, 18 Feb 1996 14:18:57 -0500
  • Re: robots.txt , authors of robots , webmasters .... Thu, 18 Jan 1996 18:59:12 -0500
  • Re: Alta Vista searches WHAT?!? Tue, 16 Jan 1996 18:01:58 -0500
  • Re: robots.txt extensions Thu, 11 Jan 1996 00:37:14 -0500
  • Re: robots.txt extensions Wed, 10 Jan 1996 19:22:44 -0500
  • robots.txt extensions Wed, 10 Jan 1996 10:35:10 -0500 (EST)
• admin@superhot.com
  • Standard? Fri, 29 Nov 1996 14:28:15 -0700
  • Re: who/what uses robots.txt Fri, 22 Nov 1996 15:15:43 -0700
• Adminstrator
  • Mail failure Thu, 30 Nov 95 04:44:00 PST
  • Mail failure Thu, 30 Nov 95 04:28:00 PST
  • Mail failure Thu, 30 Nov 95 04:15:00 PST
  • Mail failure Thu, 30 Nov 95 03:57:00 PST
• AJAJR@aol.com
  • un-subcribe Fri, 19 Jan 1996 14:30:18 -0500
  • Polite Request #2 to be Removed form List Fri, 19 Jan 1996 08:11:55 -0500
• Alain Desilets
  • Re: Looking for a spider Fri, 1 Dec 1995 12:24:16 -0500
  • Re: Looking for a spider Thu, 30 Nov 1995 13:43:58 -0500
  • Re: Looking for a spider Thu, 19 Oct 95 12:09:49 EDT
  • Sorry! Thu, 19 Oct 95 09:39:13 EDT
  • Re: Looking for a spider Thu, 19 Oct 95 09:31:31 EDT
  • Re: Looking for a spider Thu, 19 Oct 95 09:29:15 EDT
  • Looking for a spider Wed, 18 Oct 95 14:31:39 EDT
• Alan
  • Re: word spam Wed, 17 Apr 1996 02:48:51 +0000
  • Re: word spam Sun, 14 Apr 1996 18:18:17 +0000
  • Re: Handling keyword repetitions Fri, 12 Apr 1996 02:11:07 +0000
  • Re: Handling keyword repetitions Fri, 12 Apr 1996 02:11:07 +0000
• Alex Chapman
  • Re: Robot-HTML Web Page? Mon, 27 May 1996 18:06:11 EST
  • Re: any robots/search-engines which index links? Thu, 9 May 1996 09:25:56 EST
  • any robots/search-engines which index links? Sat, 4 May 1996 12:24:33 EST
• Alex Strasheim
  • Re: The Internet Archive robot Mon, 9 Sep 1996 14:22:02 -0500 (CDT)
  • pointers for a novice? Sat, 7 Sep 1996 16:18:53 -0500 (CDT)
• Alexander Franz
  • [ANNOUNCE] CFP: AAAI-96 WS on Internet-based Information Systems Thu, 18 Jan 96 17:23:19 +0900
• Alison Gwin
  • (no subject) Sat, 13 Jan 1996 09:10:01 -0600
• Alvaro Monge
  • Re: Looking for a spider Wed, 18 Oct 1995 15:00:01 -0700 (PDT)
  • Re: Looking for a spider Wed, 18 Oct 1995 13:13:55 -0700 (PDT)
• Alyne Mochan & Warren Baker
  • Re: Advice Sun, 30 Jun 1996 10:43:54 +0900
  • Re: Advice Sun, 30 Jun 1996 10:18:53 +0900
  • Advice Sat, 29 Jun 1996 08:59:29 +0900
• Andrew Daviel
  • wwwbot.pl problem Thu, 23 Nov 1995 12:42:51 -0800 (PST)
  • Re: Proposed URLs that robots should search Tue, 24 Oct 1995 02:48:19 -0700 (PDT)
• Andrew Leonard
  • Re: More ways to spam search engines? Tue, 10 Sep 1996 09:09:45 -0700 (PDT)
  • Re: Unfriendly robot at 205.252.60.50 Sat, 21 Oct 1995 07:17:03 -0700 (PDT)
  • Re: Unfriendly robot owner identified! Wed, 18 Oct 1995 15:18:53 -0700 (PDT)
• Andrey A. Krasov
  • ??: reload problem Thu, 23 May 1996 15:34:44 +0700
  • Quiz playing robots ? Wed, 24 Apr 1996 17:21:55 +0700
  • Re: word spam Thu, 18 Apr 1996 15:51:32 +0700
• Andy Rollins
  • Re: email grabber Thu, 16 Jan 1997 10:10:24 -0600
• Andy Warner
  • hoohoo.cac.washington = bad Sun, 28 Jul 1996 15:58:52 +0000
  • Re: Try robot... Thu, 25 Apr 1996 00:12:56 +0000
  • Clean up Bots... Thu, 4 Apr 1996 14:23:45 +0000
• andy@andy.net
  • Re: Inktomi & large scale spidering Sun, 26 Jan 1997 00:52:09 +0000
  • Re: ActiveAgent Fri, 8 Nov 1996 14:38:37 +0000
  • Re: ActiveAgent Fri, 8 Nov 1996 13:51:03 +0000
• Ann Cantelow
  • RE: alta vista and virtualvin.com Fri, 28 Jun 1996 11:47:42 -0600 (MDT)
  • Re: RCPT: Re: Introducing myself Fri, 21 Jun 1996 12:45:58 -0600 (MDT)
  • Re: BackRub robot Tue, 18 Jun 1996 08:12:19 -0600 (MDT)
  • RE: alta vista and virtualvin.com Sun, 2 Jun 1996 00:11:15 -0600 (MDT)
• Anna Torti
  • SetEnv a problem Wed, 20 Nov 1996 12:01:21 -0100
  • HEAD Thu, 5 Sep 1996 14:11:28 -0100
• anna@nerve.itim.mi.cnr.it
  • Re: robots, what else! Mon, 10 Jun 1996 09:32:58 +0200
• Anne Greene
  • FS-Jan 7 Row 1 Sec 211 Thu, 26 Dec 1996 10:42:17 -0800
• Anthony D. Thomas
  • RE: Introducing myself Tue, 18 Jun 1996 11:13:56 -0400
• Anthony John Carmody
  • Dead account Tue, 25 Jun 1996 11:51:45 +1000
• Art Matheny
  • Re: Java and robots... Thu, 13 Feb 1997 17:14:20 -0500 (EST)
  • Re: email address grabber Wed, 15 Jan 1997 15:16:27 -0500 (EST)
  • Re: email address grabber Tue, 14 Jan 1997 15:37:42 -0500 (EST)
  • Re: Servers vs Agents Sat, 30 Nov 1996 13:36:43 -0500 (EST)
  • Re: java applet sockets Sat, 23 Nov 1996 19:00:33 -0500 (EST)
  • Re: defining "robot" Thu, 21 Nov 1996 23:03:27 -0500 (EST)
  • Re: Broadness of Robots.txt (Re: Washington again !!!) Thu, 21 Nov 1996 11:54:05 -0500 (EST)
  • Re: An updated extended standard for robots.txt Wed, 13 Nov 1996 10:07:26 -0500 (EST)
  • Re: An extended version of the Robots... Tue, 12 Nov 1996 17:27:42 -0500 (EST)
  • Re: An updated extended standard for robots.txt Tue, 12 Nov 1996 15:56:14 -0500 (EST)
• arutgers
  • Re: in-document directive to discourage indexing ? Fri, 21 Jun 1996 15:01:44 -0700
  • Re: word spam Sat, 13 Apr 1996 15:46:29 -0700
• autoresponder@WhiteHouse.gov
  • Re: message to USSA Senate Fri, 14 Feb 1997 14:42:17 -0500 (EST)
  • Re: message to USSA House of Representatives Fri, 14 Feb 1997 12:44:23 -0500 (EST)
• Bakin, David
  • RE: alta vista and virtualvin.com Mon, 03 Jun 96 20:11:00 PDT
• Baron Timothy de Vallee
  • Re: robot ? Tue, 03 Dec 1996 09:34:05 +0000
• Barry A. Dobyns
  • Re: FILEZ Tue, 11 Feb 1997 08:15:49 -0800
• bbh@xenodata.com
  • FILEZ Mon, 10 Feb 1997 20:45:07 -0600
  • Re: Crawling & DNS issues Wed, 22 Jan 1997 12:30:40 -0600
  • Re: More Robot Talk Fri, 17 Jan 1997 13:29:37 -0600
  • Re: More Robot Talk (was Re: email grabber) Fri, 17 Jan 1997 02:02:42 -0600
  • Re: hipcrime Mon, 25 Nov 1996 23:49:57 -0600
  • Re: What to rate limit/lock on, name or IP address? Wed, 23 Oct 1996 10:41:58 -0500
• Benjamin Franz
  • Re: More with the Cypherpunk antics Sat, 15 Feb 1997 04:40:31 -0800 (PST)
  • Re: indexing via redirectors Tue, 21 Jan 1997 07:28:31 -0800 (PST)
  • Re: It's not only robots we have to worry about ... Thu, 26 Dec 1996 07:50:36 -0800 (PST)
  • [...]Re: Cache Filler Fri, 29 Nov 1996 08:34:40 -0800 (PST)
  • Re: make people use ROBOTS.txt? Sat, 23 Nov 1996 06:01:17 -0800 (PST)
  • Re: ActiveAgent Fri, 8 Nov 1996 15:57:36 -0800 (PST)
  • Re: ActiveAgent Fri, 8 Nov 1996 12:14:56 -0800 (PST)
  • Re: Domains and HTTP_HOST Thu, 7 Nov 1996 09:57:00 -0800 (PST)
  • Re: ActiveAgent Mon, 14 Oct 1996 10:10:36 -0700 (PDT)
  • Re: META tag standards, search accuracy Mon, 14 Oct 1996 08:13:35 -0700 (PDT)
  • Re: META tag standards, search accuracy Mon, 14 Oct 1996 07:56:41 -0700 (PDT)
  • Re: servers that don't return a 404 for "not found" Mon, 14 Oct 1996 05:25:06 -0700 (PDT)
  • Re: Another rating scam! (And a proposal on how to fix it) Mon, 14 Oct 1996 04:56:53 -0700 (PDT)
  • Re: ActiveAgent Mon, 14 Oct 1996 04:54:33 -0700 (PDT)
  • Re: ActiveAgent Mon, 14 Oct 1996 04:48:00 -0700 (PDT)
  • Re: META tag standards, search accuracy Sun, 13 Oct 1996 12:51:36 -0700 (PDT)
  • Re: Another rating scam! (And a proposal on how to fix it) Sun, 13 Oct 1996 06:07:21 -0700 (PDT)
  • Bad agent...A *very* bad agent. Wed, 18 Sep 1996 10:45:22 -0700 (PDT)
  • Re: Copyrights on the web Mon, 9 Sep 1996 19:52:26 -0700 (PDT)
  • Re: Anyone know who owns this one? Thu, 1 Aug 1996 19:02:33 -0700 (PDT)
  • Re: the POST myth... a web admin's opinions.. Thu, 18 Jul 1996 15:08:36 -0700 (PDT)
  • Re: Safe Methods Thu, 18 Jul 1996 13:26:39 -0700 (PDT)
  • Re: the POST myth... a web admin's opinions.. Thu, 18 Jul 1996 10:29:52 -0700 (PDT)
  • Re: Safe Methods Thu, 18 Jul 1996 07:15:48 -0700 (PDT)
  • Re: PS Mon, 15 Jul 1996 08:23:44 -0700 (PDT)
  • Re: interactive generation of URL's Mon, 15 Jul 1996 08:18:58 -0700 (PDT)
  • Re: Suggestion to help robots and sites coexist a little better Mon, 15 Jul 1996 08:08:33 -0700 (PDT)
  • Re: the POST myth... a web admin's opinions.. Fri, 12 Jul 1996 07:29:26 -0700 (PDT)
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 10:27:31 -0700 (PDT)
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 07:12:46 -0700 (PDT)
  • Re: in-document directive to discourage indexing ? Mon, 24 Jun 1996 12:36:49 -0700 (PDT)
  • Re: Client Robot 'Ranjan' Thu, 20 Jun 1996 12:34:51 -0700 (PDT)
  • Re: Defenses against bad robots Sat, 18 May 1996 07:43:01 -0700 (PDT)
  • Re: Defenses against bad robots Fri, 17 May 1996 17:21:03 -0700 (PDT)
  • Re: alta vista and virtualvin.com Fri, 17 May 1996 17:07:46 -0700 (PDT)
  • Re: Web spaces of strange topology. Where? Fri, 17 May 1996 07:29:13 -0700 (PDT)
  • Re: any robots/search-engines which index links? Wed, 8 May 1996 14:39:01 -0700 (PDT)
  • Re: alta vista and virtualvin.com Mon, 6 May 1996 06:53:11 -0700 (PDT)
  • Re: Political economy of distributed search (was topical search...) Fri, 3 May 1996 05:21:35 -0700 (PDT)
  • Re: word spam Tue, 16 Apr 1996 06:21:40 -0700 (PDT)
  • Re: word spam Mon, 15 Apr 1996 08:14:05 -0700 (PDT)
  • Re: Search accuracy Fri, 5 Apr 1996 06:16:16 -0800 (PST)
  • Re: Search accuracy Mon, 1 Apr 1996 18:01:11 -0800 (PST)
  • Re: Search accuracy Mon, 1 Apr 1996 15:13:30 -0800 (PST)
  • Re: Links (don't bother checking; I've done it for you) Fri, 29 Mar 1996 10:53:28 -0800 (PST)
  • Re: image map traversal Mon, 25 Mar 1996 09:37:27 -0800 (PST)
  • Re: robots.txt , authors of robots , webmasters .... Thu, 18 Jan 1996 09:03:56 -0800 (PST)
  • Re: robots.txt , authors of robots , webmasters .... Wed, 17 Jan 1996 10:59:43 -0800 (PST)
  • Re: Does this count as a robot? Mon, 8 Jan 1996 08:09:24 -0800 (PST)
  • Re: Does this count as a robot? Mon, 8 Jan 1996 08:09:24 -0800 (PST)
  • Bad robot: WebHopper bounch! Owner: peter@cartes.hut.fi Fri, 17 Nov 1995 09:24:00 -0800 (PST)
• Betsy Dunphy
  • Re: ActiveAgent Mon, 14 Oct 96 15:07:04 -0400
  • Re: ActiveAgent Mon, 14 Oct 96 12:59:54 -0400
  • Re: ActiveAgent Mon, 14 Oct 96 11:09:24 -0400
  • Re: I vote NO (Was: Robot Gripes forum?) - I vote YES Sat, 3 Aug 96 09:03:21 -0400
  • Robot Gripes forum? (Was: Anyone know who owns this one?) Fri, 2 Aug 96 13:48:08 -0400
  • Anyone know who owns this one? Thu, 1 Aug 96 19:01:50 -0400
  • Re: hoohoo.cac.washington = bad Mon, 29 Jul 96 10:36:57 -0400
• Bhupinder S. Sran
  • What is your favorite search engine - a survey Sun, 4 Aug 1996 15:17:17 GMT
  • Looking for a search engine Mon, 29 Apr 1996 18:50:01 -0400
  • Robot Research Thu, 18 Jan 1996 10:36:33 GMT
• Bill Day
  • Re: Re: Thu, 25 Apr 1996 08:41:43 -0500
• Bj\xrn-Olav Strand
  • RE: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 10:56:25 +0200 (MET DST)
  • RE: nastygram from xxx.lanl.gov Tue, 9 Jul 1996 12:50:44 +0200 (MET DST)
  • Re: RCPT: Re: Introducing myself Fri, 21 Jun 1996 11:58:16 +0200 (MET DST)
• Bjorn-Olav Strand
  • Re: [2]RE>[5]RE>Checking Log fi Sat, 16 Dec 1995 14:47:55 +0100 (MET)
• blea@hic.net
  • Re: Mail robot? Tue, 27 Aug 1996 23:02:38 -0500
  • (no subject) Fri, 9 Aug 1996 22:07:30 -0500
  • Re: Polite Request #2 to be Removed form List Sat, 20 Jan 1996 09:21:16 -0600
• Bob Worthy
  • Re: Looking for subcontracting spider-programmers Fri, 06 Sep 1996 14:38:34 -0600
• Bonnie
  • How to get listed #1 on all search engines (fwd) Mon, 30 Sep 1996 21:24:11 -2800 (EDT)
• Bonnie Scott
  • Quick--who knows listproc? Thu, 26 Dec 1996 15:06:58 -0500 (EST)
  • Apology -- I didn't mean to send that last message to the list Thu, 18 Jul 1996 18:02:09 -0400 (EDT)
  • Re: the POST myth... a web admin's opinions.. Thu, 18 Jul 1996 17:52:31 -0400 (EDT)
  • Re: psycho at xxx.lanl.gov Tue, 9 Jul 1996 18:02:51 -0400 (EDT)
  • Re: communal bots Fri, 26 Apr 1996 13:54:20 -0400 (EDT)
  • Re: Robots in the client? Fri, 19 Apr 1996 16:03:18 -0400 (EDT)
  • Any info on "E-mail America"? Fri, 26 Jan 1996 12:16:41 -0400
  • Re: [2]RE>[5]RE>Checking Lo Fri, 15 Dec 1995 15:00:35 -0400
• boogieoogie goobnie
  • Allow/deny robots from major search services Mon, 17 Jun 96 21:51:20 PDT
  • Robot books Mon, 17 Jun 96 21:07:07 PDT
• Brad Fox
  • Re[2]: SpamBots Thu, 16 Jan 1997 12:37:57 -0500
• Brenden Portolese
  • Re: Client Robot 'Ranjan' Thu, 20 Jun 1996 12:44:40 -0700
  • Client Robot 'Ranjan' Wed, 19 Jun 1996 11:30:08 -0700
  • looking for specific bot... Fri, 14 Jun 1996 09:18:20 -0700
• Brent Boghosian
  • RE: What to rate limit/lock on, name or IP address? Wed, 23 Oct 1996 16:04:33 -0400 (EDT)
  • Re: robots.txt buffer question. Tue, 22 Oct 1996 21:29:08 -0400 (EDT)
  • Re: robots.txt syntax Fri, 18 Oct 1996 01:26:56 -0400 (EDT)
• Brewster Kahle
  • Re: The Internet Archive robot (fwd) Wed, 11 Sep 1996 17:05:46 -0700
  • RE: The Internet Archive robot (fwd) Fri, 06 Sep 1996 20:41:38 -0700
• Brian Clark
  • Re: Web pages being served from an SQL database Tue, 17 Dec 96 16:33:08 -0500
  • Re: Junkie-Mail was Re: Standard? Tue, 03 Dec 96 08:47:51 -0500
  • Re: Junkie-Mail was Re: Standard? Mon, 02 Dec 96 14:16:15 -0500
  • Re: Standard? Thu, 28 Nov 96 23:10:54 -0500
  • Re: defining "robot" Tue, 26 Nov 96 08:16:16 -0500
  • Re: defining "robot" Mon, 25 Nov 96 16:59:31 -0500
  • Re: Broadness of Robots.txt (Re: Washington again !!!) Wed, 20 Nov 96 21:21:17 -0500
  • Re: Broadness of Robots.txt (Re: Washington again !!!) Wed, 20 Nov 96 21:21:10 -0500
  • Broadness of Robots.txt (Re: Washington again !!!) Wed, 20 Nov 96 09:34:16 -0500
  • Re: Domains and HTTP_HOST Wed, 06 Nov 96 22:18:17 -0500
  • Domains and HTTP_HOST Wed, 06 Nov 96 09:24:14 -0500
  • Disallow/Allow by Action (Re: robots.txt syntax) Fri, 18 Oct 96 08:14:46 -0500
  • Re: ActiveAgent Mon, 14 Oct 96 08:43:38 -0500
  • Re: The End of The World (Wide Web) / Part II Thu, 03 Oct 96 20:01:39 -0500
  • Re: Copyrights, let them be ! Wed, 11 Sep 96 13:33:55 -0500
  • Re: Copyrights (was Re: The Internet Archive robot) Wed, 11 Sep 96 12:04:44 -0500
  • Re: Copyrights (was Re: The Internet Archive robot) Tue, 10 Sep 96 17:13:15 -0500
  • Copyrights (was Re: The Internet Archive robot) Mon, 09 Sep 96 17:52:01 -0500
  • Re: The Internet Archive robot Sun, 08 Sep 96 08:08:04 -0500
  • Re: Suggestion to help robots and sites coexist a little better Thu, 18 Jul 96 08:49:29 -0500
  • Re: PS Mon, 15 Jul 96 15:52:40 -0500
  • Re: roverbot - perhaps the worst robot yet Thu, 27 Jun 96 17:05:26 -0500
  • Re: robots.txt usage Tue, 18 Jun 96 12:21:22 -0500
  • Re: Web spaces of strange topology. Where? Fri, 17 May 96 13:10:22 -0500
• Brian Fitzgerald
  • topical search tool -- help?! Wed, 1 May 1996 17:09:22 -0400 (EDT)
• Brian Hancock
  • robot.polite Fri, 14 Jun 1996 22:16:46 -0230 (NDT)
• Brian Pinkerton
  • Re: WebCrawler & Excite Wed, 4 Dec 96 09:27:19 -0800
  • Re: Horror story Sun, 14 Jan 96 22:40:09 -0800
• Brian Ulicny
  • Re: Keyword indexing Wed, 19 Jun 96 01:16:06 -0700 (PDT)
  • Re: Keyword indexing Tue, 18 Jun 96 17:29:01 -0700 (PDT)
  • Re: Political economy of distributed search (was topical Mon, 6 May 96 19:18:55 -0700 (PDT)
  • Re: topical search tool -- help?! Thu, 2 May 96 22:05:05 -0700 (PDT)
  • Re: topical search tool -- help?! Thu, 2 May 96 18:05:28 -0700 (PDT)
  • Re: topical search tool -- help?! Thu, 2 May 96 17:56:55 -0700 (PDT)
  • Re: topical search tool -- help?! Thu, 2 May 96 16:57:51 -0700 (PDT)
  • Re: topical search tool -- help?! Thu, 2 May 96 02:37:04 -0700 (PDT)
  • Re: topical search tool -- help?! Wed, 1 May 96 22:46:19 -0700 (PDT)
  • Re: About integrated search engines Thu, 25 Apr 96 21:54:41 -0700 (PDT)
• Bruce Rhodewalt
  • Re: Robot Gripes forum? (Was: Anyone know who owns this one?) Fri, 02 Aug 1996 14:27:43 -0700
• Bryan Cort
  • RE: Copyrights on the web Tue, 10 Sep 1996 12:29:39 -0500
• Bryan Cromartie
  • RE: ActiveAgent and E-Mail Spam Mon, 14 Oct 1996 13:31:53 -0500
• Byung-Gyu Chang
  • Re: Please Help ME!! Wed, 3 Jan 1996 22:57:23 +0900 (KST)
  • Wobot? Fri, 15 Dec 1995 21:54:17 +0900 (KST)
  • Q: Cooperation of robots Sat, 25 Nov 1995 23:19:12 +0900 (KST)
• Cafe
  • Cannot believe it "Morons" Tue, 01 Oct 1996 18:14:02 +1000
• Can Ozturan
  • Re: "hidden text" vs. META tags for robots/search engines Fri, 06 Sep 1996 13:58:20 -0400
• Captain Napalm
  • More with the Cypherpunk antics Wed, 12 Feb 1997 23:54:10 -0500 (EST)
  • Re: indexing via redirectors Tue, 21 Jan 1997 15:47:22 -0500 (EST)
  • Re: More Robot Talk (was Re: email grabber) Fri, 17 Jan 1997 03:19:19 -0500 (EST)
  • More Robot Talk (was Re: email grabber) Thu, 16 Jan 1997 22:26:06 -0500 (EST)
  • Re: email address grabber Wed, 15 Jan 1997 15:10:56 -0500 (EST)
  • Re: email address grabber Wed, 15 Jan 1997 14:32:08 -0500 (EST)
  • It's not only robots we have to worry about ... Thu, 26 Dec 1996 03:22:27 -0500 (EST)
  • Re: Standard? Thu, 28 Nov 1996 23:31:12 -0500 (EST)
  • Re: user-agent in Java Mon, 25 Nov 1996 14:40:13 -0500 (EST)
  • Re: ActiveAgent Sat, 23 Nov 1996 14:48:22 -0500 (EST)
  • Re: Broadness of Robots.txt (Re: Washington again !!!) Wed, 20 Nov 1996 18:50:43 -0500 (EST)
  • Re: Broadness of Robots.txt (Re: Washington again !!!) Wed, 20 Nov 1996 18:41:19 -0500 (EST)
  • Re: RFC, draft 1 Tue, 19 Nov 1996 16:05:13 -0500 (EST)
  • Re: Getting a Reply-to: field ... Sat, 16 Nov 1996 01:05:36 -0500 (EST)
  • Getting a Reply-to: field ... Fri, 15 Nov 1996 23:18:39 -0500 (EST)
  • Re: RFC, draft 1 Fri, 15 Nov 1996 23:15:35 -0500 (EST)
  • Re: An updated extended standard for robots.txt Tue, 12 Nov 1996 15:53:28 -0500 (EST)
  • An updated extended standard for robots.txt Mon, 11 Nov 1996 18:29:35 -0500 (EST)
  • Re: An extended version of the Robots... Mon, 11 Nov 1996 16:07:32 -0500 (EST)
  • Re: An extended version of the Robots... Mon, 11 Nov 1996 14:37:35 -0500 (EST)
  • Re: An extended verion of the robot exclusion standard Mon, 11 Nov 1996 14:29:59 -0500 (EST)
  • Re: An extended version of the Robots... Sat, 9 Nov 1996 02:32:19 -0500 (EST)
  • An extended verion of the robot exclusion standard Fri, 8 Nov 1996 14:22:35 -0500 (EST)
  • Re: robots.txt syntax Thu, 17 Oct 1996 23:45:55 -0400 (EDT)
  • Re: agents ignoring robots.txt Wed, 16 Oct 1996 17:51:11 -0400 (EDT)
  • Re: robots.txt syntax Mon, 14 Oct 1996 12:01:54 -0400 (EDT)
  • Re: robots.txt syntax Mon, 14 Oct 1996 09:56:42 -0400 (EDT)
  • Re: robots.txt syntax Sat, 12 Oct 1996 15:40:30 -0400 (EDT)
  • Re: HEAD Sat, 7 Sep 1996 00:16:11 -0400 (EDT)
  • Re: HEAD Sat, 7 Sep 1996 00:16:11 -0400 (EDT)
  • Re: Anyone know who owns this one? Fri, 2 Aug 1996 06:36:12 -0400 (EDT)
  • Re: Anyone know who owns this one? Fri, 2 Aug 1996 06:36:12 -0400 (EDT)
• Carlos Baquero
  • Re: any robots/search-engines which index links? Sun, 12 May 1996 22:40:48 +0100 (GMT+0100)
  • Re: any robots/search-engines which index links? Sun, 12 May 1996 22:40:48 +0100 (GMT+0100)
  • Re: any robots/search-engines which index links? Wed, 8 May 1996 18:06:27 +0000
  • Re: verify URL Wed, 13 Mar 1996 18:58:02 +0100 (GMT+0100)
  • Re: robots.txt , authors of robots , webmasters .... Thu, 18 Jan 1996 22:37:34 +0100 (GMT+0100)
  • Re: Inter-robot Comms Port Wed, 27 Dec 1995 01:45:49 +0100 (GMT+0100)
• Carlos Horowicz
  • Re: [...]Re: Cache Filler Mon, 2 Dec 1996 12:48:12 -0300 (ARG)
• Cees Hek
  • Re: image map traversal Mon, 25 Mar 1996 13:23:03 -0600 (CST)
  • Re: Checking Log files Mon, 18 Dec 1995 10:42:39 -0600 (CST)
  • Checking Log files Thu, 14 Dec 1995 14:32:43 -0600 (CST)
• Chad Zimmerman
  • Re: More with the Cypherpunk antics Thu, 13 Feb 1997 13:19:19 -0700 (MST)
  • Possible robot? Tue, 1 Oct 1996 10:35:52 -0600 (MDT)
  • Re: "hidden text" vs. META tags for robots/search engines Sat, 14 Sep 1996 10:02:05 -0600 (MDT)
  • Re: Copyrights on the web Tue, 10 Sep 1996 19:33:16 -0600 (MDT)
• Charles P. Kollar
  • Robot Exclusion Standard Revisited Thu, 30 May 1996 11:30:15 -0400 (EDT)
• Charlie Brown
  • Copyrights on the web Mon, 09 Sep 1996 20:43:41 -0500
  • Re: Perl Spiders Thu, 8 Feb 1996 19:45:31 -0600 (CST)
  • Wanted: Web Robot code - C/Perl Wed, 07 Feb 1996 21:44:02 -0600
• Chiaki Ohta
  • mini-robot Tue, 20 Aug 1996 18:52:42 JST
• Chris Brown
  • Re: email grabber Thu, 16 Jan 1997 10:05:10 +0000
  • Re: Links (don't bother checking; I've done it for you) Fri, 29 Mar 1996 13:01:18 +0000
• chris cobb
  • RE: Client Robot 'Ranjan' Thu, 20 Jun 1996 14:39:04 -0700
  • RE: VB. page grabber... Mon, 6 May 1996 14:24:01 -0400
  • alta vista and virtualvin.com Mon, 6 May 1996 00:56:11 -0400
  • RE: To: ???? Robot Sun, 28 Apr 1996 01:52:23 -0400
  • word spam Fri, 12 Apr 1996 00:22:55 -0400
• Chris Crowther
  • Re: interactive generation of URL's Mon, 15 Jul 1996 21:25:20 +0000
  • Re: nastygram from xxx.lanl.gov Sat, 13 Jul 1996 20:23:10 +0000
  • xxx.lanl.gov/robots.txt Sat, 13 Jul 1996 09:55:05 +0000
  • xxx.lanl.gov - The thread continues.... Fri, 12 Jul 1996 08:52:00 +0000
  • Re: nastygram from xxx.lanl.gov Thu, 11 Jul 1996 10:19:40 +0000
  • Re: nastygram from xxx.lanl.gov Thu, 11 Jul 1996 10:19:40 +0000
  • Re: Should I index all ... Tue, 9 Jul 1996 23:53:59 +0000
  • Re: nastygram from xxx.lanl.gov Tue, 9 Jul 1996 23:44:11 +0000
  • Re: nastygram from xxx.lanl.gov Tue, 9 Jul 1996 15:40:24 +0000
  • Re: Should I index all ... Tue, 9 Jul 1996 14:00:18 +0000
  • Re: RCPT: Re: Introducing myself Sat, 22 Jun 1996 23:27:46 +0000
  • Re: RCPT: Re: Introducing myself Sat, 22 Jun 1996 23:27:46 +0000
• christophe grandjacquet
  • unsubscibe Sat, 30 Mar 1996 17:26:19 +0000
  • unscribe Sat, 30 Mar 1996 17:24:36 +0000
• Christopher J. Farrell IV
  • Mail robot? Mon, 05 Aug 1996 23:26:21 -0400
• Christopher Penrose
  • Here is WebWalker Fri, 9 Feb 96 08:58:47 -0800
  • Re: Perl Spiders Thu, 8 Feb 96 16:31:58 -0800
  • Perl Spiders Thu, 8 Feb 96 08:43:59 -0800
  • Re: Server name in /robots.txt Fri, 19 Jan 96 10:21:09 -0800
  • McKinley Spider hit us hard Wed, 29 Nov 95 18:15:27 -0800
• Chuck Doucette
  • Limiting robots to top-level page only (via robots.txt)? Tue, 26 Mar 1996 00:49:04 -0500
• Cleder Catherine
  • Re: Should I index all ... Mon, 08 Jul 1996 10:34:14 +0200
  • Should I index all ... Thu, 4 Jul 1996 15:31:46 +0200
• clonez1@one.net
  • Cyclones sign MacLeod Thu, 26 Dec 1996 12:15:00 -0800
• Colin Goodier
  • Re: (Fwd) Re: Search accuracy Fri, 5 Apr 96 16:54 BST-1
• Convegno NATO HPC 1996
  • Re: in-document directive to discourage indexing ? Tue, 25 Jun 1996 20:40:06 +0200
• Craig McQueen
  • RE: Looking for... Fri, 21 Jun 1996 13:59:52 -0400
  • RE: WebAnalyzer - introduction Tue, 18 Jun 1996 16:42:43 -0400
  • WebAnalyzer - introduction Tue, 18 Jun 1996 09:00:39 -0400
• Cristian Ionitoiu
  • inquiry about robots Fri, 1 Dec 1995 17:20:47 +0200 (EET)
• D. Williams
  • Test server for robot development? Tue, 04 Jun 1996 07:23:46 -0600
  • A new robot...TOPjobs(tm) USA JOBbot 1.0a Tue, 04 Jun 1996 07:03:10 -0600
• DA
  • Re: Q: size of the web in bytes, comprehensive list Sat, 18 Jan 1997 16:35:04 -0400 (AST)
  • Re: Junkie-Mail was Re: Standard? Mon, 2 Dec 1996 16:25:19 -0400 (AST)
  • Re: infoseeks robot is dumb Fri, 15 Nov 1996 21:28:07 -0400 (AST)
  • Re: infoseeks robot is dumb Fri, 15 Nov 1996 18:41:34 -0400 (AST)
  • Re: databases for spiders Thu, 14 Nov 1996 19:40:00 -0400 (AST)
  • Re: changes to robots.txt Tue, 12 Nov 1996 21:33:47 -0400 (AST)
  • Re: changes to robots.txt Tue, 12 Nov 1996 14:15:15 -0400 (AST)
  • Re: Is their a web site... Mon, 11 Nov 1996 16:52:42 -0400 (AST)
  • Re: Get official! Mon, 11 Nov 1996 14:43:35 -0400 (AST)
• Dan Gildor
  • robot meta tags Tue, 21 Jan 1997 18:42:31 -0800
  • escaped vs unescaped urls Fri, 17 Jan 1997 16:23:43 -0800
  • IIS and If-modified-since [was Re: stingy yahoo server?] Mon, 9 Dec 1996 08:25:02 -0800 (PST)
  • Re: stingy yahoo server? Sun, 8 Dec 1996 09:06:16 -0800 (PST)
  • robots that index comments Tue, 26 Mar 1996 10:28:25 -0800
  • Re: robot authentication parameters Wed, 13 Mar 1996 18:16:31 -0800
• Dan Howard
  • Re: robot source code Tue, 21 Jan 1997 17:02:11 -0800
• Dan Hurwitz
  • Newbie question Mon, 15 Jul 1996 09:29:32 +0100
• Dan Quigley
  • RE: How can IR Agents be evaluate ? Sat, 10 Aug 1996 14:56:33 -0700
• Daniel C Grigsby
  • Re: Search accuracy Thu, 4 Apr 1996 16:50:08 -0600 (CST)
• Daniel Lo
  • Re: HEAD Thu, 5 Sep 1996 15:40:54 -0700
  • Re: we should help spiders and not say NO! Fri, 28 Jun 1996 13:49:25 -0700
  • we should help spiders and not say NO! Fri, 28 Jun 1996 09:28:16 -0700
  • Re: robots.txt usage Tue, 18 Jun 1996 07:43:24 -0700
• Daniel T. Martin
  • Re: Is a robot visiting? Fri, 01 Nov 1996 08:31:54 -0600 (CST)
  • Re: AltaVista's Index is obsolete; but what about the other Mon, 22 Jul 1996 06:56:04 -0500 (CDT)
  • Re: How long to cache robots.txt Thu, 18 Jul 1996 20:44:24 -0500 (CDT)
  • Re: How long to cache robots.txt Thu, 18 Jul 1996 09:36:11 -0500 (CDT)
  • Re: nastygram from xxx.lanl.gov Tue, 09 Jul 1996 08:04:13 -0500 (CDT)
  • Re: robots.txt unavailability Tue, 09 Jul 1996 07:05:42 -0500 (CDT)
  • robots.txt unavailability Mon, 08 Jul 1996 21:17:59 -0500 (CDT)
  • Identifying identical documents Wed, 03 Jul 1996 14:58:28 -0500 (CDT)
• Daniel Williams
  • Re: RCPT: Re: Introducing myself Fri, 21 Jun 1996 09:41:26 -0600
• Danny Sullivan
  • Re: robots & copyright law Mon, 27 Jan 1997 22:48:29 0000
  • Re: Meta Tags Wed, 22 Jan 1997 01:31:32 0000
  • Re: AltaVista Meta Tag Rumour Tue, 21 Jan 1997 10:41:35 0000
  • Re: "real-time" spidering by Lycos Sat, 18 Jan 1997 23:00:09 0000
  • Re: Newbie question Fri, 19 Jul 1996 12:49:29 -0700
  • Search Engine article Thu, 18 Jul 1996 15:19:39 -0700
• Darren Hardy
  • Re: An extended version of the Robots... Sat, 16 Nov 1996 14:25:56 -0800
  • Re: RFC, draft 1 Sat, 16 Nov 1996 14:15:43 -0800
  • Re: RFC, draft 1 Sat, 16 Nov 1996 13:46:58 -0800
• Darren R. Hardy
  • Re: Inter-robot communication Thu, 20 Jun 1996 20:29:26 -0700
  • URL measurement studies? Fri, 16 Feb 1996 11:33:26 -0800
• Darrin Chandler
  • Re: Robot-HTML Web Page? Thu, 23 May 1996 21:48:29 -0700
  • Re: default documents Sun, 21 Apr 1996 11:59:26 -0700
  • Re: VB and robot development Sun, 7 Apr 1996 23:49:24 -0700
  • Re: Links (don't bother checking; I've done it for you) Fri, 29 Mar 1996 13:51:42 -0700
  • Re: Links (don't bother checking; I've done it for you) Fri, 29 Mar 1996 10:11:37 -0700
  • Re: robots.txt changes how often? Wed, 14 Feb 1996 12:14:16 -0700
• Dave Bakin
  • Just when you thought it might be interesting to standardize Wed, 04 Dec 1996 11:58:29 -0800
• Dave Rothwell
  • McKinley robot Sat, 1 Jun 1996 12:58:18 +0100
• Dave White
  • Re: Keyword indexing Tue, 18 Jun 96 17:11:46 -0700
• David A Weeks
  • Re: Search accuracy Fri, 5 Apr 1996 02:00:15 +0100 (BST)
  • Re: Anyone doing a Java-based robot yet? Tue, 20 Feb 1996 11:47:26 +0000 (GMT)
• David Bakin
  • Is it a robot or a link-updater? Wed, 18 Oct 1995 12:49:14 -0700
• David Banes
  • Re: robot ? Wed, 04 Dec 1996 09:06:25 +1100
  • Re: robots.txt Fri, 22 Nov 1996 08:26:25 +1100
  • Re: Lycos' HEAD vs. GET Sun, 24 Nov 1996 17:14:54 +1100
  • User-Agent Wed, 20 Nov 1996 23:54:32 +1100
• David Eagles
  • Last message Fri, 19 Jul 1996 12:45:24 +-1000
  • Unusual request - sorry! Fri, 19 Jul 1996 11:06:41 +-1000
  • RE: "Good Times" hoax Thu, 25 Apr 1996 22:32:47 +-1000
  • RE: Server name in /robots.txt Sat, 20 Jan 1996 09:13:25 +-1100
  • RE: Recursion Wed, 10 Jan 1996 09:07:27 +-1100
  • Recursion Tue, 9 Jan 1996 22:12:58 +-1100
  • RE: avoiding infinite regress for robots Tue, 9 Jan 1996 08:03:08 +-1100
  • RE: Inter-robot Communications - Part II Sat, 30 Dec 1995 09:06:36 +-1100
  • Inter-robot Communications - Part II Fri, 29 Dec 1995 11:42:56 +-1100
  • Inter-robot Comms Port Sat, 23 Dec 1995 21:54:20 +-1100
  • Re: Q: Cooperation of robots Sun, 26 Nov 1995 09:24:39 +1000 (EST)
  • yet another robot, volume 2 Sat, 25 Nov 1995 07:53:43 +1000 (EST)
• David Eichmann
  • Re: depth first vs breadth first Thu, 29 Aug 1996 23:41:11 -0800
  • Re: depth first vs breadth first Thu, 29 Aug 1996 23:22:14 -0800
  • Re: Social Responsibilities (was Safe Methods) Fri, 19 Jul 1996 09:19:45 -0500
  • Re: Robot Research Wed, 17 Jul 1996 15:46:51 -0500
  • Re: Newbie question Mon, 15 Jul 1996 13:43:14 -0500
• David Henderson
  • Re: Links (don't bother checking; I've done it for you) Fri, 29 Mar 1996 11:09:48 -0800
  • Robots not Frames savy Sat, 13 Jan 1996 11:00:17 -0800
  • Re: test; please ignore Thu, 21 Dec 1995 07:57:20 -0800
  • New Robot??? Sun, 17 Dec 1995 18:53:55 -0800
  • Re: [2]RE>[5]RE>Checking Lo Sat, 16 Dec 1995 12:28:38 -0800
• david jost
  • Re: Who are you robots.txt? was Re: Servers vs Agents Mon, 02 Dec 96 09:07:17 est
• David Koblas
  • Crawlers and "dynamic" urls Mon, 9 Dec 1996 16:23:28 -0800
• David L. Sifry
  • Re: Crawling & DNS issues Tue, 21 Jan 1997 23:17:08 -0800
  • Conceptbot spider Sun, 08 Sep 1996 01:52:00 -0700
• David Levine
  • RE: The Internet Archive robot Fri, 13 Sep 1996 09:19:51 -0400
  • RE: The Internet Archive robot Thu, 12 Sep 1996 08:48:31 -0400
  • Re: nastygram for xxx.lanl.gov Wed, 10 Jul 1996 14:16:50 -0400
  • RE: Should I index all ... Tue, 9 Jul 1996 16:25:18 -0400
  • RE: Java Robot Mon, 24 Jun 1996 16:05:10 -0400
  • RE: Java Robot Mon, 24 Jun 1996 11:12:46 -0400
  • RE: robots.txt usage Tue, 18 Jun 1996 13:46:41 -0400
  • (OTP) RE: Political economy of distributed search (was topical search...) Fri, 3 May 1996 09:42:18 -0400
  • RE: http directory index request Tue, 16 Apr 1996 14:39:40 -0400
  • Re: New Robot Announcement Tue, 16 Jan 1996 14:17:25 -0500
• David M Banes
  • Re: Servers vs Agents Fri, 29 Nov 1996 10:12:12 +1100
  • Re: defining "robot" Tue, 26 Nov 1996 14:30:15 +1100
  • Re: defining "robot" Fri, 22 Nov 1996 17:52:24 +1100
  • Re: defining "robot" Fri, 22 Nov 1996 17:52:24 +1100
  • Re: robots.txt Thu, 21 Nov 1996 12:53:16 +1100
• David McGrath
  • Not so Friendly Robot - Teleport Fri, 15 Nov 1996 13:37:40 -0500
• David Reilly
  • Re: Anyone know who owns this one? Sat, 03 Aug 1996 11:42:01 +1000
  • Re: netscape spec for RDM Thu, 01 Aug 1996 09:27:59 +1000
  • Re: hoohoo.cac.washington = bad Mon, 29 Jul 1996 10:17:40 +1000
  • Re: loc(SOIF) Thu, 27 Jun 1996 11:49:32 +1000
  • Re: Inter-robot communication Wed, 26 Jun 1996 08:05:37 +1000
  • Re: Inter-robot communication Thu, 01 Jan 1970 10:00:00 +1000
  • Inter-robot communication Thu, 01 Jan 1970 10:00:00 +1000
  • Re: Keyword indexing Thu, 01 Jan 1970 10:00:00 +1000
  • Keyword indexing Thu, 01 Jan 1970 10:00:00 +1000
• David Schnardthorst
  • Re: Robots available for Intranet applications Fri, 22 Mar 1996 16:45:14 -0600 (CST)
  • Robot for Sun Mon, 19 Feb 1996 12:31:18 -0600 (CST)
• David Steele
  • Re: Info on large scale spidering? Mon, 20 Jan 1997 23:07:07 -0800 (PST)
  • Re: an image observer Thu, 16 Jan 1997 10:51:59 -0800 (PST)
  • Re: Is a robot visiting? Fri, 1 Nov 1996 15:27:52 -0800 (PST)
• Davide Musella
  • Re: "hidden text" vs. META tags for robots/search engines Mon, 16 Sep 1996 10:21:43 -0100 (GMT)
  • Re: Description or Abstract? Tue, 09 Jul 1996 11:45:34 -0100
  • Re: Description or Abstract? Fri, 5 Jul 1996 10:56:49 -0100 (GMT)
  • Re: Description or Abstract? Thu, 4 Jul 1996 14:21:51 -0100
  • Re: implementation fo HEAD response with meta info Mon, 10 Jun 1996 10:40:18 -0100 (GMT)
  • implementation fo HEAD response with meta info Fri, 7 Jun 1996 18:38:54 -0100
  • Re: keywords in META-element Wed, 27 Mar 1996 14:08:22 -0100
  • fdsf Thu, 15 Feb 1996 19:24:59 -0100
  • Re: HEAD request [was Re: Server name in /robots.txt] Mon, 29 Jan 1996 18:21:28 -0100
  • HEAD request [was Re: Server name in /robots.txt] Tue, 23 Jan 1996 19:26:19 -0100
  • Re: meta tag implementation Tue, 7 Nov 1995 13:05:21 +0100 (MET)
  • meta tag implementation Tue, 31 Oct 1995 15:59:26 +0100 (MET)
• Davis, Ian
  • Re: Standard? Tue, 03 Dec 96 14:02:00 GMT
  • Servers vs Agents Wed, 27 Nov 96 15:21:00 GMT
• Debbie Swanson
  • RE: verify URL Wed, 13 Mar 1996 14:00:15 -0500
• DECLAN FITZPATRICK
  • Re: Domains and HTTP_HOST Wed, 6 Nov 1996 20:05:49 GMT
• Denis McKeon
  • Re: Standard? Tue, 3 Dec 1996 16:07:31 -0700
  • Re: RFC, draft 1 Sat, 16 Nov 1996 07:31:39 -0700
  • RE: The Internet Archive robot Sat, 14 Sep 1996 11:38:04 -0600
  • RE: The Internet Archive robot Thu, 12 Sep 1996 15:14:47 -0600
  • Re: Copyrights, let them be ! Wed, 11 Sep 1996 10:00:24 -0600
  • Re: Copyrights on the web Wed, 11 Sep 1996 09:29:59 -0600
  • Re: Extracting info from SIG forum archives Sun, 8 Sep 1996 06:16:36 -0600
  • Re: Alta Vista getting stale? Wed, 10 Jul 1996 18:02:41 -0600
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 16:13:49 -0600
  • Re: Alta Vista getting stale? Tue, 9 Jul 1996 15:08:22 -0600
  • Re: in-document directive to discourage indexing ? Mon, 24 Jun 1996 10:16:48 -0600
  • in-document directive to discourage indexing ? Tue, 18 Jun 1996 21:20:18 -0600
• Detlev Kalb
  • keywords in META-element Wed, 27 Mar 1996 11:28:31 +0000
  • Re: Info on authoring a Web Robot Tue, 26 Mar 1996 10:09:42 +0000
• Digital Universe Inc.
  • (no subject) Mon, 10 Jun 1996 12:05:52 -0700 (PDT)
• dino
  • Re: Here is WebWalker Sat, 10 Feb 1996 12:56:50 +0800 (CST)
  • Re: Perl Spiders Fri, 9 Feb 1996 08:11:08 +0800 (CST)
• Dirk.vanGulik
  • Re: Suggestion to help robots and sites coexist a little better Mon, 15 Jul 1996 10:30:19 +0200
• Donald E. Eastlake 3rd
  • Re: Tagging a document with language Fri, 7 Jun 1996 09:18:55 -0400 (EDT)
• Dong-Hyun Kim
  • Please Help ME!! Tue, 2 Jan 96 17:40:33 KST
• Dorian Ellis
  • email address grabber Tue, 14 Jan 1997 13:40:00 +0000
• Douglas Summersgill
  • Robots available for Intranet applications Mon, 11 Mar 1996 09:31:55 -0500
• dpp@peg.apc.org
  • NetJet Tue, 26 Nov 1996 15:27:51 +1000 (EST)
• Dr. Detlev Kalb
  • Re: Introducing myself Thu, 20 Jun 1996 14:41:38 +0000
  • Re: Introducing myself Wed, 19 Jun 1996 17:56:35 +0000
• Drew Hamilton
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 11:59:29 -0400
  • Re: in-document directive to discourage indexing ? Mon, 24 Jun 1996 14:12:52 -0400
• drose@AZStarNet.com
  • Re: Somebody is turning 23! Wed, 15 May 1996 17:56:04 -0700 (MST)
  • Re: Somebody is turning 23! Wed, 15 May 1996 13:42:16 -0700 (MST)
  • Re: Looking for a spider Fri, 1 Dec 1995 20:52:46 -0700
• dws
  • roverbot - perhaps the worst robot yet Thu, 27 Jun 1996 13:02:20 -0400 (EDT)
  • More dangers of spiders... Sun, 16 Jun 1996 19:18:52 -0400 (EDT)
• ecarp@tssun5.dsccc.com
  • Re: [2]RE>[5]RE>Checking Log fi Fri, 15 Dec 1995 14:03:20 -0600
  • Freely available robot code in C available? Tue, 12 Dec 1995 22:25:31 -0600
• Ed Carp
  • Re: your mail Tue, 25 Jun 1996 23:08:34 -0500 (CDT)
  • Re: web topology Wed, 26 Jun 1996 23:51:26 -0500 (CDT)
  • Re: your mail Tue, 25 Jun 1996 23:08:34 -0500 (CDT)
  • Re: looking for specific bot... Fri, 14 Jun 1996 12:15:42 -0500 (CDT)
  • Re: VB. page grabber... Mon, 6 May 1996 11:43:28 -0500 (CDT)
• Ed Carp @ TSSUN5
  • Re: robots.txt , authors of robots , webmasters .... Thu, 18 Jan 1996 14:27:35 -0600
  • Re: New Robot Announcement Wed, 17 Jan 1996 09:35:00 -0600
  • Alta Vista searches WHAT?!? Tue, 16 Jan 1996 13:05:36 -0600
  • Re: New Robot Announcement Tue, 16 Jan 1996 10:28:59 -0600
  • Re: Does anyone else consider this irresponsible? Wed, 10 Jan 1996 13:59:57 -0600
• Ed Costello
  • Use of robots.txt to "check status"? Sun, 29 Sep 1996 10:47:47 -0400 (EDT)
• edas@micros.et.byu.edu
  • (no subject) Tue, 18 Jun 1996 21:16:49 -0700
• Eddie Rojas
  • file retrieval Mon, 23 Sep 1996 10:18:27 -0600 (MDT)
• Edward Ooi
  • Webfetch Thu, 22 Aug 1996 09:11:08 -0500
• Edward Stangler
  • Search Engine Tutorial for Web Developers Thu, 8 Aug 1996 22:32:26 CDT
  • Re: Alta Vista searches WHAT?!? Thu, 18 Jan 1996 20:59:44 CST
• Edwin Carp
  • Re: Freely available robot code in C available? Wed, 13 Dec 1995 15:15:17 +0000
• Elias Hatzis
  • databases for spiders Thu, 14 Nov 1996 18:16:13 +0200
  • databases for spiders Thu, 14 Nov 1996 18:16:13 +0200
  • Re: pointers for a novice? Sun, 8 Sep 1996 13:37:40 +0300
• Elias Sideris
  • RE: nastygram from xxx.lanl.gov Tue, 9 Jul 1996 12:04:41 +-300
• Ellen M Voorhees
  • Re: Search accuracy Fri, 5 Apr 1996 07:59:11 -0500 (EST)
• erc@dal1820.computek.net
  • Re: VB and robot development Sun, 7 Apr 1996 04:51:52 -0400
• Eric
  • Somebody is turning 23! Wed, 15 May 1996 20:35:33 +0100 (MET)
• Eric Hollander
  • Re: Does anyone else consider this irresponsible? Wed, 10 Jan 1996 11:24:35 -0800
• Eric Knight
  • Re: Book about robots (was Re: Defenses against bad robots) Mon, 20 May 1996 08:04:24 -0700
• Eric Kristoff
  • Netscape Catalog Server Tue, 8 Oct 96 08:48:51 -0500
  • Netscape Catalog Server Tue, 8 Oct 96 08:47:20 -0500
  • Netscape Catalog Server: An Eval Sun, 22 Sep 96 17:42:03 -0500
  • Re: The Internet Archive robot Fri, 20 Sep 96 16:07:31 -0500
  • Re: The Internet Archive robot Thu, 19 Sep 96 17:51:00 -0500
  • Re: Copyrights on the web Fri, 13 Sep 96 13:06:19 -0500
• Eric Mackie
  • Web pages being served from an SQL database Tue, 17 Dec 1996 15:04:32 -0500
• Eric Miller
  • Re: Meta Tags Fri, 24 Jan 1997 10:52:42 -0500
  • Re: indexing via redirectors Tue, 21 Jan 1997 10:27:59 -0500
  • Re: indexing via redirectors Tue, 21 Jan 1997 08:52:32 -0500
  • Re: META tag standards, search accuracy Mon, 14 Oct 1996 14:48:42 -0400
  • Re: META tag standards, search accuracy Mon, 14 Oct 1996 14:26:21 -0400
  • Re: META tag standards, search accuracy Mon, 14 Oct 1996 13:10:26 -0400
  • Re: META tag standards, search accuracy Mon, 14 Oct 1996 10:12:03 -0400
• Erik Selberg
  • Re: Single tar (Re: Inktomi & large scale spidering) 27 Jan 1997 12:14:46 -0800
  • Re: Inktomi & large scale spidering 26 Jan 1997 12:28:58 -0800
  • Re: AltaVista Meta Tag Rumour 21 Jan 1997 12:18:59 -0800
  • Re: Standard? 03 Dec 1996 14:08:26 -0800
  • Re: make people use ROBOTS.txt? 01 Dec 1996 16:10:31 -0800
  • Re: Servers vs Agents 28 Nov 1996 12:36:21 -0800
  • Re: defining "robot" 25 Nov 1996 13:00:17 -0800
  • Re: defining "robot" 25 Nov 1996 12:02:23 -0800
  • Re: make people use ROBOTS.txt? 25 Nov 1996 00:49:45 -0800
  • Re: make people use ROBOTS.txt? 24 Nov 1996 14:55:16 -0800
  • Re: defining "robot" 24 Nov 1996 14:44:25 -0800
  • Re: robots.txt (A *little* off the subject) 24 Nov 1996 14:32:08 -0800
  • Re: robots.txt (A *little* off the subject) 22 Nov 1996 15:24:10 -0800
  • Re: who/what uses robots.txt 22 Nov 1996 13:56:09 -0800
  • Re: who/what uses robots.txt 22 Nov 1996 13:48:28 -0800
  • Re: Broadness of Robots.txt (Re: Washington again !!!) 21 Nov 1996 15:30:14 -0800
  • Re: who/what uses robots.txt 21 Nov 1996 15:18:09 -0800
  • Re: Broadness of Robots.txt (Re: Washington again !!!) 21 Nov 1996 15:09:31 -0800
  • Re: who/what uses robots.txt 21 Nov 1996 14:40:03 -0800
  • Re: Agent Categories (was Re: Broadness of Robots.txt (Re: Washington again !!!) 21 Nov 1996 14:38:44 -0800
  • Re: Broadness of Robots.txt (Re: Washington again !!!) 21 Nov 1996 09:43:47 -0800
  • Re: Broadness of Robots.txt (Re: Washington again !!!) 20 Nov 1996 23:50:25 -0800
  • Re: Broadness of Robots.txt (Re: Washington again !!!) 20 Nov 1996 13:25:53 -0800
  • Re: Broadness of Robots.txt (Re: Washington again !!!) 20 Nov 1996 13:15:06 -0800
  • Re: Washington again !!! 19 Nov 1996 19:37:31 -0800
  • Re: Washington again !!! 19 Nov 1996 17:20:03 -0800
  • Re: Notification protocol? 18 Nov 1996 14:30:46 -0800
  • Re: agents ignoring robots.txt 17 Oct 1996 13:21:41 -0700
  • Re: agents ignoring robots.txt 16 Oct 1996 14:49:26 -0700
  • Re: I vote NO (Was: Robot Gripes forum?) - I vote YES 03 Aug 1996 15:12:16 -0700
  • Re: hoohoo.cac.washington = bad 29 Jul 1996 10:17:54 -0700
  • Re: Meta-seach engines 04 May 1996 14:04:29 -0700
  • Re: Political economy of distributed search (was topical search...) 03 May 1996 02:19:48 -0700
  • Re: Robots and search engines technical information. Fri, 16 Feb 1996 10:54:53 PST
  • Re: The Robot And Search Engine FAQ Wed, 14 Feb 1996 11:55:36 PST
  • Re: Alta Vista searches WHAT?!? Fri, 19 Jan 1996 12:47:48 PST
  • Re: Alta Vista searches WHAT?!? Wed, 17 Jan 1996 10:47:36 PST
  • Re: Robots / source availability? Thu, 11 Jan 1996 09:23:23 PST
• Ethan Lee
  • robots source code in C Thu, 18 Jul 1996 08:02:02 +0800
• Fabio Arciniegas A.
  • Java Robot Sat, 22 Jun 1996 21:19:08 -0500
• Filippo Menczer
  • info for newbie Sun, 15 Sep 1996 23:42:02 +1111
  • InfoSpiders/0.1 Sat, 14 Sep 96 18:41:27 -0700
• Francesco
  • How to...??? Mon, 25 Mar 96 11:59:46 -0100
• Francois Rouaix
  • Back of the envelope computations Wed, 06 Nov 1996 17:35:46 +0100
  • Re: Possible robots.txt addition Wed, 06 Nov 1996 16:37:34 +0100
• Frank Smadja
  • FAQ? Thu, 12 Sep 1996 09:43:33 +0300
  • Re: About integrated search engines Sat, 27 Apr 1996 10:01:10 +0300
  • Re: Anyone doing a Java-based robot yet? Wed, 21 Feb 1996 09:50:03 +0000
  • Re: Indexing two-byte text Thu, 07 Dec 1995 09:27:25 -0500
• Frank Wales
  • RE: nastygram from xxx.lanl.gov Thu, 11 Jul 1996 20:44:08 +-100
  • RE: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 01:54:25 +-100
  • RE: nastygram from xxx.lanl.gov Tue, 9 Jul 1996 14:50:11 +-100
• Fred Douglis
  • Re: The Internet Archive robot Fri, 13 Sep 1996 13:55:24 -0400
  • Re: The Internet Archive robot (fwd) Mon, 09 Sep 1996 10:13:48 -0400
  • Re: The Internet Archive robot Fri, 06 Sep 1996 19:00:39 -0400
  • Re: wwwbot.pl problem Mon, 27 Nov 1995 11:15:47 -0500
• Fred K. Lenherr
  • Re: Notification protocol? Tue, 12 Nov 1996 04:27:27 -0500
  • Re: ActiveAgent Sat, 09 Nov 1996 07:20:39 -0500
  • Re: ActiveAgent Fri, 08 Nov 1996 03:24:30 -0500
  • Filtering queries on a robot-built database Wed, 30 Oct 1996 10:43:09 -0500
  • Comparing robots/search sites Sun, 20 Oct 1996 19:48:53 -0400
  • infoseek Sun, 20 Oct 1996 17:48:36 -0400
  • robots.txt syntax Fri, 11 Oct 1996 07:15:17 -0400
  • Topic-specific robots Wed, 09 Oct 1996 08:54:59 -0400
  • Re: Keyword indexing Sat, 27 Jul 1996 10:38:14 -0400
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 14:48:19 -0400
  • Re: robots.txt unavailability Tue, 09 Jul 1996 06:57:05 -0400
  • Blackboard for Discussing Domain-specific Robots Fri, 05 Jul 1996 07:50:37 -0400
  • web topology Tue, 25 Jun 1996 09:43:52 -0400
  • Re: RCPT: Re: Introducing myself Fri, 21 Jun 1996 08:20:19 -0400
  • Re: Introducing myself Tue, 18 Jun 1996 04:10:00 -0400
  • (no subject) Tue, 11 Jun 1996 04:55:34 -0400
  • robots, what else! Sat, 08 Jun 1996 05:23:32 -0400
• Fred Melssen
  • interactive generation of URL's Mon, 15 Jul 1996 11:03:57 +0100 (MET)
  • Harvest-like use of spiders Wed, 03 Jul 1996 18:07:12 +0100 (MET)
  • Indexing a set of URL's Thu, 02 May 1996 16:04:54 +0100 (MET)
• frizzlefry@nucleus.atom.com
  • image map traversal Mon, 25 Mar 1996 13:14:40 -0500
• G. Edward Johnson
  • More ways to spam search engines? Tue, 10 Sep 1996 11:44:42 -0400 (EDT)
  • Description or Abstract? Wed, 3 Jul 1996 17:11:41 -0400 (EDT)
  • Re: implementation fo HEAD response with meta info Fri, 7 Jun 1996 13:50:15 -0400 (EDT)
• Gareth R White
  • Re: The Internet Archive robot 23 Sep 96 9:53:55 EDT
• Garth T Kidd
  • Re: Safe Methods Thu, 18 Jul 1996 10:28:56 -1000
  • Re: nastygram from xxx.lanl.gov Thu, 11 Jul 1996 14:33:45 -1000
  • Re: nastygram from xxx.lanl.gov Thu, 11 Jul 1996 10:01:35 -1000
  • Re: nastygram from xxx.lanl.gov Thu, 11 Jul 1996 10:01:32 -1000
• Gary L. Burt
  • Re: robots & copyright law Mon, 27 Jan 1997 17:23:49 -0800
  • Re: Standard? Tue, 03 Dec 1996 22:42:30 -0500
  • Re: Junkie-Mail was Re: Standard? Mon, 02 Dec 1996 23:10:50 -0500
• Gen-ichiro Kikui
  • Re: Tagging a document with language Fri, 07 Jun 1996 16:55:22 +0900
• Gene Essman
  • Re: Looking for a spider Fri, 1 Dec 1995 09:40:26 -0800
• Gerry McKiernan
  • The Big Picture(sm): Visual Browsing in Web and non-Web Databases Tue, 10 Dec 96 09:19:54 CST
  • CitedSites(sm): Citation Indexing of Web Resources Sun, 6 Oct 96 13:39:28 CDT
  • Seeing is Believing: Candidate Web Resources for Information Visualization Fri, 4 Oct 96 13:49:41 CDT
  • The End of The World (Wide Web) / Part II Thu, 3 Oct 96 17:17:16 CDT
  • Bye Bye HyperText: The End of the World (Wide Web) As We Know It! Tue, 1 Oct 96 12:10:38 CDT
  • Library Agents(sm): Library Applications of Intelligent Software Agents Fri, 23 Aug 96 19:10:39 CDT
  • Project Aristotle(sm) Sun, 11 Aug 96 18:15:38 CDT
• Giacomo Fiorentini
  • How can IR Agents be evaluate ? Sat, 10 Aug 96 15:56:02 MET DST
• gil cosson
  • please add my site Sun, 31 Dec 1995 11:49:24 -0600 (CST)
  • Re: How frequently should I check /robots.txt? Mon, 27 Nov 1995 23:27:08 -0600 (CST)
• Glburt@aol.com
  • Re: Copyrights on the web Tue, 10 Sep 1996 16:30:47 -0400
• Gordon Bainbridge
  • Re: nastygram from xxx.lanl.gov Thu, 11 Jul 1996 08:32:50 -0700
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 09:01:18 -0700
  • Re: Mailing list Tue, 23 Apr 1996 07:42:58 -0700
  • Re: [2]RE>[5]RE>Checking Log fi Fri, 15 Dec 1995 11:22:29 -0800
  • Re: [3]RE>[5]RE>Checking Log fi Fri, 15 Dec 1995 10:14:19 -0800
• Gordon V. Cormack
  • BSE-Slurp/0.6 Sun, 3 Mar 1996 02:34:32 -0500
• Greg Fenton
  • Re: Info on large scale spidering? Tue, 21 Jan 1997 09:14:51 -0500
  • an article... (was: Re: Standard?) Fri, 29 Nov 1996 09:07:28 -0500
  • Re: Standard Wed, 27 Nov 1996 09:36:55 -0500
  • RE: Is a robot visiting? Sat, 2 Nov 1996 19:10:22 -0500
  • RE: What to rate limit/lock on, name or IP address? Wed, 23 Oct 1996 14:19:57 -0400
  • crawling FTP sites Wed, 11 Sep 1996 11:22:15 -0400
  • Re: How long to cache robots.txt for? Wed, 10 Jul 1996 10:45:51 -0400
  • Re: Looking for good one Mon, 17 Jun 1996 10:16:42 -0400
  • Re: Meta-seach engines Sat, 04 May 1996 12:37:00 -0400
• Greg Tanzola
  • Re: WRITERS WANTED (re-post) Thu, 26 Dec 1996 12:11:15 -0800
• Gregg Steinhilpert
  • RE: WebAnalyzer - introduction Tue, 18 Jun 1996 12:11:32 -0400
• Gregory Lauckhart
  • Re: Washington again !!! Wed, 20 Nov 1996 08:11:31 -0800 (PST)
• Hal Belisle
  • Gopher Protocol Question Sun, 14 Jan 1996 19:24:47 -0800
• Hallvard B Furuseth
  • Re: RFC, draft 1 Tue, 10 Dec 1996 23:32:19 +0100 (MET)
  • Re: Broadness of Robots.txt (Re: Washington again !!!) Fri, 22 Nov 1996 12:44:19 +0100 (MET)
  • Re: RFC, draft 1 Mon, 18 Nov 1996 17:19:36 +0100 (MET)
  • Re: An extended version of the Robots... Fri, 15 Nov 1996 13:25:10 +0100 (MET)
  • Re: An extended version of the Robots... Fri, 15 Nov 1996 13:09:03 +0100 (MET)
  • Re: Get official! Tue, 12 Nov 1996 11:19:01 +0100 (MET)
  • Re: An extended version of the Robots... Tue, 12 Nov 1996 10:41:30 +0100 (MET)
  • Regexps (Was: Re: An extended version of the Robots...) Mon, 11 Nov 1996 16:50:22 +0100 (MET)
  • Regexps (Was: Re: An extended version of the Robots...) Mon, 11 Nov 1996 16:49:33 +0100 (MET)
  • Re: An extended version of the Robots... Mon, 11 Nov 1996 16:21:45 +0100 (MET)
  • Get official! Mon, 11 Nov 1996 13:04:10 +0100 (MET)
  • anti-robot regexps Fri, 8 Nov 1996 20:46:34 +0100 (MET)
  • anti-robot regexps Fri, 8 Nov 1996 20:43:11 +0100 (MET)
  • RE: Is a robot visiting? Mon, 4 Nov 1996 21:09:52 +0100 (MET)
  • Re: Is a robot visiting? Mon, 4 Nov 1996 20:56:25 +0100 (MET)
  • Re: Is a robot visiting? Mon, 4 Nov 1996 20:53:46 +0100 (MET)
  • Robot exclustion for for non-'unix file' hierarchy Tue, 1 Oct 1996 00:21:19 +0200 (MET DST)
• Hani Yakan
  • Re: Looking for subcontracting spider-programmers Fri, 06 Sep 1996 15:57:38 -0500
• Harold Gibbs
  • Image Maps Thu, 03 Oct 96 11:31:56 PST
• Harry Munir Behrens
  • Re: Search Engine Wed, 14 Aug 1996 07:52:05 +0900
  • Re: Inter-robot communication Tue, 25 Jun 1996 01:08:26 +0900
  • Meta-seach engines Sun, 05 May 1996 01:07:08 +0900
  • Re: default documents Mon, 22 Apr 1996 11:11:29 +0900
  • Indexing two-byte text Fri, 08 Dec 1995 00:26:27 +0900
  • Indexing two-byte text Wed, 06 Dec 1995 15:43:23 +0900
• Hauke Loens
  • Re: RCPT: Re: Introducing myself Mon, 24 Jun 1996 08:41:57 -0100
  • RCPT: Re: Introducing myself Fri, 21 Jun 1996 09:02:44 -0100
  • RCPT: Re: Introducing myself Thu, 20 Jun 1996 10:28:29 -0100
  • RCPT: Re: Robot books Wed, 19 Jun 1996 09:58:10 -0100
  • Re: Robot books Tue, 18 Jun 1996 19:30:40 -0100
• Hayssam Hasbini
  • Robots and search engines technical information. Thu, 15 Feb 1996 13:27:07 -0500 (EST)
• Henk Alles
  • RE: Tagging a document with language Wed, 19 Jun 1996 00:12:43 +-200
  • RE: Tagging a document with language Tue, 18 Jun 1996 01:03:18 +-200
  • RE: Tagging a document with language Fri, 7 Jun 1996 10:01:54 +-200
• Henrik Fagrell
  • Robot to collect web pages per site Wed, 5 Jun 1996 13:19:24 GMT
• HipCrime
  • Too Many Admins (TMA) !!! Fri, 17 Jan 1997 10:28:36 -0800
  • make people use ROBOTS.txt? Fri, 22 Nov 1996 23:57:09 -0800
  • robot? Fri, 22 Nov 1996 09:01:15 -0800
  • not a robot Fri, 22 Nov 1996 07:56:10 -0800
  • robot defined Fri, 22 Nov 1996 07:52:37 -0800
  • define a page? Thu, 21 Nov 1996 21:05:11 -0800
  • defining "robot" Thu, 21 Nov 1996 17:32:26 -0800
  • Re: who/what uses robots.txt Thu, 21 Nov 1996 15:33:34 -0800
  • Re: who/what uses robots.txt Thu, 21 Nov 1996 14:52:10 -0800
  • who/what uses robots.txt Wed, 20 Nov 1996 23:35:48 -0800
  • robots.txt Wed, 20 Nov 1996 13:13:26 -0800
  • http://HipCrime.com Mon, 18 Nov 1996 09:03:02 -0800
  • ActiveAgent Thu, 07 Nov 1996 09:40:00 -0800
  • sockets in PERL Thu, 31 Oct 1996 13:08:47 -0800
  • Re: ActiveAgent Mon, 14 Oct 1996 09:01:21 -0700
  • ActiveAgent Sun, 13 Oct 1996 19:17:47 -0700
  • Re: ActiveAgent Sun, 13 Oct 1996 19:17:51 -0700
• HiPromote@aol.com
  • Need help again. Wed, 29 Jan 1997 16:49:37 -0500 (EST)
  • Need help on Search Engine accuracy test. Wed, 29 Jan 1997 11:59:59 -0500 (EST)
• Howard Strachman
  • FS- Sharks Tkts- Front Row (2nd Deck) Jan 13 Thu, 26 Dec 1996 10:33:57 -0800
• Howard, Dan: CIO
  • RE: How to get the document info ? Fri, 14 Feb 1997 12:02:49 -0500
  • Lack of support for "If-Modified-Since" Thu, 13 Feb 1997 15:25:02 -0500
  • Re: Single tar (Re: Inktomi & large sca Mon, 27 Jan 1997 13:44:29 -0500
• Hrvoje Niksic
  • Re: More with the Cypherpunk antics 13 Feb 1997 09:41:47 +0100
  • Re: indexing via redirectors 22 Jan 1997 01:30:59 +0100
  • Re: indexing via redirectors 21 Jan 1997 21:41:11 +0100
  • Re: indexing via redirectors 21 Jan 1997 14:24:21 +0100
  • Re: Standard 27 Nov 1996 11:47:18 +0100
  • Re: make people use ROBOTS.txt? 25 Nov 1996 03:43:32 +0100
  • Re: defining "robot" 23 Nov 1996 19:38:09 +0100
  • Re: not a robot 23 Nov 1996 12:23:24 +0100
  • Re: defining "robot" 22 Nov 1996 09:16:11 +0100
  • Re: who/what uses robots.txt 22 Nov 1996 06:18:43 +0100
  • Re: Broadness of Robots.txt (Re: Washington again !!!) 21 Nov 1996 05:02:20 +0100
  • Re: RFC, draft 1 20 Nov 1996 00:58:26 +0100
  • Re: RFC, draft 1 16 Nov 1996 20:48:36 +0100
  • Re: infoseeks robot is dumb 16 Nov 1996 02:34:40 +0100
  • Re: infoseeks robot is dumb 16 Nov 1996 02:12:33 +0100
  • Re: An extended version of the Robots... 12 Nov 1996 08:02:05 +0100
  • Re: itelligent agents 10 Nov 1996 06:06:12 +0100
  • Re: An extended version of the Robots... 09 Nov 1996 17:14:13 +0100
  • Re: An extended verion of the robot exclusion standard 09 Nov 1996 17:01:15 +0100
  • Matching the user-agent in /robots.txt 07 Nov 1996 20:58:38 +0100
  • A new robot -- ask for advice 05 Nov 1996 00:36:54 +0100
• http
  • Tutorial Proposal for WWW95 Fri, 16 Feb 1996 12:45:55 +0100
• Huaiyu Liu
  • Information about AltaVista and Excite Mon, 10 Feb 1997 09:59:55 -0800
• I think therefore I spam
  • Admitting the obvious 13 Nov 1996 07:47:45 -0000
• i.bromwich
  • Re: Looking for a spider Mon, 4 Dec 1995 10:23:23 +0000
• Ian Graham
  • Re: Crawlers and "dynamic" urls Wed, 11 Dec 1996 10:07:36 -0500 (EST)
  • Re: Server Indexing -- Helping a Robot Out Thu, 5 Dec 1996 18:13:31 -0500 (EST)
  • Server Indexing -- Helping a Robot Out Wed, 4 Dec 1996 19:21:29 -0500 (EST)
  • Re: an article... (was: Re: Standard?) Fri, 29 Nov 1996 15:07:12 -0500 (EST)
  • Re: Cache Filler Fri, 29 Nov 1996 14:57:07 -0500 (EST)
  • Re: Possible robots.txt addition Thu, 7 Nov 1996 11:34:34 -0500 (EST)
  • Re: Possible robots.txt addition (fwd) Wed, 6 Nov 1996 17:57:19 -0500 (EST)
  • Re: Domains and HTTP_HOST Wed, 6 Nov 1996 14:41:30 -0500 (EST)
  • Re: Possible robots.txt addition Wed, 6 Nov 1996 12:34:49 -0500 (EST)
  • Re: Possible robots.txt addition Wed, 6 Nov 1996 12:23:11 -0500 (EST)
  • Possible robots.txt addition Mon, 4 Nov 1996 14:35:49 -0500 (EST)
• Ian King
  • RE: email grabber Thu, 16 Jan 1997 13:50:14 -0800
  • RE: Netscape-Catalog-Robot Mon, 30 Dec 1996 14:25:59 -0800
• Ian McKellar
  • Re: Offline Agents for UNIX Wed, 21 Aug 1996 08:56:44 +0800 (WST)
  • Lynx. The one true browser. Thu, 15 Aug 1996 10:36:50 +0800 (WST)
  • Re: VB and robot development Mon, 8 Apr 1996 09:09:56 +0800 (WST)
• Ian Samson
  • RE: AltaVista's Index is obsolete; but what about the other Mon, 22 Jul 1996 08:10:34 GMT+200
  • AltaVista's Index is obsolete Fri, 19 Jul 1996 16:51:25 GMT+200
  • Updating Robots Wed, 10 Jul 1996 10:15:52 GMT+200
  • Re: Unsubscribing from Robots (was "your mail") Wed, 26 Jun 1996 09:44:56 GMT+200
  • Re: Unsubscribing from Robots (was "your mail") Wed, 26 Jun 1996 09:44:56 GMT+200
• Issac Roth
  • Re: email address grabber Tue, 14 Jan 1997 22:46:48 -0800 (PST)
  • Re: Notification protocol? Mon, 18 Nov 1996 03:44:30 -0800 (PST)
  • Re: Possible robots.txt addition Tue, 5 Nov 1996 20:47:09 -0800 (PST)
  • Re: ActiveAgent Tue, 15 Oct 1996 19:46:53 -0700 (PDT)
  • Re: Robot Gripes forum? (Was: Anyone know who owns this one?) Fri, 2 Aug 1996 17:05:36 -0700 (PDT)
  • Re: BackRub robot Mon, 17 Jun 1996 16:07:19 -0700 (PDT)
• Istvan
  • Safe Methods Wed, 17 Jul 1996 14:13:37 +0800
  • Re: the POST myth... a web admin's opinions.. Fri, 12 Jul 1996 08:48:25 +0800
  • RE: nastygram from xxx.lanl.gov Thu, 11 Jul 1996 14:06:57 +0800
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 20:42:07 +0800
  • RE: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 12:01:35 +0800
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 11:32:35 +0800
  • RE: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 10:40:06 +0800
  • Re: nastygram from xxx.lanl.gov Tue, 9 Jul 1996 10:07:02 +0800
  • Re: nastygram from xxx.lanl.gov Tue, 9 Jul 1996 08:56:53 +0800
• J.E. Fritz
  • Re: Tagging a document with language Fri, 07 Jun 1996 10:06:50 -0400 (EDT)
  • Re: Web spaces of strange topology. Where? Fri, 17 May 1996 10:16:45 -0400 (EDT)
• J.Y.K.
  • Robot-HTML Web Page? Thu, 23 May 96 19:07:57 -0700
• Jaakko Hyvatti
  • Re: crawling FTP sites Wed, 11 Sep 1996 22:11:56 +0300 (EETDST)
  • Re: Suggestion to help robots and sites coexist a little better Thu, 18 Jul 1996 01:52:57 +0300 (EET DST)
  • Re: nastygram from xxx.lanl.gov Thu, 11 Jul 1996 10:25:11 +0300 (EET DST)
  • Re: How long to cache robots.txt for? Wed, 10 Jul 1996 09:42:05 +0300 (EET DST)
  • Re: robots.txt unavailability Tue, 9 Jul 1996 09:11:29 +0300 (EET DST)
  • Re: Should I index all ... Fri, 5 Jul 1996 14:32:45 +0300 (EET DST)
  • Re: Identifying identical documents Thu, 4 Jul 1996 02:26:51 +0300 (EET DST)
  • Re: in-document directive to discourage indexing ? Tue, 25 Jun 1996 10:41:27 +0300 (EET DST)
  • makerobots.perl (Re: in-document directive..) Wed, 19 Jun 1996 11:18:30 +0300 (EET DST)
  • Finding the canonical name for a server Mon, 10 Jun 1996 14:11:15 +0300 (EET DST)
  • Re: Defenses against bad robots Mon, 20 May 1996 11:14:08 +0300 (EET DST)
  • Re: Links (don't bother checking; I've done it for you) Fri, 29 Mar 1996 15:09:30 +0200 (EET)
  • Re: Limiting robots to top-level page only (via robots.txt)? Tue, 26 Mar 1996 12:02:40 +0200 (EET)
  • Re: robots.txt Thu, 15 Feb 1996 10:38:43 +0200 (EET)
  • Re: robots.txt changes how often? Thu, 15 Feb 1996 08:18:47 +0200 (EET)
  • Re: Horror story Mon, 15 Jan 1996 02:04:58 +0200 (EET)
  • Re: robots.txt extensions Thu, 11 Jan 1996 02:38:12 +0200 (EET)
  • Recursing heuristics (Re: Does this..) Tue, 9 Jan 1996 11:38:47 +0200 (EET)
  • Re: MacPower (an apology, I am very sorry) Mon, 4 Dec 1995 14:47:14 +0200 (EET)
  • Re: MacPower Mon, 4 Dec 1995 14:36:07 +0200 (EET)
  • Re: Q: Cooperation of robots Mon, 27 Nov 1995 11:13:46 +0200 (EET)
  • Simple load robot Thu, 2 Nov 1995 20:35:19 +0200 (EET)
• Jakob Faarvang
  • Possible MSIIS bug? Mon, 15 Jul 1996 21:19:36 +0100
  • Re: VB. page grabber... Mon, 6 May 1996 19:19:19 +0100
  • Re: any robots/search-engines which index links? Sat, 4 May 1996 19:26:45 +0100
  • Re: search engine Wed, 24 Apr 1996 19:48:47 +0100
  • default documents Sun, 21 Apr 1996 14:44:43 +0100
  • Re: VB and robot development Mon, 8 Apr 1996 04:05:51 +0100
  • Re: VB and robot development Sun, 7 Apr 1996 14:35:54 +0100
  • Re: New Robot Announcement Tue, 16 Jan 96 21:49:19 +0100 (CET)
  • Re: Web Robots Sun, 7 Jan 96 23:30:07 +0100 (CET)
• James
  • Re: Add This Search Engine to your Results. Wed, 7 Aug 1996 14:22:13 +1000
  • Re: Search Engine article Fri, 19 Jul 1996 09:54:03 +1000
  • Re: Web Robots Mon, 8 Jan 96 08:15:57 +1100 (EST)
  • Up to date list of Robots Sat, 6 Jan 96 09:20:51 +1100 (EST)
  • Re: yet another robot, volume 2 Sat, 25 Nov 95 11:13:05 +1100 (EST)
  • Re: yet another robot, volume 2 Sat, 25 Nov 95 09:29:44 +1100 (EST)
  • Site Announcement Sat, 14 Oct 95 11:07:39 0000
• James Black
  • Re: crawling FTP sites Wed, 11 Sep 1996 19:34:56 -0400 (EDT)
• Jane Doyle
  • Re: robots on an intranet Tue, 06 Aug 1996 00:23:53 +0100
• Jared Williams
  • [ MERCHANTS ] My Sincerest Apologies Thu, 25 Apr 1996 13:25:25 -0700 (PDT)
  • (no subject) Wed, 24 Apr 1996 16:40:32 -0700
  • Try robot... Wed, 24 Apr 1996 15:22:00 -0700
  • Re: Mailing list Mon, 22 Apr 1996 17:53:55 -0700
  • Mailing list Mon, 22 Apr 1996 13:41:20 -0700
  • Web Robot Thu, 18 Apr 1996 15:03:34 -0700
  • The Letter To End All Letters Fri, 29 Mar 1996 13:16:45 -0800
  • Re: Links Thu, 28 Mar 1996 22:15:58 -0800
  • Re: Links Thu, 28 Mar 1996 14:16:53 -0800
  • Links Mon, 25 Mar 1996 15:14:41 -0800
  • Info on authoring a Web Robot Mon, 25 Mar 1996 10:12:28 -0800
  • ReHowto...??? Mon, 25 Mar 1996 10:09:40 -0800
  • Re: Robots available for Intranet applications Thu, 14 Mar 1996 09:29:17 -0800
  • Re: Robots available for Intranet applications Wed, 13 Mar 1996 10:09:04 -0800
• Jason_Murray_at_FCRD@cclink.tfn.com
  • Re: Smart Agent help Mon, 27 Nov 95 15:55:32 EST
• Jeannine Washington
  • RCPT: Re: Info on authoring a Web Robot Tue, 26 Mar 1996 08:31:01 MST
• Jeff Drost
  • Re: More with the Cypherpunk antics Sat, 15 Feb 1997 13:04:06 -0500 (EST)
  • Re: Meta Tags Tue, 21 Jan 1997 19:03:47 -0500 (EST)
  • Re: Meta Tags Tue, 21 Jan 1997 15:35:15 -0500 (EST)
  • Re: indexing via redirectors Tue, 21 Jan 1997 14:42:26 -0500 (EST)
  • Re: email grabber Thu, 16 Jan 1997 09:56:58 -0500 (EST)
  • Re: email address grabber Wed, 15 Jan 1997 12:54:33 -0500 (EST)
  • Re: email address grabber Wed, 15 Jan 1997 09:39:20 -0500 (EST)
  • Re: email address grabber Tue, 14 Jan 1997 19:54:51 -0500 (EST)
• Jeffrey C. Chen
  • Re: meta tag implementation Thu, 2 Nov 1995 12:28:47 -0500 (EST)
• Jeffrey Kerns
  • unscribe Mon, 24 Jun 1996 19:34:25 -0400
• Jennifer C. O'Brien
  • McKinley -- 100% error rate Mon, 21 Oct 1996 17:03:57 +0100
• Jeremy Sigmon
  • MOMSpider problem. Broken Pipe Wed, 23 Oct 1996 14:01:16 -0400 (EDT)
  • robots.txt buffer question. Tue, 22 Oct 1996 19:23:46 -0400 (EDT)
  • Re: The Internet Archive robot Tue, 24 Sep 1996 09:29:59 -0400 (EDT)
  • Re: The Internet Archive robot Mon, 23 Sep 1996 10:53:46 -0400 (EDT)
  • Re: Topic drift (archive robot, copyright...) Fri, 20 Sep 1996 22:02:06 -0400 (EDT)
  • Re: The Internet Archive robot Fri, 20 Sep 1996 11:04:24 -0400 (EDT)
• Jeremy.Ellman
  • Re: hoohoo.cac.washington = bad Mon, 29 Jul 1996 17:40:31 +0100
  • Re: Robot to collect web pages per site Thu, 6 Jun 1996 16:00:50 +0100 (BST)
  • Re: Political economy of distributed search (was topical search...) Fri, 3 May 1996 11:41:19 +0100
  • Re: How to...??? Mon, 25 Mar 1996 13:53:30 +0000 (GMT)
  • Re: robots.txt changes how often? Mon, 19 Feb 1996 10:25:06 +0000 (GMT)
  • Re: Robots and search engines technical information. Fri, 16 Feb 1996 13:58:55 +0000 (GMT)
  • Re: New Robot Announcement Tue, 16 Jan 1996 09:39:23 GMT
  • Re: Does this count as a robot? Mon, 8 Jan 1996 09:59:44 GMT
  • Re: Announcement and Help Requested Mon, 18 Dec 1995 12:20:21 +0000 (GMT)
  • Re: Announcement and Help Requested Mon, 18 Dec 1995 12:20:21 +0000 (GMT)
• Jerry Walsh
  • Re: email spider Wed, 14 Aug 1996 20:28:58 +0000
• Jim Ausman
  • netscape spec for RDM Mon, 29 Jul 1996 14:17:32 -0700
  • Re: dumb robots and xxx Fri, 12 Jul 1996 17:17:31 -0700
• Jim Harris
  • RE: email spider Sat, 17 Aug 1996 14:26:04 -0400
  • RE: How can IR Agents be evaluate ? Sat, 10 Aug 1996 18:16:14 -0400
• Jim Meritt
  • Re[2]: verify URL Wed, 08 May 96 11:54:30 EST
  • Re[2]: verify URL Wed, 13 Mar 96 14:01:05 EST
  • verify URL Wed, 13 Mar 96 11:43:59 EST
  • Re: BOUNCE robots: Admin request Wed, 17 Jan 96 16:19:00 EST
  • Re[2]: Harvest question Thu, 14 Dec 95 08:47:11 EST
  • Harvest question Wed, 13 Dec 95 08:42:23 EST
• jmills@caribsurf.com
  • Re: robots on an intranet Tue, 6 Aug 1996 10:17:00 -0400
  • Re: What is your favorite search engine - a survey Tue, 6 Aug 1996 09:37:03 -0400
• Joao Moreira
  • Copyrights, let them be ! Wed, 11 Sep 1996 12:58:16 +0200
  • Re: mini-robot Tue, 20 Aug 1996 16:12:38 +0200
• Joe Nieten
  • Re: Java Robot Mon, 24 Jun 1996 14:28:52 +0700
  • Re: Java Robot Mon, 24 Jun 1996 09:24:48 +0700
  • Re: robot.polite Mon, 17 Jun 1996 11:52:26 +0700
• Joe Pruett
  • do robots send HTTP_HOST? Thu, 3 Oct 1996 09:04:56 -0700 (PDT)
• John Bro, InterSoft Solutions, Inc
  • Q: meta name="robots" content="noindex" ? Mon, 15 Jul 1996 15:40:30 -0400
• John D. Pritchard
  • Re: Junkie-Mail was Re: Standard? Mon, 02 Dec 1996 14:41:04 -0500
  • Re: Standard? Mon, 02 Dec 1996 14:35:53 -0500
  • Junkie-Mail was Re: Standard? Mon, 02 Dec 1996 12:41:38 -0500
  • legal equivalence of fax and email was Re: Standard? Mon, 02 Dec 1996 12:22:00 -0500
  • Re: make people use ROBOTS.txt? Sun, 01 Dec 1996 20:54:39 -0500
  • Re: make people use ROBOTS.txt? Sat, 30 Nov 1996 17:11:35 -0500
  • Who are you robots.txt? was Re: Servers vs Agents Fri, 29 Nov 1996 09:18:00 -0500
  • Re: Standard Wed, 27 Nov 1996 05:06:33 -0500
  • IROS 97 Call for Papers Sun, 24 Nov 1996 22:59:32 -0500
  • Re: make people use ROBOTS.txt? Sun, 24 Nov 1996 20:25:57 -0500
  • java applet sockets Sat, 23 Nov 1996 15:54:06 -0500
  • Re: Hip Crime Sat, 23 Nov 1996 13:27:13 -0500
  • Re: ActiveAgent Fri, 22 Nov 1996 18:15:49 -0500
  • Koen Holtman: Content negotiation draft 04 submitted Thu, 21 Nov 1996 19:08:08 -0500
  • Re: Broadness of Robots.txt (Re: Washington again !!!) Thu, 21 Nov 1996 18:20:06 -0500
  • Re: Broadness of Robots.txt (Re: Washington again !!!) Thu, 21 Nov 1996 18:15:15 -0500
  • Re: Broadness of Robots.txt (Re: Washington again !!!) Thu, 21 Nov 1996 17:32:28 -0500
  • Re: Notification protocol? Tue, 19 Nov 1996 16:28:32 -0500
  • Re: databases for spiders Thu, 14 Nov 1996 13:56:53 -0500
  • Re: UN/LINK protocol is standardized! wasn't that quick! Wed, 13 Nov 1996 03:09:11 -0500
  • UN/LINK protocol is standardized! wasn't that quick! Wed, 13 Nov 1996 02:40:03 -0500
  • Re: Notification protocol? Tue, 12 Nov 1996 15:38:25 -0500
  • Re: Domains and HTTP_HOST Thu, 07 Nov 1996 17:50:31 -0500
  • Re: Back of the envelope computations Wed, 06 Nov 1996 12:48:25 -0500
  • Re: Possible robots.txt addition Wed, 06 Nov 1996 11:54:10 -0500
  • Re: Possible robots.txt addition Wed, 06 Nov 1996 11:35:26 -0500
  • Re: agents ignoring robots.txt Thu, 17 Oct 1996 12:25:43 -0400
  • Re: robots.txt syntax Tue, 15 Oct 1996 07:05:30 -0400
  • Re: ActiveAgent Mon, 14 Oct 1996 12:23:23 -0400
  • Re: robots.txt syntax Mon, 14 Oct 1996 12:17:25 -0400
  • Re: ActiveAgent Mon, 14 Oct 1996 03:04:27 -0400
  • Re: robots.txt syntax Mon, 14 Oct 1996 02:58:18 -0400
  • Re: robots.txt syntax Fri, 11 Oct 1996 09:53:08 -0400
  • Re: Preferred access time Mon, 23 Sep 1996 17:43:55 -0400
  • Re: Copyrights, let them be ! Wed, 11 Sep 1996 14:34:28 -0400
  • Re: Java intelligent agents and compliance? Mon, 15 Jul 1996 12:31:43 -0400
  • Prasad Wagle: Webhackers: Java servlets and agents Wed, 10 Jul 1996 13:05:26 -0400
  • Re: robot clusion; was Re: we should help spiders and not say NO! Mon, 01 Jul 1996 11:06:00 -0400
  • robot clusion; was Re: we should help spiders and not say NO! Sat, 29 Jun 1996 10:20:38 -0400
  • Re: Advice Sat, 29 Jun 1996 09:39:52 -0400
  • loc(SOIF) Wed, 26 Jun 1996 16:06:31 -0400
  • Re: Java Robot Wed, 26 Jun 1996 10:42:45 -0400
  • Re: Inter-robot communication Tue, 25 Jun 1996 11:16:32 -0400
  • Re: Java Robot Mon, 24 Jun 1996 16:23:59 -0400
  • Re: Inter-robot communication Mon, 24 Jun 1996 13:50:30 -0400
  • Re: Java Robot Mon, 24 Jun 1996 11:45:51 -0400
  • Re: Inter-robot communication Mon, 24 Jun 1996 10:18:51 -0400
  • Re: Client Robot 'Ranjan' Wed, 19 Jun 1996 15:55:41 -0400
  • www.kollar.com/robots.html Thu, 06 Jun 1996 16:13:50 -0400
  • Re: alta vista and virtualvin.com Mon, 03 Jun 1996 12:04:05 -0400
  • Re: Defenses against bad robots Mon, 20 May 1996 13:49:27 -0400
  • Re: Defenses against bad robots Mon, 20 May 1996 13:44:11 -0400
  • Re: Defenses against bad robots Mon, 20 May 1996 13:09:04 -0400
  • Re: Somebody is turning 23! Wed, 15 May 1996 19:58:15 -0400
  • Re: Political economy of distributed search (was topical search...) Fri, 03 May 1996 11:20:55 -0400
  • Re: To: ???? Robot Mon, 29 Apr 1996 14:40:14 -0400
  • Re: To: ???? Robot Mon, 29 Apr 1996 14:17:13 -0400
  • Re: communal bots Fri, 26 Apr 1996 14:52:51 -0400
  • Re: Search accuracy Thu, 04 Apr 1996 15:09:42 -0500
  • Re: Anyone doing a Java-based robot yet? Mon, 19 Feb 1996 17:02:35 -0500
  • Re: Anyone doing a Java-based robot yet? Mon, 19 Feb 1996 14:51:54 -0500
  • Re: URL measurement studies? Fri, 16 Feb 1996 16:16:33 -0500
  • Re: Inter-robot Communications - Part II Tue, 02 Jan 1996 15:28:06 -0500
  • Re: Inter-robot Comms Port Tue, 26 Dec 1995 15:42:08 -0500
  • Re: Indexing two-byte text Wed, 06 Dec 1995 02:30:07 -0500
  • mortgages with: Re: harvest Mon, 27 Nov 1995 19:38:34 -0500
  • harvest Mon, 27 Nov 1995 15:29:38 -0500
• John Heard
  • Meta refresh tags Wed, 08 Jan 1997 00:17:32 -0600
• John Lammers
  • xxx.lanl.gov a real threat? 12 Jul 96 10:39:58 EDT
  • re: privacy, courtesy, protection 18 Jan 96 16:04:51 EST
• John Lindroth
  • Re: ActiveAgent Fri, 8 Nov 1996 14:08:00 -0500 (EST)
  • Re: Web spaces of strange topology. Where? Fri, 17 May 1996 10:37:46 -0500
  • Re: New Robot Announcement Tue, 16 Jan 1996 16:49:50 +0000
  • Re: unknown robot Mon, 8 Jan 1996 11:03:54 +0000
• John McGrath - Money Spider Ltd.
  • Money Spider WWW Robot for Windows Wed, 14 Feb 1996 06:21:20 -0700
• John Messerly
  • RE: Altavista indexing password files Fri, 1 Mar 1996 12:18:21 -0800
  • Altavista indexing password files Wed, 28 Feb 1996 19:54:52 -0800
• John W. James
  • Re: Lack of support for "If-Modified-Since" Thu, 13 Feb 1997 17:02:24 -0800
• John W. Kulp
  • Re: HTML query to .ps? .... Mon, 1 Jul 1996 10:34:59 -0400 (EDT)
  • Re: HTML query to .ps? .... Mon, 1 Jul 1996 10:34:59 -0400 (EDT)
• John_R_R_Leavitt@NL.CS.CMU.EDU
  • Re: Java and robots... Thu, 13 Feb 97 18:38:26 EST
  • Re: Unfriendly Lycos , again ... Fri, 29 Dec 95 20:00:37 EST
• Jon Knight
  • Re: Welcome to cypherpunks Wed, 12 Feb 1997 23:54:02 +0000 (GMT)
  • Re: Meta Tags only on home page ? Mon, 27 Jan 1997 14:55:15 +0000 (GMT)
• jon madison
  • Re: unscribe Mon, 20 May 1996 15:14:49 -0500
  • Re: Image Maps Mon, 20 May 1996 15:13:29 -0500
  • Re: image map traversal Mon, 20 May 1996 15:11:58 -0500
  • Re: Robots available for Intranet applications Fri, 22 Mar 1996 15:15:52 -0500
• JoongSub Lee
  • Looking for good one Sun, 16 Jun 1996 13:58:38 +0900 (KST)
• Jose Raul Vaquero Pulido
  • search engine Wed, 24 Apr 1996 08:30:20 +0100 (GMT+0100)
  • Re: algorithms too Tue, 13 Feb 1996 16:58:42 +0100 (GMT+0100)
• Josef Pellizzari
  • Re: Robots available for Intranet applications Thu, 14 Mar 1996 08:36:04 +0100 (MET)
• Joseph Whitmore
  • Re: Standard? Tue, 03 Dec 1996 19:01:59 -0800
  • Re: Standard Sat, 30 Nov 1996 23:15:08 -0800
  • Standard Tue, 26 Nov 1996 23:53:56 -0800
• joseph williams
  • Re: Robot's Book. Wed, 26 Jun 1996 12:38:52 -0600
  • Re: Robot books Mon, 17 Jun 1996 23:17:02 -0600
  • Re: Robots and search engines technical information. Sat, 17 Feb 1996 07:36:12 -0700
• Juan
  • We need robot information Fri, 08 Nov 1996 11:15:14 +0100
• Juan Jose Amor
  • [ROBOTS JJA] Lib-WWW-perl5 Wed, 13 Nov 1996 09:59:57 +0100 (MET)
• Judy Feder
  • Re: Search accuracy 4 Apr 96 7:05:32
• Julian Gorodsky
  • Re: Returned mail: Service unavailableHELP AGAIN HELP AGAIN! Tue, 5 Dec 1995 22:02:39 -0800
  • Re: Returned mail: Service unavailableHELP HELP! Tue, 5 Dec 1995 22:02:01 -0800
• Julian Smith
  • Re: Meta refresh tags Mon, 13 Jan 1997 01:04:19 GMT
• K.E. HERING
  • (no subject) Wed, 3 Jul 1996 14:22:49 GMT
  • (no subject) Wed, 3 Jul 1996 09:38:56 GMT
• Karmy T. Kays
  • Game tonight Thu, 26 Dec 1996 10:45:35 -0800
• Karoly Negyesi
  • Small robot needed Thu, 30 Nov 1995 21:29:30 +0100 (MET)
• kathy@accessone.com
  • Stop 'bots using apache, etc. or php? Wed, 31 Jul 96 09:32:48 PDT
  • PHP stops robots Mon, 29 Jul 96 08:26:41 PDT
  • Defenses against bad robots Fri, 17 May 96 13:04:47 PDT
• Keiji Kanazawa
  • RE: RFC, draft 1 Sat, 16 Nov 1996 12:05:08 -0800
  • RE: RFC, draft 1 Sat, 16 Nov 1996 09:50:12 -0800
• Keith
  • Re: Links This Site is about Robots Not Censorship Thu, 30 Mar 1995 22:28:21 +1000
• Keith D. Fischer
  • Re: About integrated search engines Fri, 26 Apr 1996 00:20:04 -0500 (CDT)
  • Re: Info on authoring a Web Robot Wed, 27 Mar 1996 09:08:09 -0600 (CST)
  • The Robot And Search Engine FAQ Tue, 13 Feb 1996 12:24:15 -0600
  • The "Robot and Search Engine FAQ" Mon, 12 Feb 1996 15:19:43 -0600
• Keith Fischer
  • Re: Wanted: Web Robot code - C/Perl Thu, 8 Feb 1996 11:20:41 -0600 (CST)
  • Re: Preliminary robot.faq (Please Send Questions or Comments) Wed, 8 Nov 1995 03:30:45 -0600
  • Preliminary robot.faq (Please Send Questions or Comments) Tue, 7 Nov 1995 00:43:47 -0600
• Ken nakagama
  • Re: email address grabber Wed, 15 Jan 1997 10:51:31 -0800
  • robots and cookies Wed, 15 May 1996 23:08:27 -0700
• Ken Wadland
  • Re: To: ???? Robot Sat, 27 Apr 1996 13:19:55 -0400
  • Re: word spam Tue, 16 Apr 1996 16:36:35 -0400
  • Re: word spam Mon, 15 Apr 1996 23:56:21 -0400
• Kenneth DeMarse
  • algorithms Mon, 12 Feb 1996 15:28:08 -0700 (MST)
• Kevin Hoogheem
  • Re: pointers for a novice? Sat, 7 Sep 96 17:57:31 -0500
  • Re: Looking for subcontracting spider-programmers Sat, 7 Sep 96 13:56:21 -0500
  • Re: nastygram from xxx.lanl.gov Thu, 11 Jul 96 15:47:56 -0500
  • Re: in-document directive to discourage indexing ? Wed, 26 Jun 96 08:45:47 -0500
  • Re: in-document directive to discourage indexing ? Tue, 25 Jun 96 09:25:38 -0500
  • Re: in-document directive to discourage indexing ? Tue, 25 Jun 96 09:17:30 -0500
  • Re: in-document directive to discourage indexing ? Mon, 24 Jun 96 13:12:18 -0500
  • Re: Client Robot 'Ranjan' Fri, 21 Jun 96 10:47:13 -0500
  • Re: Client Robot 'Ranjan' Thu, 20 Jun 96 10:29:04 -0500
  • Re: (OTP) RE: Political economy of distributed search (was topical Fri, 3 May 96 10:15:45 -0500
  • (no subject) Wed, 24 Apr 96 20:31:02 -0500
  • Re: Mailing list Mon, 22 Apr 96 21:21:09 -0500
  • Re: word spam Tue, 16 Apr 96 12:33:16 -0500
  • Re: Wanted: Web Robot code - C/Perl Thu, 8 Feb 96 10:30:42 -0600
  • Re: robots.txt , authors of robots , webmasters .... Thu, 18 Jan 96 13:22:37 -0600
  • Re: New Robot Announcement Wed, 17 Jan 96 14:20:11 -0600
  • Dearnaley Auto Reply Cannon? Fri, 15 Dec 95 13:36:13 -0600
  • Re: Checking Log files Thu, 14 Dec 95 15:11:15 -0600
  • Freely available robot code in C available? Tue, 12 Dec 95 23:54:40 -0600
• Kevin Lew
  • (no subject) Wed, 19 Jun 1996 09:42:38 +0800
• Kim Davies
  • Re: Standard? Fri, 29 Nov 1996 07:56:40 +0800
  • Re: make people use ROBOTS.txt? Sun, 24 Nov 1996 07:00:41 +0800
  • Re: make people use ROBOTS.txt? Sat, 23 Nov 1996 20:04:47 +0800
  • Re: robot defined Sat, 23 Nov 1996 06:51:11 +0800
  • Re: ActiveAgent Mon, 14 Oct 1996 13:23:00 +0800
  • Re: Proposed URLs that robots should search Tue, 24 Oct 1995 08:03:58 +0800 (WST)
  • Re: Unfriendly robot at 205.252.60.50 Sun, 22 Oct 1995 08:46:39 +0800 (WST)
• Klaus Johannes Rusch
  • Re: Information about AltaVista and Excite Mon, 10 Feb 1997 22:53:21 CET
  • Re: Question about Robot.txt Thu, 30 Jan 1997 21:28:35 CET
  • Re: Referencing dynamic pages Wed, 29 Jan 1997 17:09:17 CET
  • Re: Meta Tags only on home page ? Mon, 27 Jan 1997 16:10:58 CET
  • Re: email address grabber Tue, 14 Jan 1997 22:13:15 CET
  • Re: Lycos Tue, 7 Jan 1997 19:56:09 CET
  • Re: Crawlers and "dynamic" urls Tue, 10 Dec 1996 18:13:18 CET
  • Re: stingy yahoo server? Mon, 9 Dec 1996 16:17:20 CET
  • Re: stingy yahoo server? Sat, 7 Dec 1996 17:34:47 CET
  • Re: stingy yahoo server? Sat, 7 Dec 1996 15:06:09 CET
  • Re: Just when you thought it might be interesting to standardize robots.txt... Wed, 4 Dec 1996 23:26:42 CET
  • Re: USER_AGENT and Apache 1.2 Tue, 26 Nov 1996 16:58:50 CET
  • Re: Lycos' HEAD vs. GET Sat, 23 Nov 1996 23:47:56 CET
  • Re: RFC, draft 1 Thu, 21 Nov 1996 22:50:58 CET
  • Re: User-Agent Wed, 20 Nov 1996 23:56:14 CET
  • Re: RFC, draft 1 Wed, 20 Nov 1996 00:07:49 CET
  • Re: Virtual (was: RFC, draft 1) Tue, 19 Nov 1996 23:00:15 CET
  • Re: RFC, draft 1 Mon, 18 Nov 1996 22:24:46 CET
  • Re: changes to robots.txt Tue, 12 Nov 1996 21:48:28 CET
  • Re: Get official! Mon, 11 Nov 1996 20:28:14 CET
  • Re: Domains and HTTP_HOST Fri, 8 Nov 1996 23:38:15 CET
  • Re: Domains and HTTP_HOST Thu, 7 Nov 1996 16:04:23 CET
  • Re: Possible robots.txt addition Thu, 7 Nov 1996 16:38:07 CET
  • Re: Possible robots.txt addition Wed, 6 Nov 1996 21:59:00 CET
  • Re: Domains and HTTP_HOST Wed, 6 Nov 1996 22:29:11 CET
  • Re: Is a robot visiting? Sun, 3 Nov 1996 14:31:31 CET
  • Re: Is a robot visiting? Fri, 1 Nov 1996 12:16:14 CET
• Kosta Tombras
  • Robot Research Wed, 17 Jul 1996 09:50:41 -0400 (EDT)
• L a r r y P a g e
  • Re: web topology Wed, 26 Jun 1996 20:03:32 -0700
  • Re: Java Robot Tue, 25 Jun 1996 14:32:07 -0700
• L a r r y P a g e
  • BackRub robot Thu, 13 Jun 1996 11:44:13 -0400
• Lance Ogletree
  • MacPower Fri, 01 Dec 1995 22:51:42 +0000
• Larry Burke
  • Re: Re: robots.txt , authors of robots , webmasters .... Thu, 18 Jan 1996 10:14:55 -0800
  • Re: New Robot Announcement Tue, 16 Jan 1996 11:42:17 -0800
  • Re: New Robot Announcement Mon, 15 Jan 1996 15:56:21 -0800
  • New Robot Announcement Mon, 15 Jan 1996 13:38:29 -0800
• Larry Fitzpatrick
  • RE: Notification protocol? Mon, 18 Nov 1996 07:59:38 -0500
  • RE: databases for spiders Fri, 15 Nov 1996 11:23:45 -0500
• Larry Gilbert
  • Re: nastygram from xxx.lanl.gov Tue, 9 Jul 1996 10:55:08 -0800
  • Re: alta vista and virtualvin.com Fri, 17 May 1996 16:09:51 -0800
  • Re: Defenses against bad robots Fri, 17 May 1996 15:18:04 -0800
• Larry Steinberg
  • ActiveAgent Mon, 14 Oct 1996 10:59:12 -0400
• Larry Stephen Burke
  • Proxies Tue, 9 Jul 96 12:42:03 -0000
  • (no subject) Mon, 10 Jun 96 13:55:24 -0000
• Laura M. Sebastian
  • Forsberg Thu, 26 Dec 1996 11:55:12 -0800
• Lee Fisher
  • Re: IIS and If-modified-since Wed, 11 Dec 1996 09:23:30 -0800
  • Microsoft Tripoli Web Search Beta now available Tue, 18 Jun 1996 14:44:09 -0700
  • Announce: ActiveX Search (IFilter) spec/sample Wed, 27 Mar 1996 15:31:44 -0800
• Leigh DeForest Dupee
  • Re: Unfriendly robot at 205.177.10.2 Wed, 18 Oct 1995 15:34:04 -0400
• Leslie Cuff
  • Re: Introducing myself Tue, 18 Jun 1996 10:49:37 -0230
  • Re: robot.polite Tue, 18 Jun 1996 10:46:22 -0230
  • Re: robot.polite Tue, 18 Jun 1996 10:32:26 -0230
  • robots.txt: allow directive Fri, 14 Jun 1996 08:39:59 -0230
• levitte@lp.se
  • Re: robots.txt unavailability Tue, 09 Jul 1996 17:51:14 +0200
• LIAM GUINANE
  • unix robot Wed, 27 Nov 1996 18:50:20 GMT
  • technical descripton Wed, 6 Nov 1996 12:05:24 GMT
  • download robot Mon, 4 Nov 1996 20:19:56 GMT
• Lilian Bartholo
  • Re: Remember Canseco..... Thu, 26 Dec 1996 10:00:21 -0800
• listproc@plaidworks.com
  • Error Condition Re: Invalid request Thu, 26 Dec 1996 12:22:40 -0800
  • Error Condition Re: Invalid request Thu, 26 Dec 1996 12:10:56 -0800
• Louis Monier
  • RE: On the subject of abuse/pro-activeness Tue, 9 Jul 1996 11:41:34 -0700
  • RE: robots.txt unavailability Tue, 9 Jul 1996 11:29:12 -0700
  • RE: alta vista and virtualvin.com Mon, 3 Jun 1996 11:14:47 -0700
  • RE: alta vista and virtualvin.com Sun, 2 Jun 1996 10:50:55 -0700
  • RE: alta vista and virtualvin.com Sat, 1 Jun 1996 20:21:13 -0700
  • RE: alta vista and virtualvin.com Sat, 1 Jun 1996 20:21:17 -0700
  • RE: any robots/search-engines which index links? Wed, 8 May 1996 07:54:30 -0700
• Luiz Fernando
  • Re: unknown robot Fri, 29 Dec 1995 14:43:48 -0400
• M.Levy@cs.ucl.ac.uk
  • Re: Does this count as a robot? Mon, 08 Jan 96 18:41:18 +0000
  • Re: [5]RE>Checking Log files Fri, 15 Dec 95 09:55:02 +0000
• mabzug1@gl.umbc.edu
  • Re: Recursion Tue, 9 Jan 1996 16:12:37 -0500 (EST)
• Mail Delivery Subsystem
  • Returned mail: User unknown Thu, 26 Dec 1996 14:10:43 -0500
  • Returned mail: Host unknown (Name server: webcrawler: host not found) Mon, 14 Oct 1996 14:51:05 +0100
  • Returned mail: Host unknown (Name server: webcrawler: host not found) Mon, 14 Oct 1996 14:51:05 +0100
  • Returned mail: Can't create output: Error 0 Wed, 17 Apr 1996 13:05:00 +0900 (JST)
• Majordomo@toad.com
  • Welcome to cypherpunks Wed, 12 Feb 1997 08:44:22 -0800 (PST)
• mannina bruno
  • How to get the document info ? Fri, 14 Feb 1997 12:23:33 +0000
  • Re: Announcement Thu, 4 Jul 1996 12:26:19 -0700
  • Re: Robot Mirror with Username/Password feature Wed, 3 Jul 1996 11:17:44 -0700
  • Robot's Book. Wed, 26 Jun 1996 15:27:58 -0700
  • Recherche de documentation sur les agents intelligents ou Robots. Fri, 5 Apr 1996 17:51:42 -0700
• mannina@crrm.univ-mrs.fr
  • Re:Re: Perl Spiders Fri, 9 Feb 1996 11:39:30 +0200
  • Re: Server name in /robots.txt Thu, 17 Aug 1995 13:18:40 +0200
• Manuel J. Kwak
  • Java and robots... Thu, 13 Feb 1997 18:33:18 +0100
• Manuel Jesus Fernandez Blanco
  • Thanks! Fri, 14 Feb 1997 15:07:21 +0100
• Marc Eisenstadt
  • ANNOUNCE: Don Norman (Apple) LIVE! 15-May 5PM UK = noon EDT Thu, 9 May 1996 12:19:41 +0100
• Marc Langheinrich
  • RE: Search Engine Tue, 13 Aug 1996 11:00:11 -0700 (PDT)
• Marc's internet diving suit
  • VB. page grabber... Mon, 6 May 1996 12:54:33 +-200
• Marco Genua
  • robot vaiable list Tue, 18 Jun 1996 14:24:47 +0200
• marco@specialnet.cmt.it
  • Robot logic? Tue, 18 Jun 1996 10:44:04 +0200
• Marilyn R Wulfekuhler
  • Re: The Internet Archive robot Mon, 9 Sep 1996 10:56:30 -0400 (EDT)
  • Re: Looking for a spider Fri, 1 Dec 95 10:33:28 EST
  • Re: Looking for a spider Mon, 23 Oct 95 10:50:03 EDT
• Mark Fullerton
  • RE: shot clock?!.... Thu, 26 Dec 1996 11:31:40 -0800
• Mark J Cox
  • Re: Suggestion to help robots and sites coexist a little better Mon, 15 Jul 1996 08:57:45 +0100 (BST)
• Mark Krell
  • New URL's from Equity Int'll Webcenter Mon, 29 Jan 1996 00:48:02 -0500
• Mark Norman
  • stingy yahoo server? Fri, 06 Dec 1996 17:36:47 PST
  • http directory index request Tue, 16 Apr 1996 10:32:53 -0700
  • image map traversal Mon, 25 Mar 1996 08:29:43 -0800
  • Can I retrieve image map files? Sun, 10 Mar 1996 18:09:28 -0800
  • test. please ignore. Fri, 26 Jan 1996 07:50:00 -0800
  • Re: Does anyone else consider... Wed, 10 Jan 1996 18:43:09 -0800 (PST)
  • Re: Does anyone else consider this irresponsible? Wed, 10 Jan 1996 11:28:55 -0800 (PST)
• Mark Schrimsher
  • Re: Looking for a search engine Wed, 1 May 1996 13:40:54 +0900
  • Re: BSE-Slurp/0.6 Mon, 11 Mar 1996 13:08:17 +0900
  • Re: Alta Vista searches WHAT?!? Wed, 17 Jan 1996 12:00:59 +0900
  • Re: Spam Software Sought Sun, 14 Jan 1996 09:04:30 +0900
  • Re: Does anyone else consider... Thu, 11 Jan 1996 15:02:50 +0900
  • Re: test; please ignore Tue, 19 Dec 1995 12:06:43 +0900
  • [5]RE>Checking Log files Fri, 15 Dec 1995 15:23:30 +0900
  • Re: Checking Log files Fri, 15 Dec 1995 11:11:57 +0900
  • Re: Harvest question Thu, 14 Dec 1995 10:05:47 +0900
  • Re: Freely available robot code in C available? Wed, 13 Dec 1995 15:21:33 +0900
  • RE: Indexing two-byte text Sat, 9 Dec 1995 13:16:43 +0900
  • Re: Indexing two-byte text Fri, 8 Dec 1995 19:19:15 +0900
  • Indexing two-byte text Fri, 8 Dec 1995 16:51:06 +0900
  • Re: Indexing two-byte text Fri, 8 Dec 1995 05:45:10 +0900
  • Re: Indexing two-byte text Thu, 7 Dec 1995 22:15:48 +0900
  • Re: Indexing two-byte text Thu, 7 Dec 1995 11:18:57 +0900
• Mark Slabinski
  • Re: Robots available for Intranet applications Mon, 11 Mar 96 18:18:23 EST
• Martijn De Boef
  • url locating Sat, 20 Jan 1996 00:11:52 -0100
• Martijn Koster
  • Re: AMDIN: The list is dead Tue, 25 Mar 1997 09:51:58 -0800 (PST)
  • AMDIN: The list is dead Tue, 25 Mar 1997 08:23:52 -0800 (PST)
  • Re: More with the Cypherpunk antics Wed, 12 Feb 1997 22:17:31 -0800 (PST)
  • NaughtyRobot Fri, 7 Feb 1997 10:43:10 -0800 (PST)
  • Re: Meta Tags Tue, 21 Jan 1997 14:15:54 -0800 (PST)
  • Re: Meta refresh tags Wed, 8 Jan 1997 09:54:19 -0800 (PST)
  • Re: help Mon, 6 Jan 1997 07:40:25 -0800 (PST)
  • Re: ADMIN: mailing list attack :-( Fri, 27 Dec 1996 12:45:48 -0800
  • ADMIN: mailing list attack :-( Thu, 26 Dec 1996 12:28:51 -0800
  • Re: RFC, draft 1 Tue, 24 Dec 1996 08:28:43 -0800
  • Re: Crawlers and "dynamic" urls Mon, 9 Dec 1996 18:35:54 -0800
  • Re: Server Indexing -- Helping a Robot Out Thu, 5 Dec 1996 08:30:50 -0800
  • Please take Uninvited Email discussion elsewhere Wed, 4 Dec 1996 09:39:06 -0800
  • Re: USER_AGENT and Apache 1.2 Tue, 26 Nov 1996 19:48:17 -0800
  • Re: defining "robot" Mon, 25 Nov 1996 20:32:56 -0800
  • Re: defining "robot" Mon, 25 Nov 1996 08:10:36 -0800
  • Re: defining "robot" Sat, 23 Nov 1996 08:54:26 -0800
  • Re: RFC, draft 1 Fri, 22 Nov 1996 08:14:32 -0800
  • Re: who/what uses robots.txt Fri, 22 Nov 1996 08:14:29 -0800
  • Re: robots.txt Thu, 21 Nov 1996 12:54:03 -0800
  • RFC, draft 2 (was Re: RFC, draft 1 Thu, 21 Nov 1996 12:54:16 -0800
  • Re: RFC, draft 1 Thu, 21 Nov 1996 12:54:13 -0800
  • Re: who/what uses robots.txt Thu, 21 Nov 1996 12:53:56 -0800
  • Re: Broadness of Robots.txt (Re: Washington again !!!) Thu, 21 Nov 1996 12:54:06 -0800
  • Agent Categories (was Re: Broadness of Robots.txt (Re: Washington again !!!) Thu, 21 Nov 1996 12:54:09 -0800
  • Re: Broadness of Robots.txt (Re: Washington again !!!) Thu, 21 Nov 1996 12:53:59 -0800
  • Re: Washington again !!! Wed, 20 Nov 1996 07:52:10 -0800
  • Re: RFC, draft 1 Tue, 19 Nov 1996 11:29:40 -0800
  • Re: RFC, draft 1 Tue, 19 Nov 1996 11:29:52 -0800
  • Re: RFC, draft 1 Tue, 19 Nov 1996 11:29:33 -0800
  • Re: RFC, draft 1 Tue, 19 Nov 1996 11:29:46 -0800
  • RE: RFC, draft 1 Tue, 19 Nov 1996 11:29:27 -0800
  • Hipcrime no more Sun, 17 Nov 1996 19:40:27 -0800
  • Re: An extended version of the Robots... Fri, 15 Nov 1996 17:02:46 -0800
  • Re: how do they do that? Fri, 15 Nov 1996 17:02:43 -0800
  • RFC, draft 1 Fri, 15 Nov 1996 17:02:39 -0800
  • Re: An extended version of the Robots... Tue, 12 Nov 1996 16:27:07 -0800
  • Re: Regexps (Was: Re: An extended version of the Robots...) Mon, 11 Nov 1996 11:09:42 -0800
  • Re: An extended version of the Robots... Fri, 8 Nov 96 17:13:17 -0800
  • Re: Possible robots.txt addition Wed, 6 Nov 1996 19:39:50 -0800
  • ADMIN (was Re: hypermail archive not operational Sun, 13 Oct 1996 17:49:27 -0700
  • Re: Another rating scam! (And a proposal on how to fix it) Sat, 12 Oct 1996 11:09:10 -0700
  • Re: robots.txt syntax Sat, 12 Oct 1996 11:09:07 -0700
  • Re: robots.txt syntax Fri, 11 Oct 1996 09:18:54 -0700
  • Re: Robot exclustion for for non-'unix file' hierarchy Mon, 30 Sep 1996 17:03:43 -0700
  • Re: file retrieval Mon, 23 Sep 1996 10:19:01 -0700
  • Re: The Internet Archive robot Mon, 23 Sep 1996 08:51:01 -0700
  • Re: "hidden text" vs. META tags for robots/search engines Sun, 8 Sep 1996 12:10:19 -0700
  • Re: robots on an intranet (replies to list...) Tue, 13 Aug 1996 08:07:25 -0700
  • Re: Suggestion to help robots and sites coexist a little better Thu, 18 Jul 1996 12:27:31 -0700
  • Re: Suggestion to help robots and sites coexist a little better Mon, 15 Jul 1996 11:06:19 -0700
  • ADMIN: Spoofing vs xxx.lanl.gov Thu, 11 Jul 1996 12:32:43 -0700
  • Re: Should I index all ... Mon, 8 Jul 1996 09:18:43 -0700
  • Re: Description or Abstract? Fri, 5 Jul 1996 08:23:12 -0700
  • Re: Description or Abstract? Thu, 4 Jul 1996 12:21:09 -0700
  • Re: hey man gimme a break Wed, 3 Jul 1996 11:26:30 -0700
  • Re: Advice Sat, 29 Jun 1996 13:37:29 -0700
  • Re: robot clusion; was Re: we should help spiders and not say NO! Sat, 29 Jun 1996 13:37:34 -0700
  • The Web Robots Database (was Re: robot vaiable list Wed, 26 Jun 1996 17:50:13 -0700
  • ADMIN: Archive Wed, 26 Jun 1996 17:50:08 -0700
  • ADMIN: unsubscribing (was: Re: Wed, 26 Jun 96 08:17:07 -0700
  • Report of the Distributed Indexing/Searching Workshop Mon, 24 Jun 1996 06:18:28 -0700
  • Re: in-document directive to discourage indexing ? Wed, 19 Jun 1996 20:02:20 -0700
  • Re: robot.polite Wed, 19 Jun 1996 15:40:55 -0700
  • Re: Introducing myself Tue, 18 Jun 1996 08:46:38 -0700
  • Re: robot vaiable list Tue, 18 Jun 1996 08:46:30 -0700
  • Re: Robot logic? Tue, 18 Jun 1996 08:46:26 -0700
  • Re: WebAnalyzer - introduction Tue, 18 Jun 1996 08:46:34 -0700
  • (no subject) Tue, 18 Jun 1996 08:09:24 -0700
  • Re: Introducing myself Tue, 18 Jun 1996 08:09:20 -0700
  • Re: Keyword indexing Tue, 18 Jun 1996 06:35:08 -0700
  • Re: robot.polite Mon, 17 Jun 1996 21:20:14 -0700
  • Re: Introducing myself Mon, 17 Jun 1996 21:20:09 -0700
  • Re: BackRub robot Mon, 17 Jun 1996 21:20:24 -0700
  • Re: robot.polite Mon, 17 Jun 1996 21:20:19 -0700
  • Re: Looking for News robot Mon, 17 Jun 1996 21:20:04 -0700
  • Re: Content based search engine Mon, 10 Jun 1996 11:31:35 -0700
  • Re: Robot Exclusion Standard Revisited (LONG) Tue, 4 Jun 1996 09:43:41 -0700
  • Re: Accept: Fri, 17 May 1996 17:47:42 -0700
  • Re: Defenses against bad robots Fri, 17 May 1996 17:47:47 -0700
  • Admin: List archive is back Tue, 30 Apr 1996 10:55:13 -0700
  • Re: Problem with your Index Mon, 8 Apr 1996 18:49:09 -0700
  • Admin: how to get off this list Sat, 30 Mar 1996 18:18:52 -0700
  • Re: Links Mon, 25 Mar 1996 20:07:33 -0700
  • Re: HEAD request [was Re: Server name in /robots.txt] Fri, 26 Jan 1996 18:20:24 -0700
  • Re: Server name in /robots.txt Mon, 22 Jan 1996 05:59:21 -0700
  • Re: Server name in /robots.txt Fri, 19 Jan 1996 12:28:03 -0700
  • BOUNCE robots: Admin request Tue, 16 Jan 1996 13:35:17 -0700
  • Re: Alta Vista searches WHAT?!? Tue, 16 Jan 1996 13:25:45 -0700
  • Robots / source availability? Wed, 10 Jan 1996 14:05:36 -0700
  • Re: robots.txt extensions Wed, 10 Jan 1996 14:05:30 -0700
  • FAQ again. Wed, 10 Jan 1996 12:09:45 -0700
  • Re: please add my site Mon, 1 Jan 1996 09:56:20 -0700
  • RE: Inter-robot Communications - Part II Sat, 30 Dec 1995 12:02:24 -0700
  • Re: Inter-robot Communications - Part II Fri, 29 Dec 1995 08:26:27 -0700
  • test; please ignore Mon, 18 Dec 1995 14:59:15 -0800
  • Vacation wars Fri, 15 Dec 1995 15:25:39 -0800
  • Re: Announcement and Help Requested Fri, 15 Dec 1995 16:38:28 -0500
  • Re: How frequently should I check /robots.txt? Mon, 27 Nov 1995 23:22:54 -0800
  • Re: How frequently should I check /robots.txt? Mon, 27 Nov 1995 23:22:54 -0800
  • Patents? Wed, 25 Oct 1995 16:22:18 -0700
  • Re: lycos patents Wed, 25 Oct 1995 15:58:13 -0700
  • Re: Proposed URLs that robots should search Mon, 23 Oct 1995 16:31:17 -0700
  • Re: Site Announcement Mon, 16 Oct 1995 08:25:16 -0700
  • The robots mailing list at WebCrawler Thu, 12 Oct 1995 14:39:19 -0700
• Martin Beet
  • Re: Crawling & DNS issues Wed, 22 Jan 1997 09:19:41 +0000
• Martin Hamilton
  • Re: Single tar (Re: Inktomi & large scale spidering) Tue, 11 Feb 1997 18:44:22 +0000
  • Re: Single tar (Re: Inktomi & large scale spidering) Mon, 27 Jan 1997 14:09:32 +0000
  • Re: Inktomi & large scale spidering Sun, 26 Jan 1997 12:37:04 +0000
  • Re: Info on large scale spidering? Fri, 24 Jan 1997 16:50:06 +0000
• Martin Kiff
  • Re: Standard? Mon, 2 Dec 1996 07:43:14 +0100
  • Re: Servers vs Agents Thu, 28 Nov 1996 06:20:25 +0100
  • Re: defining "robot" Sun, 24 Nov 1996 07:03:59 +0100
  • Re: defining "robot" Fri, 22 Nov 1996 22:09:22 +0100
  • Possible robots.txt addition - did I say that? Wed, 6 Nov 1996 21:25:30 +0100
  • Re: Possible robots.txt addition Wed, 6 Nov 1996 07:26:36 +0100
  • Re: "hidden text" vs. META tags for robots/search engines Fri, 20 Sep 1996 07:31:07 +0200
  • Re: "hidden text" vs. META tags for robots/search engines Sat, 14 Sep 1996 06:55:50 +0200
  • Re: How long to cache robots.txt for? Wed, 10 Jul 1996 21:13:37 +0200
  • Re: Alta Vista getting stale? Wed, 10 Jul 1996 21:11:42 +0200
  • Heuristics.... Sat, 30 Mar 1996 14:19 BST
  • Re: Links (don't bother checking; I've done it for you) Fri, 29 Mar 1996 20:31 UT
  • Re: Image Maps Wed, 27 Mar 1996 10:54 UT
  • Re: robots.txt changes how often? Fri, 16 Feb 1996 19:13 UT
  • Activity from 205.252.60.5[0-8] Wed, 24 Jan 1996 09:41 UT
  • Re: Server name in /robots.txt Mon, 22 Jan 1996 09:16 UT
  • Server name in /robots.txt Fri, 19 Jan 1996 10:32 UT
  • Re: robots.txt extensions Thu, 11 Jan 1996 09:44 UT
• Martin Paff
  • an image observer Thu, 16 Jan 1997 14:47:35 -0500
  • indexing intranet-site Fri, 15 Nov 1996 10:17:03 -0500
• Martin.Soukup
  • WININET caching Wed, 12 Feb 1997 10:31:33 -0500
  • RE: indexing via redirectors Tue, 21 Jan 1997 10:15:51 -0500
  • RE: Web pages being served from an SQL database Thu, 19 Dec 1996 14:52:37 -0500
  • RE: Broadness of Robots.txt (Re: Washington again !!!) Fri, 29 Nov 1996 16:54:56 -0500
  • robot to get specific info only? Tue, 24 Sep 1996 16:45:07 +1111
  • RE: Preferred access time Tue, 24 Sep 1996 12:42:07 -0400
  • Preferred access time Mon, 23 Sep 1996 16:28:35 -0400
  • RE: Anyone know who owns this one? Fri, 2 Aug 1996 17:38:36 -0400
• Marty Landman
  • tryme Fri, 06 Sep 96 19:47:32 US/Eastern
• Matthew Gray
  • Web Robot Sat, 6 Jan 1996 09:36:04 -0500 (EST)
  • Re: lycos patents Wed, 25 Oct 1995 19:08:47 -0400 (EDT)
• Matthew K Gray
  • Re: not a robot 22 Nov 1996 11:19:41 -0500
  • Re: defining "robot" 22 Nov 1996 00:39:41 -0500
  • Re: infoseeks robot is dumb 15 Nov 1996 14:10:21 -0500
• Micah A. Williams
  • Re: How long to cache robots.txt for? Tue, 9 Jul 96 22:53:33 EDT
  • Re: default documents Mon, 22 Apr 96 12:07:45 EDT
  • Dearnaley Auto Reply Cannon? Fri, 15 Dec 95 14:32:05 EST
  • Re: [2]RE>[5]RE>Checking Log fi Fri, 15 Dec 95 13:19:50 EST
  • Re: [3]RE>[5]RE>Checking Log fi Fri, 15 Dec 95 14:11:30 EST
  • Re: [2]RE>[5]RE>Checking Log fi Fri, 15 Dec 95 13:19:50 EST
• Michael Carnevali, Student, FHD
  • Re: Robots in the client? Wed, 24 Apr 1996 07:40:22 +0200
• Michael De La Rue
  • Bug in LibWWW perl + Data::Dumper (libwwwperl refs are strange) Thu, 03 Oct 1996 14:56:11 +0200
  • Re: Bye Bye HyperText: The End of the World (Wide Web) As We Know It! Thu, 03 Oct 1996 14:27:08 +0200
  • Re: robots on an intranet (replies to list...) Tue, 13 Aug 1996 09:24:30 +0200
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 13:49:03 +0200
  • RE: alta vista and virtualvin.com Fri, 28 Jun 1996 16:12:22 +0200 (MDT)
  • Re: Web spaces of strange topology. Where? Fri, 17 May 1996 14:35:21 +0200 (MDT)
  • Web spaces of strange topology. Where? Sat, 11 May 1996 16:24:51 +0200 (MDT)
  • (no subject) Sat, 11 May 1996 16:04:30 +0200 (MDT)
  • Re: Links This Site is about Robots Not Censorship Fri, 29 Mar 1996 15:40:33 +0100 (MET)
  • Re: Links (don't bother checking; I've done it for you) Fri, 29 Mar 1996 10:56:12 +0100 (MET)
  • Re: robot authentication parameters Sat, 16 Mar 1996 18:31:42 +0100 (MET)
  • Canonical Names for documents (was Re: Server name in /robots.txt) Thu, 15 Feb 1996 11:40:41 +0100 (MET)
  • Re: Commercial Robot Vendor Recoomendations Request Thu, 15 Feb 1996 10:43:22 +0100 (MET)
• Michael G=?iso-8859-1?Q?=F6ckel
  • Re: The Internet Archive robot Fri, 20 Sep 1996 20:47:21 +0100
  • Re: robots.txt unavailability Tue, 09 Jul 1996 13:21:51 +0100
  • Re: Should I index all ... Mon, 08 Jul 1996 16:24:20 +0100
  • Re: Should I index all ... Fri, 05 Jul 1996 11:59:31 +0100
  • Announcement Wed, 03 Jul 1996 16:20:21 +0100
  • Announcement Wed, 03 Jul 1996 16:20:21 +0100
• Michael Goldberg
  • Looking for News robot Fri, 14 Jun 1996 13:04:40 -0400
  • Re: harvest Mon, 27 Nov 1995 17:55:18 -0800
  • Smart Agent help Sat, 25 Nov 1995 13:19:03 -0500
• Michael Göckel
  • Re: Standard Wed, 27 Nov 1996 10:34:51 +0100
• Michael Schlindwein
  • Re: About the question what a robots is (was Re: PS) Thu, 18 Jul 1996 17:54:20 +0200 (MET DST)
  • About the question what a robots is (was Re: PS) Tue, 16 Jul 1996 14:35:04 +0200 (MET DST)
  • Re: dumb robots and xxx Fri, 12 Jul 1996 22:42:46 +0200 (MET DST)
  • You found it... (was Re: nastygram from xxx.lanl.gov) Fri, 12 Jul 1996 00:03:16 +0200 (MET DST)
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 19:08:18 +0200 (MET DST)
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 22:51:11 +0200 (MET DST)
  • Re: nastygram from xxx.lanl.gov Tue, 9 Jul 1996 15:13:16 +0200 (MET DST)
• Michael Van Biesbrouck
  • Re: alta vista and virtualvin.com Wed, 12 Jun 1996 12:19:26 -0400 (EDT)
  • Re: McKinley Spider hit us hard Tue, 5 Dec 1995 15:56:33 -0500 (EST)
• Michael. Gunn
  • Re: Announcement Tue, 9 Jul 1996 16:40:03 +0100 (BST)
• Mike Agostino
  • Re: How long to cache robots.txt Thu, 18 Jul 1996 12:04:21 -0700
  • Re: Description or Abstract? Thu, 4 Jul 1996 10:27:37 -0700
• Mike Burner
  • Re: indexing via redirectors Tue, 21 Jan 1997 10:42:25 -0800
  • The Internet Archive robot Thu, 05 Sep 1996 20:46:38 -0700
• Mike Fresener
  • specialized searches Thu, 6 Feb 1997 09:53:50 -0700 (MST)
• mike mulligan
  • Re: Lack of support for "If-Modified-Since" Fri, 14 Feb 1997 10:10:43 -0500
  • Good HREFs vs Bogus HREFs: 80/20 Thu, 26 Dec 1996 13:05:11 -0500
• Mike Rodriguez
  • Problem with your Index Mon, 08 Apr 1996 20:19:46 -0400
• Mike Schwartz
  • Re: Notification protocol? Sat, 16 Nov 1996 10:41:26 -0800
• Mitch Allen
  • Re: robots & copyright law Mon, 27 Jan 1997 23:04:34 -0500
  • AltaVista Meta Tag Rumour Mon, 20 Jan 1997 22:51:16 -0500
  • Re: SpamBots Thu, 16 Jan 1997 20:59:11 -0500
  • SpamBots Thu, 16 Jan 1997 07:41:11 -0500
  • Meta Tag Article Wed, 15 Jan 1997 07:00:53 -0500
  • Re: Frames ? Lycos ? Wed, 08 Jan 1997 08:50:02 -0500
  • Re: Frames ? Lycos ? Tue, 07 Jan 1997 21:56:11 -0500
• Mitchell Elster
  • Re: To: ???? Robot Mon, 29 Apr 1996 18:38:13
  • Re: To: ???? Robot Sat, 27 Apr 1996 23:38:06
  • To: ???? Robot Sat, 27 Apr 96 08:56:19 -0700
  • RE: Mailing List Mon, 22 Apr 1996 23:16:00
  • VB and robot development Sat, 06 Apr 1996 23:04:15
• Mordechai T. Abzug
  • Re: Robot-HTML Web Page? Sat, 25 May 1996 22:52:54 -0400 (EDT)
  • Re: Defenses against bad robots Fri, 17 May 1996 19:33:53 -0400 (EDT)
  • Re: Mailing list Tue, 23 Apr 1996 01:08:38 -0400 (EDT)
  • Re: Mailing list Mon, 22 Apr 1996 18:09:52 -0400 (EDT)
  • Re: http directory index request Tue, 16 Apr 1996 15:41:35 -0400 (EDT)
  • Re: Links Thu, 28 Mar 1996 22:19:43 -0500 (EST)
  • "What's new" in web pages is not necessarily reliable Mon, 11 Mar 1996 22:24:27 -0500
  • Re: BSE-Slurp/0.6 Sun, 3 Mar 1996 11:54:14 -0500 (EST)
  • Re: Server name in /robots.txt Sat, 20 Jan 1996 21:00:16 -0500 (EST)
  • Re: New Robot Announcement Mon, 15 Jan 1996 18:13:01 -0500 (EST)
  • Re: Horror story Mon, 15 Jan 1996 15:06:04 -0500 (EST)
  • Re: MD5 in HTTP headers - where? Wed, 10 Jan 1996 14:20:46 -0500 (EST)
  • Re: Inter-robot Communications - Part II Fri, 29 Dec 1995 15:09:51 -0500 (EST)
  • Announcing NaecSpyr, a new. . . robot? Fri, 15 Dec 1995 11:46:48 -0500 (EST)
• MPMC-Manhattan Premed Council & Hunter PBPMA-Post Bac PreMed Assoc 212-843-3701 Ext 2800
  • NCSA Net Access_log Analysis Tool for Win95 Tue, 09 Jul 1996 16:09:54 -0400
  • NCSA Net Access_log Analysis Tool for Win95 Tue, 09 Jul 1996 15:26:56 -0400
  • PERL Compilers & Interpretive Tools Tue, 18 Jun 1996 13:25:03 -0400
• Mr David A Weeks
  • Handling keyword repetitions Thu, 11 Apr 96 7:57:31 BST
  • Re: Anyone doing a Java-based robot yet?6 Wed, 21 Feb 96 14:55:45 GMT
• mred@neosoft.com
  • Re: Defenses against bad robots Sun, 19 May 1996 21:46:28 CST
  • Re: Accept: Fri, 17 May 1996 23:03:23 CST
  • Re: Defenses against bad robots Fri, 17 May 1996 23:03:22 CST
  • Re: Defenses against bad robots Fri, 17 May 1996 23:03:19 CST
  • Re: Web spaces of strange topology. Where? Fri, 17 May 1996 23:03:21 CST
  • Re: default documents Tue, 23 Apr 1996 22:47:28 CST
  • Re: VB and robot development Sun, 7 Apr 1996 03:37:12 CST
  • Re: Search accuracy Mon, 1 Apr 1996 21:03:07 CST
• mschrimsher@twics.com
  • Robot on the Rampage Fri, 15 Dec 1995 00:54:17 +0900
• murray bent
  • Re: robots that index comments Tue, 26 Mar 96 10:56:28 -0800
  • Re: Horror story Sun, 14 Jan 96 11:07:40 -0800
  • Unfriendly Lycos , again ... Thu, 21 Dec 95 19:29:36 -0800
  • re: Lycos patents Thu, 26 Oct 1995 09:16:39 +1000
  • lycos patents Thu, 26 Oct 1995 08:48:57 +1000
  • re: Lycos unfriendly robot Fri, 20 Oct 1995 02:18:16 +1000
• Myles Olson
  • Re: Lead Time Mon, 6 May 1996 08:18:18 +0000
• Myles Weissleder
  • i need a bot! Tue, 11 Feb 1997 18:06:05 -0500 (EST)
• Neil Cotty
  • Re: Crawling & DNS issues Wed, 22 Jan 1997 15:52:47 +1100
  • Crawling & DNS issues Wed, 22 Jan 1997 11:59:30 +1100
• Netmode
  • RCPT: Re: Robot books Tue, 18 Jun 1996 13:02:15 -0900
• Nick Arnett
  • Re: Need help again. Wed, 29 Jan 1997 16:04:22 -0800
  • Re: Need help on Search Engine accuracy test. Wed, 29 Jan 1997 13:12:56 -0800
  • Re: robots & copyright law Mon, 27 Jan 1997 16:35:32 -0800
  • Re: Inktomi & large scale spidering Sun, 26 Jan 1997 10:26:31 -0800
  • Re: Info on large scale spidering? Tue, 21 Jan 1997 09:05:58 -0800
  • Re: More Robot Talk Fri, 17 Jan 1997 08:47:48 -0800
  • Re: Do robots have to follow links ? Mon, 06 Jan 1997 18:52:21 -0800
  • Re: Netscape-Catalog-Robot Tue, 24 Dec 1996 17:31:13 -0800
  • Re: WebCrawler & Excite Wed, 04 Dec 1996 10:05:57 -0800
  • Re: WebCrawler & Excite Tue, 03 Dec 1996 20:29:32 -0800
  • Re: Standard? Mon, 02 Dec 1996 10:26:50 -0800
  • Re: make people use ROBOTS.txt? Sat, 30 Nov 1996 20:35:12 -0800
  • Re: http://HipCrime.com Mon, 18 Nov 1996 10:06:51 -0800
  • Re: indexing intranet-site Sat, 16 Nov 1996 21:11:31 -0800
  • RE: databases for spiders Fri, 15 Nov 1996 08:42:17 -0800
  • Re: indexing intranet-site Fri, 15 Nov 1996 08:22:26 -0800
  • Re: databases for spiders Fri, 15 Nov 1996 07:48:09 -0800
  • Re: databases for spiders Thu, 14 Nov 1996 13:19:52 -0800
  • Notification protocol? Mon, 11 Nov 1996 21:20:24 -0800
  • Re: META tag standards, search accuracy Mon, 14 Oct 1996 11:16:22 -0700
  • Re: META tag standards, search accuracy Mon, 14 Oct 1996 10:14:52 -0700
  • Re: META tag standards, search accuracy Mon, 14 Oct 1996 09:27:12 -0700
  • Re: META tag standards, search accuracy Mon, 14 Oct 1996 07:48:48 -0700
  • META tag standards, search accuracy Sun, 13 Oct 1996 10:25:55 -0700
  • Unregistered MIME types? Wed, 25 Sep 1996 15:07:10 -0700
  • White House/PARC "Leveraging Cyberspace" Mon, 23 Sep 1996 14:34:05 -0700
  • Topic drift (archive robot, copyright...) Fri, 20 Sep 1996 10:41:18 -0700
  • A few copyright notes Thu, 19 Sep 1996 10:18:12 -0700
  • RE: The Internet Archive robot Thu, 19 Sep 1996 10:18:16 -0700
  • RE: How can IR Agents be evaluate ? Sat, 17 Aug 1996 14:32:23 -0700
  • Re: Search Engine Sat, 17 Aug 1996 14:32:29 -0700
  • Re: I vote NO (Was: Robot Gripes forum?) Sun, 4 Aug 1996 09:35:28 -0700
  • I vote NO (Was: Robot Gripes forum?) Fri, 2 Aug 1996 17:56:23 -0700
  • Re: netscape spec for RDM Wed, 31 Jul 1996 09:08:43 -0700
  • Re: Suggestion to help robots and sites coexist a little better Thu, 18 Jul 1996 12:28:17 -0700
  • Re: Suggestion to help robots and sites coexist a little better Thu, 18 Jul 1996 12:17:33 -0700
  • Re: Suggestion to help robots and sites coexist a little better Wed, 17 Jul 1996 09:25:38 -0700
  • Re: Suggestion to help robots and sites coexist a little better Wed, 17 Jul 1996 09:25:49 -0700
  • Re: Suggestion to help robots and sites coexist a little better Mon, 15 Jul 1996 11:13:27 -0700
  • Re: Alta Vista getting stale? Fri, 12 Jul 1996 10:30:27 -0700
  • Re: Should I index all ... Mon, 8 Jul 1996 21:40:46 -0700 (PDT)
  • Re: Description or Abstract? Mon, 8 Jul 1996 21:38:20 -0700 (PDT)
  • Re: Description or Abstract? Thu, 4 Jul 1996 10:25:34 -0700
  • Re: Harvest-like use of spiders Thu, 4 Jul 1996 10:25:30 -0700
  • Re: ADMIN: Archive Thu, 27 Jun 1996 08:40:41 -0700
  • Re: Search Engine end-users Wed, 26 Jun 1996 17:23:19 -0700
  • Re: in-document directive to discourage indexing ? Wed, 26 Jun 1996 17:23:15 -0700
  • Re: web topology Wed, 26 Jun 1996 17:23:05 -0700
  • Re: in-document directive to discourage indexing ? Thu, 20 Jun 1996 05:13:07 -0700
  • Re: Robot-HTML Web Page? Thu, 30 May 1996 10:51:34 -0700
  • Political economy of distributed search (was topical search...) Thu, 2 May 1996 17:48:57 -0700
  • Re: topical search tool -- help?! Thu, 2 May 1996 11:45:31 -0700
  • Re: topical search tool -- help?! Thu, 2 May 1996 07:33:07 -0700
  • Re: Looking for a search engine Tue, 30 Apr 1996 22:27:36 -0700
  • Re: Admin: List archive is back Tue, 30 Apr 1996 12:04:22 -0700
  • "Good Times" hoax Wed, 24 Apr 1996 18:08:59 -0700
  • Re: word spam Mon, 22 Apr 1996 19:46:54 -0700
  • Re: Search accuracy Thu, 4 Apr 1996 18:11:28 -0800
  • Re: Search accuracy Thu, 4 Apr 1996 16:04:30 -0800
  • Re: Search accuracy Thu, 4 Apr 1996 11:01:45 -0800
  • Re: Search accuracy Mon, 1 Apr 1996 17:20:39 -0800
  • Re: Heuristics.... Mon, 1 Apr 1996 09:29:12 -0800
  • Search accuracy Mon, 1 Apr 1996 09:29:25 -0800
  • Re: image map traversal Mon, 25 Mar 1996 09:48:00 -0800
  • Re: Robots available for Intranet applications Thu, 14 Mar 1996 09:49:12 -0800
  • Re: Anyone doing a Java-based robot yet? Mon, 19 Feb 1996 11:42:14 -0800
  • Anyone doing a Java-based robot yet? Sun, 18 Feb 1996 10:32:25 -0800
  • Re: Dot dot problem... Sun, 18 Feb 1996 10:32:12 -0800
  • Re: Money Spider WWW Robot for Windows Wed, 14 Feb 1996 08:16:24 -0800
  • Re: News Clipper for newsgroups - Windows Thu, 8 Feb 1996 16:50:00 -0800
  • Who sets standards (was Server name in /robots.txt) Sat, 20 Jan 1996 16:20:38 -0800
  • Re: robots.txt , authors of robots , webmasters .... Thu, 18 Jan 1996 17:12:29 -0800
  • Duplicate docs (was avoiding infinite regress...) Tue, 9 Jan 1996 07:34:35 -0800
  • Re: Vacation wars Sat, 16 Dec 1995 12:49:27 -0800
  • Re: Wobot? Fri, 15 Dec 1995 15:45:18 -0800
  • Re: Freely available robot code in C available? Wed, 13 Dec 1995 07:48:15 -0800
  • Re: Looking for a spider Fri, 1 Dec 1995 10:28:27 -0800
  • Re: BOUNCE robots: Admin request Sun, 26 Nov 1995 18:28:33 -0800
  • Re: Q: Cooperation of robots Sun, 26 Nov 1995 09:10:32 -0800
  • Re: Proposed URLs that robots should search Mon, 23 Oct 1995 15:26:16 -0700
  • Re: Unfriendly robot at 205.252.60.50 Sun, 22 Oct 1995 13:13:12 -0700
  • Re: Unfriendly robot at 205.252.60.50 Sat, 21 Oct 1995 10:35:40 -0700
  • Re: Unfriendly robot at 205.252.60.50 Thu, 19 Oct 1995 08:41:05 -0700
  • Really fast searching Wed, 18 Oct 1995 16:18:02 -0700
  • Unfriendly robot owner identified! Wed, 18 Oct 1995 14:54:22 -0700
  • Re: Unfriendly robot at 205.177.10.2 Wed, 18 Oct 1995 09:30:32 -0700
  • CORRECTION -- Re: Unfriendly robot Wed, 18 Oct 1995 09:05:20 -0700
  • Unfriendly robot at 205.177.10.2 Wed, 18 Oct 1995 08:31:05 -0700
• Nick Craswell
  • Re: Need help on Search Engine accuracy test. Thu, 30 Jan 1997 12:23:40 +1100
  • Re: Info on large scale spidering? Thu, 23 Jan 1997 09:21:16 +1100
  • Info on large scale spidering? Tue, 21 Jan 1997 12:37:31 +1100
• Nick Dearnaley
  • Re: Disallow/Allow by Action (Re: robots.txt syntax) Mon, 21 Oct 1996 09:31:18 +0000
  • Re: ActiveAgent Mon, 14 Oct 1996 17:35:42 +0000
  • Re: Another rating scam! (And a proposal on how to fix it Mon, 14 Oct 1996 14:14:17 +0000
  • Re: Topic-specific robots Mon, 14 Oct 1996 10:43:57 +0000
  • Re: The End of The World (Wide Web) / Part II Fri, 4 Oct 1996 10:07:41 +0000
  • Re: A bad agent? Fri, 27 Sep 1996 11:00:53 +0000
  • A bad agent? Thu, 26 Sep 1996 18:09:26 +0000
• Nigel Rantor
  • Re: Standard? Wed, 4 Dec 1996 10:14:47 +0000 (GMT)
  • Re: Standard? Tue, 3 Dec 1996 18:03:58 +0000 (GMT)
  • Re: Standard? Tue, 3 Dec 1996 10:58:38 +0000 (GMT)
  • Re: an article... (was: Re: Standard?) Mon, 2 Dec 1996 03:34:23 +0000 (GMT)
  • Cache Filler Fri, 29 Nov 1996 12:56:05 +0000 (GMT)
• Noah Parker
  • Re: Q: size of the web in bytes, comprehensive list Wed, 22 Jan 1997 10:06:58 -0500
• nobody@physics.yale.edu
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 96 20:39:29 -0700
• Noboru Iwayama
  • RE: Indexing two-byte text Mon, 11 Dec 1995 10:30:03 +0900
• Oliver A. McBryan
  • Re: About Mother of All Bulletin Boards Tue, 23 Apr 1996 07:56:58 -0600
• olly@muscat.co.uk
  • Re: indexing via redirectors 21 Jan 1997 13:31:24 -0000
  • Re: Cron <robh@us2> /usr/home/robh/show_robots (fwd) 10 Jan 1997 11:04:54 -0000
  • Re: Crawlers and "dynamic" urls 12 Dec 1996 15:57:32 -0000
  • Re: Crawlers and "dynamic" urls 11 Dec 1996 11:57:11 -0000
• Operator
  • Re: Search Engine Tue, 13 Aug 96 22:14:42 -0400
  • Re: robots on an intranet (replies to list...) Tue, 13 Aug 96 17:57:50 -0400
• Otis Gospodnetic
  • Re: Inktomi & large scale spidering Sun, 26 Jan 1997 12:03:59 -0500 (EST)
  • Inktomi & large scale spidering Sun, 26 Jan 1997 00:59:35 -0500 (EST)
  • Re: Info on large scale spidering? Thu, 23 Jan 1997 15:01:23 -0500 (EST)
  • Re: Crawling & DNS issues Tue, 21 Jan 1997 23:51:46 -0500 (EST)
  • Re: Info on large scale spidering? Mon, 20 Jan 1997 22:42:37 -0500 (EST)
  • Re: Q: size of the web in bytes, comprehensive list Mon, 20 Jan 1997 18:59:52 -0500 (EST)
  • Re: "real-time" spidering by Lycos Sat, 18 Jan 1997 21:46:07 -0500 (EST)
  • "real-time" spidering by Lycos Sat, 18 Jan 1997 15:36:50 -0500 (EST)
  • Re: Just when you thought it might be interesting to standardize Wed, 4 Dec 1996 18:35:03 -0500 (EST)
  • Re: WebCrawler & Excite Wed, 4 Dec 1996 12:18:33 -0500 (EST)
  • Re: WebCrawler & Excite Wed, 4 Dec 1996 01:37:29 -0500 (EST)
  • WebCrawler & Excite Tue, 3 Dec 1996 21:59:19 -0500 (EST)
  • Re: spam? (fwd) Sat, 23 Nov 1996 23:06:08 -0500 (EST)
  • Re: spam? (fwd) Sat, 23 Nov 1996 20:07:31 -0500 (EST)
  • Lycos' HEAD vs. GET Sat, 23 Nov 1996 13:43:51 -0500 (EST)
  • Re: Hip Crime Sat, 23 Nov 1996 12:12:41 -0500 (EST)
  • Re: infoseeks robot is dumb Fri, 15 Nov 1996 22:13:23 -0500 (EST)
  • Re: infoseeks robot is dumb Fri, 15 Nov 1996 22:12:02 -0500 (EST)
  • Re: infoseeks robot is dumb Fri, 15 Nov 1996 20:56:18 -0500 (EST)
  • infoseeks robot is dumb Fri, 15 Nov 1996 11:38:12 -0500 (EST)
  • how do they do that? Wed, 13 Nov 1996 11:32:45 -0500 (EST)
  • robot algorithm ? Sat, 9 Nov 1996 18:32:32 -0500 (EST)
  • Re: sockets in PERL Thu, 31 Oct 1996 19:24:55 +1900 (EST)
• P. Senthil
  • Re: technical descripton Thu, 07 Nov 1996 10:12:52 +0530
• Patrick 'Zapzap' Lin
  • Re: Wanted: Web Robot code - C/Perl Thu, 8 Feb 1996 14:09:57 +0100 (MET)
• Patrick Berchtold
  • Re: Analysing the Web (was Re: Info on large scale spidering?) Fri, 31 Jan 1997 12:07:46 +0100
  • fetching .map files Mon, 20 Jan 1997 23:54:08 +0100
  • indexing via redirectors Tue, 21 Jan 1997 00:19:43 +0100
• Patrick Lee
  • Re: hoohoo.cac.washington = bad Mon, 29 Jul 1996 11:57:55 -0400
  • Re: AltaVista's Index is obsolete Fri, 19 Jul 1996 11:28:42 -0400
• Paul Bingham
  • Agent Specification Tue, 04 Feb 1997 18:33:13 -0800
  • Robot Specifications. Tue, 04 Feb 1997 18:31:43 -0800
• Paul De Bra
  • Re: WebAnalyzer - introduction Wed, 19 Jun 1996 10:08:24 +0200 (MET DST)
  • Re: Robots in the client? Fri, 19 Apr 1996 15:28:02 +0200 (MET DST)
• Paul Francis
  • Re: Another rating scam! (And a proposal on how to fix it) Mon, 14 Oct 1996 10:17:43 +0900 (JST)
  • Re: Another rating scam! (And a proposal on how to fix it) Sun, 13 Oct 1996 13:05:59 +0900 (JST)
  • Re: Robot Research Thu, 18 Jul 96 18:36:08 JST
  • RE: nastygram from xxx.lanl.gov Wed, 10 Jul 96 10:35:49 JST
  • Re: nastygram from xxx.lanl.gov Tue, 9 Jul 96 17:22:03 JST
  • Re: nastygram from xxx.lanl.gov Tue, 9 Jul 96 15:31:16 JST
  • Re: nastygram from xxx.lanl.gov Tue, 9 Jul 96 12:43:14 JST
  • Re: loc(SOIF) Thu, 27 Jun 96 09:20:01 JST
  • Re: Keyword indexing Wed, 19 Jun 96 11:12:43 JST
  • RE: alta vista and virtualvin.com Tue, 4 Jun 96 09:32:42 JST
  • Re: Robot-HTML Web Page? Fri, 24 May 96 14:18:03 JST
  • Re: Robot-HTML Web Page? Fri, 24 May 96 11:20:54 JST
  • Re: topical search tool -- help?! Fri, 3 May 96 11:34:43 JST
  • Re: topical search tool -- help?! Thu, 2 May 96 12:47:29 JST
  • Re: topical search tool -- help?! Thu, 2 May 96 13:58:14 JST
  • Re: topical search tool -- help?! Thu, 2 May 96 12:47:29 JST
  • Ingrid ready for prelim alpha testing.... Mon, 19 Feb 96 12:37:52 JST
  • Re: Indexing two-byte text Fri, 8 Dec 95 17:25:12 JST
  • Re: Indexing two-byte text Thu, 7 Dec 95 16:42:14 JST
  • Re: Indexing two-byte text Thu, 7 Dec 95 11:48:24 JST
  • Re: Indexing two-byte text Wed, 6 Dec 95 16:46:55 JST
  • Re: Q: Cooperation of robots Mon, 27 Nov 95 12:42:56 JST
  • Re: Smart Agent help Mon, 27 Nov 95 09:57:15 JST
  • yet another robot Fri, 24 Nov 95 16:45:28 JST
• Paul Ginsparg 505-667-7353
  • forwarded e-mail Thu, 11 Jul 1996 17:38:23 -0700
• Paul Phillips
  • The Metacrawler, Reborn Wed, 12 Feb 1997 22:09:49 -0800 (PST)
• perez@fusoes.com.br
  • Req for ADMIM: How to sunsubscribe? Thu, 18 Jul 1996 12:06:46 -0300
• Pertti Kasanen
  • Anyone doing a Java-based robot yet? Wed, 21 Feb 1996 16:29:27 +0200
• Peter Jurg
  • Re: Notification protocol? Thu, 21 Nov 1996 09:54:48 +0100
  • Re: Notification protocol? Sun, 17 Nov 1996 20:37:13 +0100
• Peter Small
  • Extracting info from SIG forum archives Sun, 8 Sep 1996 04:56:49 +0100
  • Re: articles or URL's on search engines Wed, 4 Sep 1996 18:07:11 +0100
  • Re: robots on an intranet Tue, 6 Aug 1996 09:03:39 +0100
• Peter.Vogt@tokai.sprint.com
  • nastygram from xxx.lanl.gov Tue, 9 Jul 1996 04:56:00 -0400
• Phil Hochstetler
  • Re: The Internet Archive robot Mon, 9 Sep 96 12:22:55 PDT
• Pinano@aol.com
  • Re: UNSUBSCRIBE ROBOTS Tue, 26 Mar 1996 19:33:04 -0500
• pinson
  • Re: Anyone know who owns this one? Mon, 5 Aug 1996 09:18:18 +0200
• RaffiK98@aol.com
  • We need to Shut down Roenick Thu, 26 Dec 1996 11:30:21 -0800
• Rafhael Cedeno
  • crawling accident Fri, 12 Jul 1996 17:59:38 -0700
  • McKinley robot Thu, 30 May 1996 14:28:36 -0700
• Randall Hill
  • Either a spider or a hacker? ww2.allcon.com Thu, 7 Dec 1995 14:52:09 -0500 (EST)
• Randy Fischer
  • Re: Web pages being served from an SQL database Tue, 17 Dec 1996 16:27:12 -0500
  • Re: Standard? Tue, 03 Dec 1996 14:01:37 -0500
• Randy Terbush
  • Excite Authors? Mon, 06 Jan 1997 11:41:30 -0600
  • Re: ActiveAgent Fri, 08 Nov 1996 12:18:20 -0600
  • Re: ActiveAgent Fri, 08 Nov 1996 09:18:59 -0600
  • Re: Safe Methods Thu, 18 Jul 1996 19:36:35 -0500
  • Re: Suggestion to help robots and sites coexist a little better Mon, 15 Jul 1996 15:15:15 -0500
  • Re: dumb robots and xxx Fri, 12 Jul 1996 20:36:17 -0500
  • Re: dumb robots and xxx Fri, 12 Jul 1996 15:47:25 -0500
• Reinier Post
  • Re: robots on an intranet (replies to list...) Tue, 13 Aug 1996 20:02:40 +0200 (MET DST)
  • Re: desperately looking for a news searcher Thu, 4 Jul 1996 20:35:54 +0200 (MET DST)
  • Re: Description or Abstract? Thu, 4 Jul 1996 20:28:28 +0200 (MET DST)
  • Re: Description or Abstract? Thu, 4 Jul 1996 09:25:02 +0200 (MET DST)
  • Re: Client Robot 'Ranjan' Fri, 21 Jun 1996 14:11:53 +0200 (MET DST)
  • Re: robots.txt usage Tue, 18 Jun 1996 14:52:27 +0200 (MET DST)
  • Re: word spam Wed, 17 Apr 1996 02:21:12 +0200 (MET DST)
  • Re: verify URL Wed, 13 Mar 1996 22:04:23 +0100 (MET)
  • Re: Dot dot problem... Sun, 18 Feb 1996 18:14:48 +0100 (MET)
  • Re: Server name in /robots.txt Sat, 20 Jan 1996 00:52:51 +0100 (MET)
  • Re: robots.txt , authors of robots , webmasters .... Fri, 19 Jan 1996 02:07:26 +0100 (MET)
  • Re: robots.txt , authors of robots , webmasters .... Wed, 17 Jan 1996 11:54:42 +0100 (MET)
  • Re: Alta Vista searches WHAT?!? Wed, 17 Jan 1996 11:23:27 +0100 (MET)
  • avoiding infinite regress for robots Mon, 8 Jan 1996 20:04:19 +0100 (MET)
  • Re: Robot on the Rampage Thu, 14 Dec 1995 19:14:53 +0100 (MET)
  • Re: Preliminary robot.faq (Please Send Questions or Comments) Wed, 8 Nov 1995 14:44:43 +0100 (MET)
  • Re: meta tag implementation Tue, 7 Nov 1995 15:17:26 +0100 (MET)
  • Re: lycos patents Thu, 26 Oct 1995 03:19:08 +0100 (MET)
  • Re: Lycos unfriendly robot Thu, 19 Oct 1995 19:01:00 +0100 (MET)
  • Re: Looking for a spider Thu, 19 Oct 1995 15:56:40 +0100 (MET)
  • Re: Unfriendly robot at 205.177.10.2 Thu, 19 Oct 1995 15:53:11 +0100 (MET)
  • Re: Unfriendly robot at 205.177.10.2 Wed, 18 Oct 1995 17:42:55 +0100 (MET)
• Renato Mario Rossello
  • Requesting info on database engines Thu, 1 Feb 1996 01:00:04 -0600
  • Re: HEAD request [was Re: Server name in /robots.txt] Wed, 31 Jan 1996 15:25:58 -0600
• Reynolds, Cathy A
  • For Sale for 12/26! Thu, 26 Dec 1996 10:51:58 -0800
• Ricardo Eito Brun
  • Re: Keyword indexing Thu, 20 Jun 1996 15:51:00 +0100
  • Re: Robot-HTML Web Page? Mon, 27 May 1996 13:49:55 +0100
  • About integrated search engines Thu, 25 Apr 1996 21:30:31 +0100
  • About Mother of All Bulletin Boards Tue, 23 Apr 1996 13:17:41 +0100
  • Re: Robots in the client? Tue, 23 Apr 1996 12:34:06 +0100
  • General Information Fri, 19 Apr 1996 20:59:49 +0100
  • Robots in the client? Fri, 19 Apr 1996 21:03:38 +0100
• Rich Dorfman
  • re: email grabber Thu, 16 Jan 1997 00:13:46 -0600
• Richard A. Paris
  • email spider Fri, 16 Aug 1996 17:35:27 -0400
• Richard Edwards
  • Have you used the Microsoft Active-X Internet controls for Visual Basic? (Or know someone who does?) Tue, 21 Jan 1997 17:11:46 -0800
• Richard Gaskin
  • Re: SpamBots Thu, 16 Jan 97 09:00:47 -0000
  • re: email grabber Thu, 16 Jan 97 00:35:53 -0000
  • Re: Standard? Tue, 3 Dec 96 11:30:56 -0000
  • Re: Admitting the obvious Wed, 13 Nov 96 15:13:11 -0000
  • Re: ActiveAgent Mon, 14 Oct 96 13:04:58 -0800
• Richard Gaskin - Fourth World
  • Re: ActiveAgent Fri, 8 Nov 96 14:37:25 -0000
  • Re: ActiveAgent Fri, 8 Nov 96 09:27:45 -0000
  • CyberPromo shut down at last!!! Fri, 18 Oct 96 15:03:24 -0000
  • Re: (Fwd) Re: Copyrights, let them be ! Wed, 11 Sep 96 23:24:57 -0800
  • Re: Copyrights, let them be ! Wed, 11 Sep 96 10:56:11 -0800
  • Re: Copyrights on the web Tue, 10 Sep 96 17:58:25 -0800
  • RE: copyright, etc. Tue, 10 Sep 96 16:13:06 -0800
  • Re: The Internet Archive robot Mon, 9 Sep 96 17:07:05 -0800
  • Re: The Internet Archive robot Mon, 9 Sep 96 16:37:04 -0800
• Richard Glenner
  • News Clipper for newsgroups - Windows Sat, 3 Feb 1996 07:42:38 +0000
• Richard J. Rossi
  • Re: Introducing myself Tue, 18 Jun 1996 03:41:05 -0500 (CDT)
  • Re: Robot books Tue, 18 Jun 1996 01:44:22 -0500 (CDT)
  • Introducing myself Sun, 16 Jun 1996 02:06:03 -0500 (CDT)
• Richard Levitte - VMS Whacker
  • Re: Standard? Fri, 29 Nov 1996 23:29:50 MET
  • Re: make people use ROBOTS.txt? Sat, 23 Nov 1996 12:19:37 MET
  • Re: ActiveAgent Fri, 8 Nov 1996 23:55:55 MET
  • Re: ActiveAgent Fri, 8 Nov 1996 22:47:19 MET
  • Re: ActiveAgent Fri, 8 Nov 1996 02:41:50 MET
  • Re: ActiveAgent Mon, 14 Oct 1996 20:01:39 MET DST
• Richard Rossi
  • Looking for subcontracting spider-programmers 6 Sep 96 10:26:26
• Rob Hartill
  • Re: More with the Cypherpunk antics Sat, 15 Feb 1997 15:43:32 +0000 (GMT)
  • Re: Lack of support for "If-Modified-Since" Fri, 14 Feb 1997 03:26:14 +0000 (GMT)
  • Re: Lack of support for "If-Modified-Since" Thu, 13 Feb 1997 23:38:30 +0000 (GMT)
  • Re: Single tar (Re: Inktomi & large scale spidering) Tue, 28 Jan 1997 10:19:59 +0000 (GMT)
  • Cron <robh@us2> /usr/home/robh/show_robots (fwd) Fri, 10 Jan 1997 01:09:57 +0000 (GMT)
  • Re: It's not only robots we have to worry about ... Thu, 26 Dec 1996 17:57:26 +0000 (GMT)
  • Re: It's not only robots we have to worry about ... Thu, 26 Dec 1996 13:43:24 +0000 (GMT)
  • Netscape-Catalog-Robot Tue, 24 Dec 1996 12:20:37 +0000 (GMT)
  • USER_AGENT spoofing Mon, 16 Dec 1996 05:01:16 +0000 (GMT)
  • Re: legal equivalence of fax and email was Re: Standard? Mon, 2 Dec 1996 19:51:45 +0000 (GMT)
  • Re: an article... (was: Re: Standard?) Fri, 29 Nov 1996 22:03:17 +0000 (GMT)
  • Re: Servers vs Agents Wed, 27 Nov 1996 19:40:34 +0000 (GMT)
  • Re: NetJet Tue, 26 Nov 1996 17:17:29 +0000 (GMT)
  • Re: NetJet Tue, 26 Nov 1996 13:59:53 +0000 (GMT)
  • USER_AGENT and Apache 1.2 Mon, 25 Nov 1996 22:03:18 +0000 (GMT)
  • Re: defining "robot" Mon, 25 Nov 1996 20:15:45 +0000 (GMT)
  • hipcrime Sun, 24 Nov 1996 15:30:00 +0000 (GMT)
  • ActiveAgent Sat, 23 Nov 1996 16:36:28 +0000 (GMT)
  • another dumb robot (possibly) Sat, 23 Nov 1996 16:41:12 +0000 (GMT)
  • another rare attack Sat, 23 Nov 1996 13:31:38 +0000 (GMT)
  • ActiveAgent Fri, 22 Nov 1996 21:50:04 +0000 (GMT)
  • Re: not a robot Fri, 22 Nov 1996 19:02:05 +0000 (GMT)
  • another suggestion Fri, 22 Nov 1996 17:23:05 +0000 (GMT)
  • Re: define a page? Fri, 22 Nov 1996 14:32:28 +0000 (GMT)
  • Re: who/what uses robots.txt Fri, 22 Nov 1996 01:02:00 +0000 (GMT)
  • Re: Broadness of Robots.txt (Re: Washington again !!!) Thu, 21 Nov 1996 22:32:26 +0000 (GMT)
  • must be something in the water Thu, 21 Nov 1996 01:03:02 +0000 (GMT)
  • Re: Washington again !!! Wed, 20 Nov 1996 02:42:19 +0000 (GMT)
  • Re: changes to robots.txt Sat, 16 Nov 1996 03:16:07 +0000 (GMT)
  • Re: changes to robots.txt Tue, 12 Nov 1996 19:27:40 +0000 (GMT)
  • changes to robots.txt Tue, 12 Nov 1996 13:05:33 +0000 (GMT)
  • Re: ActiveAgent Fri, 8 Nov 1996 14:02:31 +0000 (GMT)
  • Re: infoseek Mon, 21 Oct 1996 01:12:53 +0100 (BST)
  • Re: agents ignoring robots.txt Thu, 17 Oct 1996 00:26:54 +0100 (BST)
  • agents ignoring robots.txt Wed, 16 Oct 1996 17:49:13 +0100 (BST)
  • Re: The Internet Archive robot Thu, 3 Oct 1996 21:08:08 +0100 (BST)
  • Re: A bad agent? Fri, 27 Sep 1996 13:00:14 +0100 (BST)
  • Re: A bad agent? Thu, 26 Sep 1996 22:17:05 +0100 (BST)
  • Re: I vote NO (Was: Robot Gripes forum?) Sun, 4 Aug 1996 21:35:32 +0100 (BST)
  • Re: I vote NO (Was: Robot Gripes forum?) Sat, 3 Aug 1996 07:56:41 +0100 (BST)
  • Re: Anyone know who owns this one? Fri, 2 Aug 1996 19:38:58 +0100 (BST)
  • Re: the POST myth... a web admin's opinions.. Thu, 18 Jul 1996 20:26:05 -0600 (MDT)
  • Re: Safe Methods Thu, 18 Jul 1996 19:36:30 -0600 (MDT)
  • Re: Suggestion to help robots and sites coexist a little better Thu, 18 Jul 1996 17:32:41 -0600 (MDT)
  • Re: Suggestion to help robots and sites coexist a little better Thu, 18 Jul 1996 17:23:23 -0600 (MDT)
  • Re: Safe Methods Thu, 18 Jul 1996 14:03:46 -0500 (CDT)
  • Re: Safe Methods Thu, 18 Jul 1996 00:26:07 -0500 (CDT)
  • Re: Suggestion to help robots and sites coexist a little better Wed, 17 Jul 1996 21:04:28 -0600 (MDT)
  • Re: Suggestion to help robots and sites coexist a little better Wed, 17 Jul 1996 21:16:28 -0600 (MDT)
  • Re: HOST: header Wed, 17 Jul 1996 11:26:57 -0500 (CDT)
  • Re: Suggestion to help robots and sites coexist a little better Wed, 17 Jul 1996 08:33:54 -0600 (MDT)
  • HOST: header Tue, 16 Jul 1996 20:08:07 -0600 (MDT)
  • Re: PS Mon, 15 Jul 1996 18:06:43 -0600 (MDT)
  • Re: Suggestion to help robots and sites coexist a little better Mon, 15 Jul 1996 15:22:57 -0500 (CDT)
  • Re: Suggestion to help robots and sites coexist a little better Mon, 15 Jul 1996 15:06:40 -0500 (CDT)
  • PS Sun, 14 Jul 1996 20:16:02 -0500 (CDT)
  • Suggestion to help robots and sites coexist a little better Sun, 14 Jul 1996 20:13:03 -0500 (CDT)
  • Re: the POST myth... a web admin's opinions.. Fri, 12 Jul 1996 14:11:53 -0500 (CDT)
  • the POST myth... a web admin's opinions.. Thu, 11 Jul 1996 22:13:27 -0600 (MDT)
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 22:19:53 -0600 (MDT)
  • dumb robots and xxx Wed, 10 Jul 1996 22:14:16 -0600 (MDT)
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 19:54:58 -0600 (MDT)
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 16:08:28 -0500 (CDT)
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 12:52:51 -0500 (CDT)
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 12:20:11 -0500 (CDT)
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 11:15:50 -0500 (CDT)
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 08:16:29 -0600 (MDT)
  • RE: nastygram from xxx.lanl.gov Tue, 9 Jul 1996 21:56:40 -0600 (MDT)
  • Re: nastygram from xxx.lanl.gov Tue, 9 Jul 1996 21:42:44 -0600 (MDT)
• Rob Torchon
  • Re: Quakebots Mon, 2 Sep 1996 13:04:39 -0800
• Rob Turk
  • Re: Robot Gripes forum? (Was: Anyone know who owns this one?) Fri, 02 Aug 1996 15:25:40 -0500
  • Re: Anyone know who owns this one? Fri, 02 Aug 1996 09:18:53 -0500
  • Re: Safe Methods Thu, 18 Jul 1996 17:06:37 -0500
  • Re: About the question what a robots is (was Re: PS) Tue, 16 Jul 1996 11:38:25 -0500
  • Re: psycho at xxx.lanl.gov Tue, 09 Jul 1996 16:03:49 -0500
  • Re: in-document directive to discourage indexing ? Tue, 25 Jun 1996 12:16:53 -0500
  • A modest proposal...<snip> to discourage indexing ? Tue, 25 Jun 1996 11:52:39 -0500
  • Re: Robot's Book. Wed, 26 Jun 1996 11:20:29 -0500
  • Re: in-document directive to discourage indexing ? Tue, 25 Jun 1996 12:16:53 -0500
  • A modest proposal...<snip> to discourage indexing ? Tue, 25 Jun 1996 11:52:39 -0500
  • Re: in-document directive to discourage indexing ? Mon, 24 Jun 1996 15:46:09 -0500
  • Re: RCPT: Re: Introducing myself Fri, 21 Jun 1996 09:59:58 -0500
  • Re: (book recommendation re: net agents) Mon, 10 Jun 1996 08:48:49 -0500
  • [Fwd: Re: To: ???? Robot] Mon, 29 Apr 1996 10:13:21 -0500
  • Re: communal bots Fri, 26 Apr 1996 14:40:40 -0500
  • Re: Apologies || communal bots Fri, 26 Apr 1996 11:51:11 -0500
  • Re: search engine Wed, 24 Apr 1996 09:51:12 -0500
  • Re: Mailing list Tue, 23 Apr 1996 10:20:12 -0500
  • Re: Links This Site is about Robots Not Censorship Fri, 29 Mar 1996 13:18:50 -0600
• Robert B. Turk
  • Re: META tag standards, search accuracy Mon, 14 Oct 1996 09:03:21 -0500
  • Re: Copyrights (was Re: The Internet Archive robot) Wed, 11 Sep 1996 09:23:55 -0500
  • Re: Copyrights (was Re: The Internet Archive robot) Tue, 10 Sep 1996 09:31:08 -0500
  • Re: The Internet Archive robot (fwd) Mon, 09 Sep 1996 12:32:38 -0500
  • Re: The Internet Archive robot Mon, 09 Sep 1996 09:49:24 -0500
• Robert Nicholson
  • depth first vs breadth first Thu, 29 Aug 1996 16:51:03 -0400 (EDT)
• Robert Raisch, The Internet Company
  • Re: email grabber Thu, 16 Jan 1997 15:37:30 -0400
  • Re: email address grabber Wed, 15 Jan 1997 11:07:58 -0400
  • (Fwd) Re: Copyrights, let them be ! Wed, 11 Sep 1996 17:17:16 -0400
  • Re: Client Robot 'Ranjan' Wed, 19 Jun 1996 15:33:01 -0400
  • RE: Tagging a document with language Tue, 18 Jun 1996 17:14:35 -0400
  • Re: Keyword indexing Tue, 18 Jun 1996 08:19:29 -0400
  • RE: Tagging a document with language Fri, 7 Jun 1996 10:41:03 -0400
  • Re: Defenses against bad robots Mon, 20 May 1996 15:45:11 -0400
  • Re: Defenses against bad robots Mon, 20 May 1996 09:27:17 -0400
  • Re: topical search tool -- help?! Fri, 3 May 1996 10:00:39 -0400
  • Re: Political economy of distributed search (was topical search Fri, 3 May 1996 09:23:06 -0400
  • Re: topical search tool -- help?! Thu, 2 May 1996 16:20:39 -0400
  • Re: Search accuracy Thu, 4 Apr 1996 20:22:09 -0400
  • Re: robots.txt , authors of robots , webmasters .... Thu, 18 Jan 1996 11:20:15 -0400
  • Re: Does anyone else consider this irresponsible? Wed, 10 Jan 1996 14:48:14 -0400
  • Re: Does anyone else consider this irresponsible? Wed, 10 Jan 1996 13:40:45 -0400
  • Does anyone else consider this irresponsible? Wed, 10 Jan 1996 11:08:47 -0400
• Robert Stober
  • (no subject) Fri, 6 Sep 1996 20:24:03 -0500
• ROBERT-LF_HUANG@HP-China-om1.om.hp.com
  • Re: databases for spiders Fri, 15 Nov 96 09:48:44 +0800
  • Re: ActiveAgent Fri, 8 Nov 96 16:10:02 +0800
• Roger Dearnaley
  • Infinite e-mail loop 5 Jan 1996 13:34:36 -0800
  • [5]Wobot? 15 Dec 1995 11:49:43 -0800
  • [1]Contact for Intouchgroup 15 Dec 1995 11:43:55 -0800
  • [2]RE>[2]RE>[5]RE>Checking 15 Dec 1995 11:38:24 -0800
  • [2]RE>[3]RE>[5]RE>Checking 15 Dec 1995 11:28:25 -0800
  • [4]Announcing NaecSpyr, a n 15 Dec 1995 11:28:52 -0800
  • [4]Wobot? 15 Dec 1995 11:18:46 -0800
  • [1]RE>[2]RE>[5]RE>Checking 15 Dec 1995 11:02:20 -0800
  • [5]RE>[5]RE>Checking Log fi 15 Dec 1995 10:51:21 -0800
  • [3]Announcing NaecSpyr, a n 15 Dec 1995 10:45:57 -0800
  • [1]RE>[3]RE>[5]RE>Checking 15 Dec 1995 10:46:37 -0800
  • [3]Wobot? 15 Dec 1995 10:35:10 -0800
  • [3]RE>[5]RE>Checking Log fi 15 Dec 1995 09:06:06 -0800
  • [4]RE>[5]RE>Checking Log fi 15 Dec 1995 10:09:01 -0800
  • [2]Wobot? 15 Dec 1995 09:59:09 -0800
  • [2]Announcing NaecSpyr, a n 15 Dec 1995 09:58:31 -0800
  • [1]Announcing NaecSpyr, a n 15 Dec 1995 09:26:58 -0800
  • [1]Wobot? 15 Dec 1995 09:11:21 -0800
  • [3]RE>[5]RE>Checking Log fi 15 Dec 1995 09:06:06 -0800
  • [2]RE>[5]RE>Checking Log fi 15 Dec 1995 03:58:38 -0800
  • [1]RE>[5]RE>Checking Log fi 15 Dec 1995 02:58:18 -0800
  • [5]RE>Checking Log files 14 Dec 1995 20:46:54 -0800
  • [4]RE>Checking Log files 14 Dec 1995 20:16:52 -0800
  • [3]RE>Checking Log files 14 Dec 1995 19:46:33 -0800
  • [2]RE>Checking Log files 14 Dec 1995 19:20:30 -0800
  • [1]RE>Checking Log files 14 Dec 1995 18:50:11 -0800
  • Re: lycos patents 25 Oct 1995 16:47:13 -0800
  • How do I let spiders in? 16 Oct 1995 18:40:48 -0800
• Ron Kanagy
  • Re: BackRub robot Thu, 13 Jun 1996 18:29:22 -0400
• Ron Wolf
  • Re: dumb robots and xxx 12 Jul 1996 09:58:38 -0800
• Ronald Kanagy
  • HTML Parser Thu, 04 Jul 1996 10:09:34 -0400
  • (no subject) Wed, 03 Jul 1996 13:33:29 -0400
• root
  • Linux and Robot development... Sun, 14 Jul 1996 23:45:48 -0700
• Ross A. Finlayson
  • Do robots have to follow links ? Mon, 06 Jan 1997 19:50:32 -0500
  • Re: Lets get on task! {was Re: Standard} Tue, 03 Dec 1996 18:13:48 -0500
  • Re: define a page? Fri, 22 Nov 1996 02:11:23 -0500
  • Re: ActiveAgent Mon, 14 Oct 1996 15:03:54 -0400
  • Re: Topic drift (archive robot, copyright...) Fri, 20 Sep 1996 16:53:41 -0400
  • Re: Public Access Nodes / Copywrited Nodes Thu, 12 Sep 1996 01:00:20 -0400
  • Re: Quakebots Mon, 02 Sep 1996 18:43:38 -0400
  • Quakebots Sun, 01 Sep 1996 12:44:17 -0400
  • grammar engines Mon, 05 Aug 1996 17:50:01 -0400
• Ross Finlayson
  • robot definition Fri, 22 Nov 1996 17:09:49 +0000 ()
  • Re: Inter-robot communication Mon, 24 Jun 1996 17:39:25 -0400
  • Re: RCPT: Re: Introducing myself Fri, 21 Jun 1996 16:24:03 -0400
  • Re: robot.polite Tue, 18 Jun 1996 21:16:29 -0400
  • Re: BackRub robot Thu, 13 Jun 1996 16:39:33 -0400
• Roy T. Fielding
  • Stupid robots cache DNS and not IMS Sat, 13 Jul 1996 13:15:29 -0700
  • Re: nastygram from xxx.lanl.gov Tue, 09 Jul 1996 18:13:05 -0700
  • BackRub robot warning Wed, 12 Jun 1996 15:20:44 -0700
• Saloum Fall
  • desperately looking for a news searcher Thu, 04 Jul 1996 20:04:04 +0200
• Sankar Virdhagriswaran
  • Re: Notification protocol? Sat, 16 Nov 1996 15:25:48 -0500
• Sanna
  • Re: What is your favorite search engine - a survey Mon, 5 Aug 1996 07:52:36 +0300 (EDT)
• Santa Claus
  • Merry Christmas, HipXmas-SantaSpam! Wed, 11 Dec 1996 19:33:18 -0400 (EST)
  • Merry Christmas, spidie-boyz&bottie-girlz! Wed, 11 Dec 1996 19:19:52 -0400 (EST)
• Saul Jacobs
  • Re: [2]RE>[5]RE>Checking Lo 15 Dec 1995 11:45:14 -0800
• savron@world-net.sct.fr
  • Re: Altavista indexing password files Fri, 1 Mar 1996 12:42:51 +0000
  • about robots.txt content errors Sat, 17 Feb 1996 10:15:19 +0000
  • Web robots and gopher space -- two separate worlds Wed, 17 Jan 1996 09:07:03 +0000
  • robots.txt , authors of robots , webmasters .... Wed, 17 Jan 1996 09:07:02 +0000
• Scott 'Webster' Wood
  • Re: email spider Fri, 16 Aug 1996 21:22:54 -0400 (EDT)
  • Re: Suggestion to help robots and sites coexist a little better Wed, 17 Jul 1996 17:39:14 -0400 (EDT)
  • Re: HOST: header Wed, 17 Jul 1996 08:03:49 -0400 (EDT)
  • Re: Suggestion to help robots and sites coexist a little better Wed, 17 Jul 1996 07:48:39 -0400 (EDT)
  • Java intelligent agents and compliance? Mon, 15 Jul 1996 00:21:16 -0400 (EDT)
  • Re: psycho at xxx.lanl.gov Wed, 10 Jul 1996 01:15:00 -0400 (EDT)
  • Re: On the subject of abuse/pro-activeness Wed, 10 Jul 1996 00:50:58 -0400 (EDT)
  • On the subject of abuse/pro-activeness Tue, 9 Jul 1996 10:56:26 -0400 (EDT)
  • HTML query to .ps? Fri, 28 Jun 1996 19:01:38 -0400 (EDT)
  • New engine on the loose? Wed, 26 Jun 1996 19:17:39 -0400 (EDT)
  • Collected information standards Fri, 21 Jun 1996 15:15:40 -0400 (EDT)
  • People who live in SPAM houses... (Was re: RCPT Fri, 21 Jun 1996 15:10:41 -0400 (EDT)
  • Re: Keyword indexing Tue, 18 Jun 1996 12:52:09 -0400 (EDT)
  • Content based search engine Mon, 10 Jun 1996 12:19:41 -0400
  • Re: robots, what else! Sat, 8 Jun 1996 08:33:30 -0400 (EDT)
  • Content based robot collectors Tue, 4 Jun 1996 16:04:55 -0400 (EDT)
• Scott Johnson
  • RE: changes to robots.txt Fri, 15 Nov 1996 09:32:03 -0500
• Scott Stephenson
  • Re: lycos patents Wed, 25 Oct 95 15:55:18 -0700
• Scott W. Wood
  • Private Investigator Lists Mon, 29 Apr 1996 08:55:24 -0400
  • Re: search engine Wed, 24 Apr 1996 07:22:32 -0400
• Sean Parker
  • Dot dot problem... Fri, 17 Feb 1995 13:00:15 -0500
• Shadrach Todd
  • Re[2]: Lycos' HEAD vs. GET Mon, 25 Nov 1996 10:01:25 -0500 (EST)
• Shiraz Siddiqui
  • Robo-phopbic Mailing list Mon, 5 Aug 1996 13:19:12 -0500 (CDT)
  • Re: Java intelligent agents and compliance? Thu, 1 Aug 1996 07:48:56 -0500 (CDT)
  • *Help: Writing BOTS* Thu, 18 Jul 1996 12:05:16 -0500 (CDT)
• Showbiz Information
  • Re: Showbiz search engine Fri, 01 Nov 1996 18:06:20 -0800
• Sibylle Gonzales
  • robot authentication parameters Wed, 13 Mar 96 13:23:37 MST
• siddiqui athar shiraz
  • Re: Search Engine Wed, 14 Aug 1996 09:44:59 -0500 (CDT)
• Sigfrid Lundberg
  • Re: More with the Cypherpunk antics Sat, 15 Feb 1997 15:32:13 +0100 (MET)
  • Re: Single tar (Re: Inktomi & large scale spidering) Wed, 12 Feb 1997 12:32:52 +0100 (MET)
  • Re: Single tar (Re: Inktomi & large scale spidering) Mon, 27 Jan 1997 12:08:58 +0100 (MET)
  • Re: indexing via redirectors Tue, 21 Jan 1997 15:46:05 +0100 (MET)
  • Re: indexing via redirectors Tue, 21 Jan 1997 15:20:28 +0100 (MET)
  • Re: fetching .map files Tue, 21 Jan 1997 10:36:58 +0100 (MET)
  • Re: indexing via redirectors Tue, 21 Jan 1997 10:27:31 +0100 (MET)
  • Re: Cron <robh@us2> /usr/home/robh/show_robots (fwd) Fri, 10 Jan 1997 13:22:15 +0100 (MET)
  • Re: Web pages being served from an SQL database Wed, 18 Dec 1996 10:03:54 +0100 (MET)
  • Re: Back of the envelope computations Thu, 7 Nov 1996 09:30:01 +0100 (MET)
  • Re: What to rate limit/lock on, name or IP address? Wed, 23 Oct 1996 14:32:44 +0200 (MET DST)
  • Re: Image Maps Thu, 3 Oct 1996 06:54:39 +0100 (MET)
  • RE: The Internet Archive robot Fri, 13 Sep 1996 10:01:43 +0200 (MET DST)
  • RE: The Internet Archive robot Fri, 13 Sep 1996 10:01:43 +0200 (MET DST)
  • Re: Keyword indexing Tue, 18 Jun 1996 21:54:14 +0200 (MET DST)
• Simon Powell
  • Re: It's not only robots we have to worry about ... Mon, 30 Dec 1996 12:47:17 -0000
• Simon Wilkinson
  • Re: Single tar (Re: Inktomi & large scale spidering) Tue, 11 Feb 1997 23:37:57 GMT
• Simon.Stobart
  • Re: Money Spider WWW Robot for Windows Wed, 14 Feb 1996 20:19:17 GMT
  • Re: Announcement and Help Requested Mon, 18 Dec 1995 14:45:51 GMT
  • Announcement and Help Requested Thu, 14 Dec 1995 10:34:13 GMT
  • Announcement and Help Requested Thu, 14 Dec 1995 10:34:13 GMT
• Siu-ki Wong
  • Re: Introducing myself Tue, 18 Jun 1996 09:39:22 -0400 (EDT)
• Skip Montanaro
  • Re: Welcome to cypherpunks Wed, 12 Feb 1997 20:45:08 -0500 (EST)
  • Regexps (Was: Re: An extended version of the Robots...) Sat, 23 Nov 1996 20:55:53 -0500 (EST)
  • Suggestion to help robots and sites coexist a little better Mon, 15 Jul 1996 12:21:42 -0400
  • HTML Parser Fri, 12 Jul 1996 21:00:51 -0400
  • Re: Content based search engine Thu, 20 Jun 1996 14:43:28 -0400
  • Re: Robot Databases Tue, 20 Feb 1996 11:04:58 -0500
  • Re: robots.txt extensions Tue, 16 Jan 1996 08:54:14 -0500
  • Re: Horror story Sun, 14 Jan 1996 15:05:31 -0500
  • Re: Does anyone else consider... Thu, 11 Jan 1996 06:56:59 -0500
  • Responsible behavior, Robots vs. humans, URL botany... Wed, 10 Jan 1996 14:16:15 -0500
  • MD5 in HTTP headers - where? Wed, 10 Jan 1996 09:48:22 -0500
  • Infinite e-mail loop Fri, 5 Jan 1996 19:10:05 -0500
  • Re: [2]RE>[5]RE>Checking Log fi Fri, 15 Dec 1995 14:31:50 -0500
  • New robot turned loose on an unsuspecting public... and a DNS question Thu, 30 Nov 1995 16:15:21 -0500
  • How frequently should I check /robots.txt? Mon, 27 Nov 1995 22:52:30 -0500
• smadja@netvision.net.il
  • Re: Does anyone else consider... Sun, 14 Jan 96 12:02:20 EST
  • RE: Indexing two-byte text Fri, 8 Dec 95 18:16:44 EST
• Srinivas Padmanabhuni
  • Re: Crawling & DNS issues Wed, 22 Jan 1997 00:26:18 -0700 (MST)
• Stacy Cannady
  • (no subject) Fri, 06 Sep 96 07:54:33 CST
  • Re[2]: robots on an intranet Tue, 06 Aug 96 14:53:33 CST
• Stan Norton
  • Re: Does anyone else consider this irresponsible? Wed, 10 Jan 1996 11:32:33 -0500
• Stephane Vaillancourt
  • Source code Wed, 6 Nov 1996 10:47:30 -0500
• Stephen Kahn
  • Search Engine end-users Wed, 26 Jun 1996 09:48:25 -0800
• Steve DeJarnett
  • Re: changes to robots.txt Thu, 14 Nov 1996 11:51:30 -0800
• Steve Jones
  • Re: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 16:25:42 -0700
  • Re: RCPT: Re: Introducing myself Thu, 20 Jun 1996 18:05:22 -0700
  • Re: Defenses against bad robots Sun, 19 May 1996 18:21:42 -0700
  • Re: Political economy of distributed search (was topical Fri, 3 May 1996 11:45:31 -0700
• Steve Leibman
  • re: robot to get specific info only? Tue, 24 Sep 1996 20:57:52 -0400 (EDT)
  • htaccess Tue, 9 Jul 1996 11:59:12 -0400
  • multiple copies Thu, 20 Jun 1996 14:36:16 -0400
• Steve Livingston
  • Robot Databases Sun, 18 Feb 1996 09:03:18 -0700
• Steve Nisbet
  • re Email Grabber Wed, 15 Jan 1997 17:21:04 +0000
  • Re: Robot Research Thu, 18 Jul 96 10:20:47 BST
  • Re: nastygram from xxx.lanl.gov Tue, 9 Jul 96 10:47:54 BST
  • Re: Horror story Tue, 16 Jan 96 11:26:38 GMT
  • Re: Horror story Tue, 16 Jan 96 11:26:38 GMT
  • Re: meta tag implementation Tue, 7 Nov 95 14:41:39 GMT
  • Re: meta tag implementation Tue, 7 Nov 95 10:32:55 GMT
• steved@pmc02.pmc.philips.com
  • Re: depth first vs breadth first Thu, 29 Aug 1996 15:23:14 -0700 (PDT)
• Steven Frank
  • Looking for... Fri, 21 Jun 1996 09:30:09 -0500
• Steven L Baur
  • Re: robots.txt extensions 15 Jan 1996 22:06:27 -0800
• Steven L. Baur
  • Re: Unfriendly Lycos , again ... 28 Dec 1995 13:36:07 -0800
• Stimpy
  • Re: Search Engine Wed, 14 Aug 1996 20:01:36 +0000
• Super-User
  • Re: Does anyone else consider this irresponsible? Tue, 09 Jan 96 20:45:01 -0800
  • Re: Inter-robot Comms Port Tue, 26 Dec 95 02:40:33 -0800
• Susumu Shimizu
  • Re: Robot on the Rampage Fri, 15 Dec 1995 02:37:23 +0900
• Sylvain Duclos
  • Re: Robots available for Intranet applications Mon, 11 Mar 1996 22:31:46 GMT
• System Admin
  • RE: Search Engine Tue, 13 Aug 96 12:59:13 -0400
• Tangy Verdell
  • Re[2]: Anyone doing a Java-based robot yet? Tue, 20 Feb 96 10:03:49 EST
  • Re: Robots and search engines technical information. Fri, 16 Feb 96 09:49:39 EST
  • robots.txt Wed, 14 Feb 96 11:33:33 EST
  • robots.txt changes how often? Wed, 14 Feb 96 09:32:58 EST
• Ted Sullivan
  • RE: How to get listed #1 on all search engines (fwd) Mon, 30 Sep 1996 19:09:17 -0700
  • RE: Copyrights on the web Mon, 9 Sep 1996 20:10:19 -0700
  • RE: The Internet Archive robot Fri, 6 Sep 1996 16:42:04 -0700
  • RE: AltaVista's Index is obsolete; but what about the others Fri, 19 Jul 1996 08:45:47 -0700
• Ted Hardie
  • Re: Notification protocol? Tue, 12 Nov 1996 12:03:20 -0800 (PST)
• Ted Resnick
  • Re: Search Engine end-users Wed, 26 Jun 1996 14:59:19 +0100
• Ted Sullivan
  • Re: Search accuracy Fri, 5 Apr 1996 09:29:00 -0800
  • Re: Robot Databases Sun, 18 Feb 1996 18:44:00 -0800
  • Re: Robot Databases Sun, 18 Feb 1996 13:12:00 -0800
  • RE: Robot Databases Sun, 18 Feb 1996 13:05:00 -0800
  • RE: Alta Vista searches WHAT?!? Tue, 16 Jan 1996 17:30:00 -0800
  • Re: Horror story Sun, 14 Jan 1996 17:38:00 -0800
  • Re: Looking for a spider Sat, 2 Dec 1995 05:09:58 +0000
  • Re: Looking for a spider Sat, 2 Dec 1995 00:12:58 +0000
• terces1@postoffice.ptd.net
  • Re: word spam Wed, 17 Apr 1996 08:45:52 -0500
• Terry Coatta
  • RE: make people use ROBOTS.txt? Sun, 24 Nov 1996 16:47:09 -0800
• Terry O'Neill
  • Re: who/what uses robots.txt Thu, 21 Nov 1996 19:15:19 -0700
  • Re: Should I index all ... Tue, 09 Jul 1996 12:26:08 -0600
  • Re: Should I index all ... Tue, 09 Jul 1996 08:47:09 -0600
  • Re: Should I index all ... Fri, 05 Jul 1996 11:20:17 -0600
  • Re: in-document directive to discourage indexing ? Tue, 25 Jun 96 14:32:40 PDT
  • Re: in-document directive to discourage indexing ? Tue, 25 Jun 96 14:32:40 PDT
  • Re: robot.polite Tue, 18 Jun 96 00:30:31 PDT
• Terry Smith
  • Re: VB. page grabber... Mon, 06 May 1996 10:48:09 +0000
  • Re: Political economy of distributed search (was topical Fri, 03 May 1996 09:39:59 +0000
  • Re: To: ???? Robot] Mon, 29 Apr 1996 16:05:18 -0400
  • Re: url locating Mon, 19 Feb 1996 21:58:31 -0500
• Thaddeus O. Cooper
  • Re: robots.txt (A *little* off the subject) Tue, 26 Nov 1996 09:17:14 -0500
  • robots.txt (A *little* off the subject) Fri, 22 Nov 1996 09:42:09 -0500
  • Re: Broadness of Robots.txt (Re: Washington again !!!) Wed, 20 Nov 1996 11:44:56 -0500
• The Wild
  • Specific searches Sun, 2 Jun 1996 15:13:22 -0400 (EDT)
• The YakkoWakko. Webmaster
  • www.pl? Sat, 27 Jan 1996 12:48:08 -0500
• Theo Van Dinter
  • Re: robot meta tags Wed, 22 Jan 1997 00:33:07 -0500 (EST)
  • Re: indexing via redirectors Tue, 21 Jan 1997 09:36:22 -0500 (EST)
  • Re: More Robot Talk Fri, 17 Jan 1997 13:39:36 -0500 (EST)
  • Re: Frames ? Lycos ? Tue, 7 Jan 1997 00:42:19 -0500 (EST)
  • Re: Do robots have to follow links ? Mon, 6 Jan 1997 23:40:48 -0500 (EST)
• Thomas Abrahamsson
  • Accept: Fri, 17 May 1996 18:47:55 +0200 (MET DST)
• Thomas Bedell
  • Hip Crime Sat, 23 Nov 1996 08:54:56 -0500
  • Add This Search Engine to your Results. Mon, 5 Aug 1996 04:17:19 -0400
• Thomas Maslen
  • Re: New robot turned loose on an unsuspecting public... and a DNS question Thu, 30 Nov 1995 17:40:32 -0800
• Thomas Merlin
  • Referencing dynamic pages Tue, 28 Jan 1997 23:22:00 +0100
  • Meta Tags only on home page ? Mon, 27 Jan 1997 11:19:34 +0100
  • Lycos Tue, 07 Jan 1997 17:21:25 +0100
  • Do robots have to follow links ? Mon, 06 Jan 1997 23:38:45 +0100
  • Frames ? Lycos ? Mon, 06 Jan 1997 23:56:32 +0100
  • Referencing dynamic pages Sun, 05 Jan 1997 22:53:57 +0100
  • Image Maps Tue, 26 Mar 1996 11:20:49 GMT
• Thomas R. Bedell
  • Re: Q: size of the web in bytes, comprehensive list Sun, 19 Jan 1997 20:17:02 -0500
• Thomas Stets
  • Does this count as a robot? Sun, 7 Jan 96 2:50:47 CET
• Tim Bray
  • Re: Copyrights (was Re: The Internet Archive robot) Mon, 09 Sep 1996 15:21:51 -0700
  • Re: robots on an intranet Mon, 05 Aug 1996 14:39:45 -0700
  • Re: robots on an intranet Mon, 05 Aug 1996 14:26:18 -0700
  • RE: nastygram from xxx.lanl.gov Wed, 10 Jul 1996 10:56:37 -0700
  • Re: nastygram from xxx.lanl.gov Tue, 09 Jul 1996 11:19:49 -0700
  • Magic, Intelligence, and search engines Fri, 19 Apr 1996 08:19:00 -0700
  • Re: Who sets standards (was Server name in /robots.txt) Sat, 20 Jan 96 20:07 PST
  • RE: Server name in /robots.txt Fri, 19 Jan 96 15:27 PST
  • Re: Server name in /robots.txt Fri, 19 Jan 96 10:00 PST
  • Re: Preliminary robot.faq (Please Send Questions or Comments) Tue, 7 Nov 95 07:11 PST
  • Re: Unfriendly robot at 205.177.10.2 Wed, 18 Oct 95 08:58 PDT
  • Something that would be handy Thu, 12 Oct 95 16:09 PDT
• Tim Bunce
  • Regexp Library Cook-off Tue, 3 Dec 1996 05:31:28 -0800
• Tim Freeman
  • Thanks! Mon, 4 Nov 1996 10:42:03 -0800
  • Re: Is a robot visiting? Sat, 2 Nov 1996 15:42:06 -0800
  • Re: Is a robot visiting? Fri, 1 Nov 1996 12:37:29 -0800
  • Re: Is a robot visiting? Fri, 1 Nov 1996 11:52:10 -0800
  • Is a robot visiting? Thu, 31 Oct 1996 21:57:33 -0800
• Tim Holt
  • Unfriendly robot at 192.115.187.2 Fri, 07 Feb 1997 11:07:38 -0700
• Tim Simmons
  • Re: RE: Scalpers (SJPD does crack down) Thu, 26 Dec 1996 10:05:16 -0800
• Todd Markle
  • Re: The Internet Archive robot Mon, 23 Sep 1996 15:17:07 -0500 (CDT)
• Todd Sellers
  • "hidden text" vs. META tags for robots/search engines Fri, 6 Sep 1996 10:48:49 -0400
• Tony Barry
  • Re: Notification protocol? Wed, 13 Nov 1996 14:55:10 +1100
• Tony Rose
  • robots & copyright law Mon, 27 Jan 97 16:09:48 GMT
• Trevor Jenkins
  • Re: Should I index all ... Wed, 10 Jul 1996 08:45:24 +0000
  • Re: word spam Mon, 15 Apr 1996 09:49:19 +0000
• Tronche Ch. le pitre
  • Re: The Internet Archive robot Fri, 6 Sep 1996 14:02:26 +0200
  • Tagging a document with language Fri, 7 Jun 1996 04:42:00 +0200
  • Book about robots (was Re: Defenses against bad robots) Sat, 18 May 1996 20:22:22 +0200
  • Re: verify URL Wed, 13 Mar 96 20:57:37 +0100
  • Re: Robot Databases Sun, 18 Feb 96 19:09:19 +0100
  • Re: Alta Vista searches WHAT?!? Wed, 17 Jan 96 00:00:01 +0100
• Ulla Sandberg
  • Re: robots on an intranet Tue, 6 Aug 96 16:40:58 MET DST
• Ulrich Ruffiner
  • Re: robots.txt usage Wed, 19 Jun 1996 19:54:33 +0200
• v.sreekanth
  • robot source code Wed, 15 Jan 1997 15:07:38 +0530
• Victor F Ribeiro
  • RE: VB. page grabber... Mon, 6 May 1996 16:01:53 -0300
• Victor Ribeiro
  • Re: Robot's Book. Wed, 26 Jun 1996 16:53:36 -0300
• Vince Taluskie
  • (no subject) Wed, 24 Apr 1996 22:27:13 -0400 (EDT)
  • Re: verify URL Wed, 13 Mar 1996 12:04:28 -0500 (EST)
  • Contact for Intouchgroup.com Fri, 15 Dec 1995 12:49:41 -0600 (CST)
• Vu Quoc HUNG
  • Re: An extended version of the Robots... (fwd) Tue, 12 Nov 1996 11:30:13 +0100 (MET)
• Wanca, Vincent
  • itelligent agents Sat, 9 Nov 1996 22:04:55 -0500
• Wayne Lamb
  • Re: Alta Vista searches WHAT?!? Wed, 17 Jan 1996 09:39:32 -0500 (EST)
  • Re: robots.txt , authors of robots , webmasters ....OM Wed, 17 Jan 1996 09:44:39 -0500 (EST)
  • Re: robots.txt , authors of robots , webmasters ....OMOMOM[D Wed, 17 Jan 1996 09:43:28 -0500 (EST)
  • Re: Alta Vista searches WHAT?!? Wed, 17 Jan 1996 09:42:45 -0500 (EST)
  • Re: Web robots and gopher space -- two separate worlds Wed, 17 Jan 1996 09:41:30 -0500 (EST)
  • Re: robots.txt , authors of robots , webmasters .... Wed, 17 Jan 1996 09:40:51 -0500 (EST)
• Wayne Rust
  • Re: robots & copyright law Mon, 27 Jan 1997 14:01:37 -0700 (MST)
• WEBsmith Editor
  • RE: Search Engine Tue, 13 Aug 1996 12:37:54 -0700 (PDT)
• Wendell B. Kozak
  • Re: email grabber Thu, 16 Jan 1997 12:00:52 -0800
• Werner Schweibenz, Mizzou
  • articles or URL's on search engines Tue, 3 Sep 1996 08:46:44 -0500 (CDT)
• Wes Miller
  • Re: email address grabber Wed, 15 Jan 1997 14:31:04 -0800
  • [Fwd: WebCrawler & Excite] Tue, 03 Dec 1996 21:17:27 -0800
  • Lets get on task! {was Re: Standard} Tue, 03 Dec 1996 13:34:53 -0800
• Wiebe Weikamp
  • Re: AltaVista's Index is obsolete Sun, 21 Jul 1996 00:10:45 +0200 (MET DST)
  • Re: nastygram from xxx.lanl.gov Tue, 09 Jul 1996 11:22:52 +0200 (MET DST)
  • robots.txt usage Tue, 18 Jun 1996 11:28:57 +0200 (MET DST)
• William Dan Terry
  • RE: The Internet Archive robot Thu, 12 Sep 96 19:31:53
  • RE: crawling FTP sites Wed, 11 Sep 96 16:39:51
  • RE: crawling FTP sites Wed, 11 Sep 96 16:39:51
  • RE: Copyrights, let them be ! Wed, 11 Sep 96 10:25:30
• William Neuhauser
  • Re: ActiveAgent Mon, 14 Oct 1996 12:25:39 -0700
• Xiaodong Zhang
  • Re: Looking for a spider 18 Oct 1995 15:13:44 -0700
• Yossi Cohen
  • Q: size of the web in bytes, comprehensive list Sat, 18 Jan 1997 03:23:21 GMT
• YukYukYo69@aol.com
  • Re: Remember Canseco..... Thu, 26 Dec 1996 09:59:38 -0800
• YUWONO BUDI
  • Re: Magic, Intelligence, and search engines Fri, 26 Apr 1996 00:20:55 +0800 (HKT)
  • Re: word spam Tue, 16 Apr 1996 13:50:51 +0800 (HKT)
  • Re: Search accuracy Thu, 4 Apr 1996 02:11:41 +0800 (HKT)
  • Re: Does this count as a robot? Mon, 8 Jan 1996 19:34:51 -0500 (EST)
  • Re: Does this count as a robot? Tue, 9 Jan 1996 02:10:06 +0800 (HKT)
  • Re: Preliminary robot.faq (Please Send Questions or Comments) Thu, 9 Nov 1995 00:37:33 +0800 (HKT)
• Z Smith
  • Re: The Internet Archive robot Thu, 03 Oct 1996 15:28:17 -0700
  • Re: The Internet Archive robot Thu, 03 Oct 1996 10:39:23 -0700
• z960849@rice.farm.niu.edu
  • Is their a web site... Mon, 11 Nov 1996 10:52:12 -0600 (CST)