Re: defining "robot"
Hrvoje Niksic (hniksic@srce.hr)
22 Nov 1996 09:16:11 +0100
Matthew K. Gray (mkgray@mit.edu) wrote:
> I would propose the following as a definition for a robot, in terms of
> what should and should not follow robots.txt.
>
> --------------------
> A "robot" is any piece of software which automatically retrieves web
> pages/documents* and does not immediately present them via some
> mechanism for human consumption before proceeding further.
>
> * I define here "web pages/documents" as a file and any content it
> inlines (this includes images, java applets, or other embedded
> content, but not other documents it points to).
> --------------------
This is an excellent definition (together with the rationale). It
could be polished to include definitions for "inlines", and what it
menas to "point to", "present", etc., but I it is sufficiently clear
in the original form.
--
Hrvoje Niksic <hniksic@srce.hr> | Hocemo 101-icu!
--------------------------------+--------------------------------
Good pings come in small packets.
_________________________________________________
This messages was sent by the robots mailing list. To unsubscribe, send mail
to robots-request@webcrawler.com with the word "unsubscribe" in the body.
For more info see http://info.webcrawler.com/mak/projects/robots/robots.html