| Name | Harvest |
|---|---|
| Cover Page | http://harvest.cs.colorado.edu |
| Details Page | |
| Operational Status | |
| Description | Harvest's motivation is to index community- or topic- specific collections, rather than to locate and index all HTML objects that can be found. Also, Harvest allows users to control the enumeration several ways, including stop lists and depth and count limits. Therefore, Harvest provides a much more controlled way of indexing the Web than is typical of robots. Pauses 1 second between requests (by default). |
| Robot Purpose | indexing |
| Software Type | |
| Software Platform | |
| Software Language | |
| Availability | |
| Owner's Name | |
| Owner's Home Page | |
| Owner's Email Address | |
| Exclusion Protocol | |
| Exclusion Tag | |
| Supports NOINDEX | |
| Robot Host | bruno.cs.colorado.edu |
| HTTP From | yes |
| HTTP User-Agent | yes |
| History | |
| Environment | |
| Identifier | harvest |
| Updated | |
| Update By |
The Web Robots Database