| Name | Conceptbot |
|---|---|
| Cover Page | http://www.aptltd.com/~sifry/conceptbot/tech.html |
| Details Page | http://www.aptltd.com/~sifry/conceptbot |
| Operational Status | development |
| Description | The Conceptbot spider is used to research concept-based search indexing techniques. It uses a breadth first seach to spread out the number of hits on a single site over time. The spider runs at irregular intervals and is still under construction. |
| Robot Purpose | indexing |
| Software Type | standalone |
| Software Platform | unix |
| Software Language | perl5 |
| Availability | data |
| Owner's Name | David L. Sifry |
| Owner's Home Page | http://www.aptltd.com/~sifry |
| Owner's Email Address | david@sifry.com |
| Exclusion Protocol | yes |
| Exclusion Tag | conceptbot |
| Supports NOINDEX | yes |
| Robot Host | router.sifry.com |
| HTTP From | yes |
| HTTP User-Agent | conceptbot/0.3 |
| History | This spider began as a research project at Sifry Consulting in April 1996. |
| Environment | research |
| Identifier | conceptbot |
| Updated | Mon, 9 Sep 1996 15:31:07 GMT |
| Update By | David L. Sifry |
The Web Robots Database