| Name | ArchitextSpider |
|---|---|
| Cover Page | http://www.excite.com/ |
| Details Page | |
| Operational Status | |
| Description | Its purpose is to generate a Resource Discovery database, and to generate statistics. The ArchitextSpider collects information for the Excite and WebCrawler search engines. |
| Robot Purpose | indexing, statistics |
| Software Type | standalone |
| Software Platform | |
| Software Language | perl 5 and c |
| Availability | |
| Owner's Name | Architext Software |
| Owner's Home Page | http://www.atext.com/spider.html |
| Owner's Email Address | spider@atext.com |
| Exclusion Protocol | yes |
| Exclusion Tag | |
| Supports NOINDEX | no |
| Robot Host | *.atext.com |
| HTTP From | yes |
| HTTP User-Agent | ArchitextSpider |
| History | |
| Environment | |
| Identifier | architext |
| Updated | Tue Oct 3 01:10:26 1995 |
| Update By |
The Web Robots Database