The Web Robots Pages

JCrawler

Name	JCrawler
Cover Page	http://www.nihongo.org/jcrawler/
Details Page
Operational Status	active
Description	JCrawler is currently used to build the Vietnam topic specific WWW index for VietGATE <URL:http://www.vietgate.net/>. It schedules visits randomly, but will not visit a site more than once every two minutes. It uses a subject matter relevance pruning algorithm to determine what pages to crawl and index and will not generally index pages with no Vietnam related content. Uses Unicode internally, and detects and converts several different Vietnamese character encodings.
Robot Purpose	indexing
Software Type	standalone
Software Platform	unix
Software Language	perl5
Availability	none
Owner's Name	Benjamin Franz
Owner's Home Page	http://www.nihongo.org/snowhare/
Owner's Email Address	snowhare@netimages.com
Exclusion Protocol	yes
Exclusion Tag	jcrawler
Supports NOINDEX	yes
Robot Host	db.netimages.com
HTTP From	yes
HTTP User-Agent	JCrawler/0.2
History
Environment	service
Identifier	jcrawler
Updated	Wed, 08 Oct 1997 00:09:52 GMT
Update By	Benjamin Franz