robot-id: search_au
robot-name: Search.Aus-AU.COM
robot-details-url: http://Search.Aus-AU.COM/
robot-cover-url: http://Search.Aus-AU.COM/
robot-owner-name: Dez Blanchfield
robot-owner-url: not currently available
robot-owner-email: dez@geko.com
robot-status: - development: robot under development
robot-purpose: - indexing: gather content for an indexing service
robot-type: - standalone: a separate program
robot-platform: - mac - unix - windows95 - windowsNT
robot-availability: - none
robot-exclusion: yes
robot-exclusion-useragent: Search-AU
robot-noindex: yes
robot-host: Search.Aus-AU.COM, 203.55.124.29, 203.2.239.29
robot-from: no
robot-useragent: not available
robot-language: c, perl, sql
robot-description: Search-AU is a development tool I have built
 to investigate the power of a search engine and web crawler
 to give me access to a database of web content ( html / url's )
 and address's etc from which I hope to build more accurate stats
 about the .au zone's web content.
 the robot started crawling from http://www.geko.net.au/ on
 march 1st, 1998 and after nine days had 70mb of compressed ascii
 in a database to work with. i hope to run a refresh of the crawl
 every month initially, and soon every week bandwidth and cpu allowing.
 if the project warrants further development, i will turn it into
 an australian ( .au ) zone search engine and make it commercially
 available for advertising to cover the costs which are starting
 to mount up. --dez (980313 - black friday!)
robot-environment: - hobby: written as a hobby
modified-date: Fri Mar 13 10:03:32 EST 1998