The Web Robots Pages
Search.Aus-AU.COM
Name
Search.Aus-AU.COM
Cover Page
http://Search.Aus-AU.COM/
Details Page
http://Search.Aus-AU.COM/
Operational Status
- development: robot under development
Description
Search-AU is a development tool I have built to investigate the power of a search engine and web crawler to give me access to a database of web content ( html / url's ) and address's etc from which I hope to build more accurate stats about the .au zone's web content. the robot started crawling from http://www.geko.net.au/ on march 1st, 1998 and after nine days had 70mb of compressed ascii in a database to work with. i hope to run a refresh of the crawl every month initially, and soon every week bandwidth and cpu allowing. if the project warrants further development, i will turn it into an australian ( .au ) zone search engine and make it commercially available for advertising to cover the costs which are starting to mount up. --dez (980313 - black friday!)
Robot Purpose
- indexing: gather content for an indexing service
Software Type
- standalone: a separate program
Software Platform
- mac - unix - windows95 - windowsNT
Software Language
c, perl, sql
Availability
- none
Owner's Name
Dez Blanchfield
Owner's Home Page
not currently available
Owner's Email Address
dez@geko.com
Exclusion Protocol
yes
Exclusion Tag
Search-AU
Supports NOINDEX
yes
Robot Host
Search.Aus-AU.COM, 203.55.124.29, 203.2.239.29
HTTP From
no
HTTP User-Agent
not available
Environment
- hobby: written as a hobby
Identifier
search_au
Updated
Fri Mar 13 10:03:32 EST 1998
The Web Robots Database
