I have read the FAQ on robots, the HTTP 1.0 spec, info on robots.txt
and several other documents and am about to create the TOPjobs(tm) USA
JOBbot 1.0a. This robot has a very narrow focus. It is not
free-wandering. Rather, it is designed to run from a list of url's
gathering the information about job openings published directly by
employers on the web.
I'm just starting the programming and testing phase. I doubt that
many (if any) of you will see it in the next several weeks. However, I
did want to let you know in advance.
Because this robot will run from a list of url's, I anticipate being
able to contact the webmaster (and/or human resources) in advance. We
are also developing a software submission program that could be used in
place of having the robot check the site.
This robot is being designed at the request of several companies that
have requested that we gather their job information from the internet.
If you have any questions or comments, feel free to contact me.
-- Sincerely,Daniel Williams TOPjobs(tm) USA by DiSX 801/377-3479 http://www.topjobsusa.com daniel_williams@topjobsusa.com
=====================================================================