One of the *programmers* on this list (bbh@xenodata.com),
so eloquently wrote:
> At the risk of getting kicked off this list, it seems there
> are too many admins and not enough tech types subscribing to
> this here list.
No kidding. That was my reason for ignoring this list for the
past couple of weeks. Too much discussion of robot purposes
and how to block them, not enough about implementation details.
> Protocol and hostname should always be mapped to lowercase.
> TCP port should always be tacked on.
> Trailing slash should always be preserved.
> Domain name dot com should be assumed to be the same as
> www.domainname.com and ftp.domainname.com and
> mail.domainname.com etc until discoverered otherwise.
This is the solution that ActiveAgent is working towards.
It already does the first and the third on that list. Hadn't
considered the port numbers, but just ran a test, and noticed that
AA treats http://hipcrime.com & http://hipcrime.com:80 differently.
Oh well. Something to fix next round.
The last item on that list is the most interesting, and yet again,
tests reveal that it's destined to be in the next upgrade.
As for the #-sign-type URLs, ActiveAgent ignores those already.
Can't imagine why any agent would grab sub-page references, since
it's probably got the whole page already.
Once again, thanks for the great ideas. But then, that's what
mailing lists are all about.
... HipCrime
P.S. the thing you can't figure out, where my place of residence
is, can be summed up in one word: homelessness. Have been
for almost a year now. Didn't you know that? Consult the
CypherPunk list archives.
Either way, all HipCrime work has been done from public sites
or friends' computers. Don't even have a system of my own.
That's due to change though, since patrons are coming out of
the woodwork, who want to help support my web-vagrant's art.
P.P.S. The InterNIC gets approximately 20,000 registrations per
week. Hey Mr. Ambassador@FourthWorld.com, you want to be
the one to read/verify them? You sure seem to care about
what they say. Why don't you check them all?
_________________________________________________
This messages was sent by the robots mailing list. To unsubscribe, send mail
to robots-request@webcrawler.com with the word "unsubscribe" in the body.
For more info see http://info.webcrawler.com/mak/projects/robots/robots.html