More dangers of spiders...

dws (dws@intercom.com)
Sun, 16 Jun 1996 19:18:52 -0400 (EDT)


On March 20th, 1996, CERT released in advisory about a "vulnerability in
NCSA/Apache CGI example code", which had the impact that "a remote user
may retrieve any world readable files, execute arbitrary commands and create
files on the server with the privileges of the httpd process which answers
HTTP requests. This may be used to compromise the http server and under
certain configurations gain privileged access." and "the CGI program 'phf',
included with those distributions, is an example of such a vulnerable
program.".

According to Web Week 05/20/96, Apache has 31.35% of the web server
market and is growing, and NCSA has 24.98% of the market. Apache and NCSA
patched and put out patched versions in a few days, but over 50% of the
web had the vulnerability up to that point. And in fact, very few sites
have even as of yet patched this vulnerability.

I saw the thread about search engines gobbling up /etc/passwd, so I
thought I'd see if this was gobbled up too. I went to Alta Vista and
typed in "url:cgi-bin/phf"...and got back over 300 responses. Some of
the sites had obviously set up a phone directory and were aware of the
program (which _still_ doesn't mean they were aware of the problem), but
many sites had the standard "Form for CSO PH query" header, with
"ns.uiuc.edu" as the PH server and "Questions, comments" directed to Jim
Browne at UIUC. Some of them even had the default UIUC title page as
their main URL announcing this was version 1.0, or 1.1 or so of httpd.
As I did not have the web masters permissions, I did not send the one
line "script" to see if what seemed to be was, that the bug was on their
system and I could remotely execute commands as httpd (from which, even
if one could not "hack root", would be able to modify their web pages),
however I think it's safe to say that the vulnerability is on most of the
systems.

I went on webcrawler and looked for "Form for CSO PH query" and, again,
webcrawler gave me a list of sites which seemed to have the bug.
Infoseek and Lycos also gave me a large number of leads. I have to say
that Excite didn't seem to turn up anything, but then again, I can never
find what I'm looking for on Excite ;-). Hotbot as well seemed to have
some kind of mechanism that prevented it from swallowing up cgi-bin, it's
father search engine Inktomi had no problems gaving a list of probably
phf-holed sites. Opentext, which has fallen off the list of the big top
seven search engines (which I define statistically via analyses I have
made on multiple machines, they are - Yahoo, Alta Vista, Webcrawler,
Infoseek, Lycos, Excite and Inktomi/Hotbot), had the problem too.

I realized after this that this problem was much more widespread than I
had first envisioned. Probably 50% of web sites on the net have yet to
patch this security hole. I'd say 50% of the people I talk to that has a
web site and I tell them to check has it. Originally I thought the bug
was more obscure, or at least has been widely patched, and that these
search engines were automatically compiling lists of the unchecked ones, but
now I see that the problem is more widespread.

Nevertheless, here is another example of the dangers of spiders/robots
and search engines...

I mailed people at Alta Vista and Webcrawler (who I saw on this list) and
they responded that they'd fix it. I also E-mailed a few of the web
sites, but there are obviously too many for me to do that to many more.

BTW: I put up a page on my web site about this with links to the CERT
advisory etc.

http://www.gonif.com/spiders/security/phf

Salut,
Dennis

10 Megs/sec World Wide Web Hosting - Intercom Online http://www.intercom.com
web hosting, dedicated servers, co-location, ISDN, T-1
Wall Street Web Farm - http://www.intercom.com/webfarm.htm
Have your own dedicated server on our ethernet get hit at 10 Megs a second!