Ya but one of the stupidest things about sites that do this is =
that they
never support the newest browser. I am using netscape 3.0 at work =
which
supports 2.0 and more but sites that do what you are saying think =
its no=20
browser. Because people do things like this you have to set =
fields to make
them think your useing differant browsers. Blame the tag makers =
;)- and the browser
makers.. The best thing a browser maker can do is ignore a tag =
they dont know about
so people dont have to say if this broswer do this.. =20
Take a look at frames where you can set it up in the same page for =
browsers that
support and dont support without having to send pages based on =
brower type via
some stupid cgi
Begin forwarded message:
From: David Levine <David@InterWorld.com>
To: "'robots@webcrawler.com'" <robots@webcrawler.com>
Subject: (OTP) RE: Political economy of distributed search (was =
topical search...)
Date: Fri, 3 May 1996 09:42:18 -0400
X-Mailer: Microsoft Exchange Server Internet Mail Connector =
Version 4.12.736
Sender: owner-robots@webcrawler.com
Reply-To: robots@webcrawler.com
And one of the stupidest things about browsers lying about their =
UserAgent
field
is that some sites automatically send out various pages based on =
what
browsers
can display. And you'll find that while browser's claiming =
another
browser's UserAgent
may support the same HTML tags and so forth, they rarely lay out =
the page in
the
same manner. In some cases the page gets an entirely different =
look. One
of the
points of a UserAgent field is the ability to tailor a page to the =
user's
capabilities and
and another point is the ability to maintain a consistent look to =
all users
by
designing specificially for various browsers.
-- David Levine, Application Engineer InterWorld Technology Ventures, Inc. david@interworld.com [An attachment was originally included = here]http://www.interworld.com/staff/david/
>---------- >From: Benjamin Franz[SMTP:snowhare@netimages.com] >Sent: Friday, May 03, 1996 8:21 AM >To: robots@webcrawler.com >Subject: Re: Political economy of distributed search (was = topical >search...) > >On Fri, 3 May 1996, Jeremy.Ellman wrote: > >> >> > The simple fact is that integration is >> It's impossible to block, because there's >> > no way for a service to determine a normal user query from a >> > WebCompass query (unless the WebCompass folks choose to do = so, eh >> > Brian?) >> > >> >> Disagree. You can refuse to reply to any GET/POST etc that does = not >contain a USER-AGENT >> you like. Eg Reply straight in MOZILLA etc but ask interactive = questions >of other >> browsers. This would block these robots (unless they were going = to spoof >-- but what >> commercial product could do this on copyright grounds?) > >Based on the number of agents *already* showing in my logs as >'Mozilla....(xzcxcxv;compatible;sdfsdf) - a lot of products. It = has caused >me a fair amount of trouble having to progesssively modify my >browsercounter software to detect the spoofers. > >Microsoft started this rather disturbing trend of user agents = lying about >who they are - but they were most certainly not the last. Also - = the >potential infringment is not copyright but trademark. > >-- >Benjamin Franz > >
--NeXT-Mail-1597835661-1 Content-Type: text/enriched; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
Ya but one of the stupidest things about sites that do this is = that they
never support the newest browser. I am using netscape 3.0 at work = which
supports 2.0 and more but sites that do what you are saying think = its no=20
browser. Because people do things like this you have to set = fields to make
them think your useing differant browsers. Blame the tag makers = ;)- and the browser
makers.. The best thing a browser maker can do is ignore a tag = they dont know about
so people dont have to say if this broswer do this.. =20
Take a look at frames where you can set it up in the same page for = browsers that
support and dont support without having to send pages based on = brower type via
some stupid cgi
Begin forwarded message:
<bold>From:=20 </bold>David Levine <<David@InterWorld.com>
<bold>To:=20 </bold>"'robots@webcrawler.com'" <<robots@webcrawler.com>
<bold>Subject:=20 </bold>(OTP) RE: Political economy of distributed search (was = topical search...)
<bold>Date:=20 </bold>Fri, 3 May 1996 09:42:18 -0400
<bold>X-Mailer:=20 </bold>Microsoft Exchange Server Internet Mail Connector Version = 4.12.736
<bold>Sender:=20 </bold>owner-robots@webcrawler.com
<bold>Reply-To:=20 </bold>robots@webcrawler.com
And one of the stupidest things about browsers lying about their = UserAgent
field
is that some sites automatically send out various pages based on = what
browsers
can display. And you'll find that while browser's claiming = another
browser's UserAgent
may support the same HTML tags and so forth, they rarely lay out = the page in
the
same manner. In some cases the page gets an entirely different = look. One
of the
points of a UserAgent field is the ability to tailor a page to the = user's
capabilities and
and another point is the ability to maintain a consistent look to = all users
by
designing specificially for various browsers.
--David Levine, Application Engineer
InterWorld Technology Ventures, Inc.
<http://www.interworld.com/staff/david/
>----------
>From: Benjamin Franz[SMTP:snowhare@netimages.com]
>Sent: Friday, May 03, 1996 8:21 AM
>Subject: Re: Political economy of distributed search (was = topical
>search...)
>
>On Fri, 3 May 1996, Jeremy.Ellman wrote:
>
>>
>> > The simple fact is that integration is
>> It's impossible to block, because there's
>> > no way for a service to determine a normal user query from a
>> > WebCompass query (unless the WebCompass folks choose to do = so, eh
>> > Brian?)
>> >
>>
>> Disagree. You can refuse to reply to any GET/POST etc that does = not
>contain a USER-AGENT
>> you like. Eg Reply straight in MOZILLA etc but ask interactive = questions
>of other
>> browsers. This would block these robots (unless they were going = to spoof
>-- but what
>> commercial product could do this on copyright grounds?)
>
>Based on the number of agents *already* showing in my logs as
>'Mozilla....(xzcxcxv;compatible;sdfsdf) - a lot of products. It = has caused
>me a fair amount of trouble having to progesssively modify my
>browsercounter software to detect the spoofers.
>
>Microsoft started this rather disturbing trend of user agents = lying about
>who they are - but they were most certainly not the last. Also - = the
>potential infringment is not copyright but trademark.
>
>--
>Benjamin Franz
>
>
--NeXT-Mail-1597835661-1--