> [Refresh spam method deleted]
>
>If my crawler detects any of these "ploys" it simply drops the page.
Well, we've been doing that for a while too; it does mean you drop people
who have fashionable (?) splash screens on their home page for cosmetic
"user experience" rather than spam purposes.
>If every indexer did this
If you do some searches against the major search services it's clear
everyone is taking some measures...
>then the "outsmart the robot writer" game
>would be stopped cold. These people are obviously very interested in
>being indexed, they'd learn very quickly.
Fortunately we do have some success stories about reformed former spammers
who have seen the error of their ways after being dropped altogether.
However, sometimes it's like the monkeys and infinite time; the spammers
seem to find all possible ways to spam faster than you do. And some
things are just impossible to defend against.
-- Martijn
Email: m.koster@webcrawler.com
WWW: http://info.webcrawler.com/mak/mak.html