“Online risks are becoming more complex and pervasive by the day,” says the home page of the Cyveillance web site.
Indeed, one of the risks is that your Web site might be targeted by Cyveillance.
The company says that it crawls the Internet looking for phishing scams, identity theft, illegal credit card numbers, trademark and copyright infringement, and more. It’s also been known to work on behalf of the government to spy on whistleblowers who expose waste, fraud and abuse. Some have even alleged that Cyveillance bots attempt to illegally hack into Web servers.
Cyveillance uses robots which crawl your Web site pretending to be a legitimate Web browser and completely ignoring your robots.txt file. Then it tries to figure out whether you’ve downloaded any illegal music, or said something bad about some company. It then sends you threatening letters ordering you to take your site down, even when you haven’t done anything wrong.
Bad Behavior doesn’t just block spammers. It’s meant to target any bot which overloads a Web site, attempts to hack in, delivers spam, acts in an unethical manner, etc.
By relying on analyzing the HTTP requests themselves, rather than simply the IP address, Bad Behavior has blocked Cyveillance almost from the very first release last year, regardless of what IP address range they move to in order to hide their connections. An audit of six months worth of logs shows that Bad Behavior was able to successfully identify and block Cyveillance bots even when they used previously unknown IP address ranges.
Bad Behavior 2.0.8 has been released.
This version contains updates for various “false positive” reports and is recommended for all users.
Updated in this release (since 2.0.7):
- Verizon Wireless EV-DO users are no longer blocked.
- Blocked requests will be subject to a two-second delay before a response is sent. (See below.)
- Some blackhole lists previously used in Bad Behavior have been scaled back or removed.
- The address for the Bad Behavior Blackhole has been added. (See below.)
- Some new spambots have been identified and blocked.
In recent days spam attacks have been on the rise, with one especially obnoxious bot delivering requests so fast that some sites have been taken offline by them. While the requests aren’t especially numerous or resource-intensive, the most common software used by Web hosting providers is very inefficient at serving dynamic pages such as PHP-based Web sites. So even a moderate number of requests can take a whole server down, or lead the hosting provider to take the site down before the whole server goes down.
Bad Behavior now counters this by introducing a short two second delay to blocked requests, before the HTTP response is sent. Since most spambots wait for the response before going on to the next request, this should sufficiently slow down most of the overly aggressive spambots and give Web site operators some breathing room. While I would have liked to put in a delay of a minute or more, there remains the slight chance that an actual human being would be blocked, and they should be able to get a response back in a reasonable time.
With respect to realtime blackhole lists, all of the existing lists target e-mail spam, and since spambots who send link spam are almost always also sending e-mail spam through the same servers, these are a fairly effective means of blocking link spam. However, since they target e-mail spam, they also block legitimate users. The primary issue here is that while an IP address may be added to a blackhole list quickly, it is not removed quickly — or at all — once the spam stops. Thus, people with dynamic IP addresses are unfairly blocked because some other customer was sending spam.
Bad Behavior Blackhole, which should go online within the next few weeks, is designed specifically for link spam. It adds IP addresses to its database quickly when actual spam is received, and in addition, drops the IP addresses once the spam stops. This helps prevent dynamic IP customers from being blocked because another user’s computer was sending spam. Once Bad Behavior Blackhole is online, all other realtime blackhole lists will be dropped from Bad Behavior.
As always, if you find Bad Behavior valuable, please consider making a financial contribution. I develop Bad Behavior in my spare time, and every little bit means I have more spare time to devote to its development.