Lunacy Unleashed

Bad Behavior 2 Alpha 2 is now available for wide testing. If you’ve used Bad Behavior in the past, or if you currently use Akismet or Spam Karma 2 and those spam numbers just keep going up, it’s time to learn what Bad Behavior 2 can do for you.

Bad Behavior 2 is a ground-up rewrite of Bad Behavior, the only Web spam killer which stops spammers before they even have a chance to get started. It does this by focusing not on the content of the messages, but on the delivery method. As such, for maximum effect, you should use it in conjunction with another content-based plugin, such as Spam Karma 2 or Akismet. But even on its own, Bad Behavior is once again shockingly effective at stopping spam.

When Bad Behavior was first introduced a year ago, (holy crap it HAS been that long!) it was the first tool of its kind targeting malicious activity on a wide variety of Web sites and platforms. While a few other similar solutions exist, such as mod_security for Apache, they can’t be installed by the user, and they don’t specifically target blog and forum spam, wiki vandalism and the like.

By contrast, Bad Behavior is a set of PHP scripts which pre-screens every request to your PHP-based Web site. The first major version of Bad Behavior was ported to nearly a dozen different blogs, wikis, forums and guestbooks, and many more generic ports were reported that their authors kept privately and never released. Bad Behavior 2 intends to keep the tradition of being portable to any PHP-based platform and expand on it by providing a more comprehensive and structured general API which can be wrapped into virtually anything.

Unfortunately, this wasn’t possible with the previous major version of Bad Behavior, owing to its design, thus the ground-up rewrite. Much to my surprise, Bad Behavior 2 is actually smaller than its predecessor, and catches virtually all spam with virtually no false positives. As of the time of this writing, it allowed only one spam to escape, and on investigation I found that spam had been manually posted by a very bored spammer. (In the final release, he too will be blocked.)

Now, down to business. As I said in the previous post, I haven’t completed the MediaWiki and ExpressionEngine ports yet, primarily due to time constraints, and the constraints of having thousands of people being hit by millions of spams and crying out for a solution now. So for now, this test release only runs on WordPress. It requires WP 1.5 or any later version.

Because this is a test release, there are some special installation instructions. First, if you installed 2.0 Alpha 1, delete it first before uploading this version.

This version can be installed alongside Bad Behavior 1, and in fact I recommend it. Upload the files in the usual way for any plugin. Then go to Manage Plugins. You’ll see both versions listed. Deactivate Bad Behavior 1, then activate Bad Behavior 2. To switch back, deactivate Bad Behavior 2, then activate Bad Behavior 1. Do not allow both version 1 and 2 to be active at the same time.

There are no show-stopping bugs that I’m aware of in this release; it’s stable enough for everyday use. However, it is not feature-complete; several items on the roadmap remain unfinished. For instance, a screener for requests which are suspicious but not certainly spam is only partially implemented. (Which is how that manual spammer got through.) The administrative screen located under Options > Bad Behavior is also not yet implemented.

Even so, I believe that this release will cut your spam flow on your WordPress blog to virtually nothing, without any false positives. However, in the extremely rare event that there is a false positive, the user will receive a technical support key and a brief explanation of what he can do to fix the problem (e.g. scan for spyware). Collect this key from the user and then mail it to me and I’ll get back to you with further information. The error page also provides a link the user can click for extended information; this part is also partially implemented and will be what I work on next.

And as always, if you find Bad Behavior valuable, please consider making a financial contribution. I develop Bad Behavior in my spare time, and every little bit counts.

Download Bad Behavior Now!

And don’t forget to subscribe to the RSS feed or the mailing list. (They’re the same content.)

Updated with new information and — by request — a tentative timeline.

Return to Bad Behavior

As many of you are no doubt aware, there’s a new class of automated spambots out there which Bad Behavior and other spam tools don’t yet handle. Spammers have indeed adapted their techniques to get past tools such as Bad Behavior, Spam Karma, and Akismet, and are actually succeeding. I first caught wind of this new generation a few months ago, and began working on Bad Behavior 2, my attempt to deal with the new generation of malicious spambots.

I had hoped to have a final release long before now, but various problems cropped up and prevented me from completing the project. To date I’ve only been able to release a very early alpha so that other software authors who write code that depends on Bad Behavior can begin to update their tools. The alpha, while it’s functional, not only requires WordPress 2,0, but also provides no more protection than the current release code. In fact, it provides a bit less because one of the checks in Bad Behavior 1 is not present (right now).

As I’ve heard from every such author and they have either updated their code or have plans to do so, it’s time to move this forward.

A representative from a major open source project informed me that the project would be willing to contribute financially to Bad Behavior, but wanted to ensure that it would get something in return, and have a better idea of the timeframe of development. Thus I’m updating the previously posted roadmap.

I have the basic structure of Bad Behavior laid out to allow Bad Behavior to drop in much more easily into packages such as DotClear and Geeklog, where the plugin architecture is quite different than everything else. This will also allow Bad Behavior to be ported to even more software packages. It consists of two components: a core consisting of the test suite itself, and a glue component for each host platform. I’m also planning an administrative interface that will hook into each host platform, though I am not sure if this will be ready for all platforms at the time of release. Finally you’ll be able to configure Bad Behavior and view its activity within WordPress, MediaWiki, or whatever platform, using the native administrative interface provided by the host platform. This is the largest design change in version 2. (Estimated timeframe: 8 development hours per platform.)

Bad Behavior’s API needs improvement. It started as a simple generic interface, and has already outgrown that interface. Version 2 will feature a completely redesigned API for integration into the host PHP program, offering more flexibility, and hopefully the ability for the host program to provide services to Bad Behavior, such as statistics and log viewing.

Bad Behavior needs to deal with the database more intelligently. In version 1, I kept a log of requests which had been denied, expanded it to optionally include all requests, and expanded it again to include the reasons for denial. Then I started using the information in the log to make decisions. Version 2 will feature a complete redesign of the database table, and expansion into two tables, one strictly for logging (for you to stare at), one strictly for making decisions. I expect to gain significant performance improvements thereby, as well as being able to make more intelligent decisions on which requests should be allowed and which should not be. (Estimated timeframe: Complete.)

Most legitimate users unfortunate enough to see the Bad Behavior error page have no idea what to do, even though the page does provide suggestions. It needs to be shortened, clarified and contain links to expanded information sources so that users can solve the problem on their own whenever possible. It should also customize the message based on the specific reasons for denial. Though the ideal is that Bad Behavior should never present the page to a legitimate user — only to spammers. The architecture is in place for Bad Behavior to show more informative error messages, each one including a unique key which either the user or the blog admin can look up to determine what went wrong and how to fix it. While all of the keys have been set, the documentation for each remains to be written. Bad Behavior will now serve errors such as 400 and 403, depending on the request, rather than 412. (Estimated timeframe: 12 development hours.)

Bad Behavior needs to provide better tools for site administrators to search for and eliminate any false positives that may arise. While version 1 contains whitelisting capability, it’s not easy for a site owner to determine why a particular request was blocked, due to being unable to find it in the logs. As I mentioned above, Version 2 will provide a unique key to each denied request which the site owner can use to immediately find the problem, if any, and take any necessary corrective action.

In addition, in certain circumstances when the access is suspicious but it can’t be conclusively shown that the access is malicious, Bad Behavior 2 will attempt to clear the access using as-yet unreleased methods, only providing an error message if the access can’t be confirmed as a human being through multiple techniques. (I would say more on these, but I don’t want to give the spammers a head start on me!) (Estimated timeframe: 6 development hours.)

And I’m experimenting with automated methods of detecting spam attack runs which may originate from dozens of different IP addresses and have somewhat different signatures. I may call for some assistance with this in the near future, and this isn’t likely to make it into 2.0, but it is in the works. (Estimated timeframe: 75-100 development hours.)

Finally, Bad Behavior must continue to keep up with spammers as they attempt to adapt and find new ways to post their automated garbage. To date, this has been at most a minor issue, as there is only so much the spammers can do, while maintaining their high rates of spamming (10,000 or more posts in a single run is not unusual). Bad Behavior attempts to drive up the cost of link spamming, by blocking as many of those spammy requests as possible, forcing the spammers to resort to MUCH slower manual methods, or ideally, give up and find more honest work.

This is my vision for Bad Behavior 2.

Bad Behavior is open source software, released under the GNU General Public License, which you can find copies of all over the Internet, or included with the program. You don’t have to pay a cent to download or use it. However, developing it still costs me time and money, which is why it can go so long between minor releases. Unless (until) some cash comes in, it doesn’t get updated except in cases of dire emergency. Which only happens if I ship code with a typo in it, or Microsoft changes their search engine, or something like that.

Complicating the issue is the fact that a month ago, my laptop, which was my main development platform, died a horrible death. So I’ve had to suspend development of Bad Behavior as well as some of my blogging and other work, until I can get a replacement laptop and get it up and running. As of now I’m about $250 short of where I need to be to have a laptop which is inexpensive and yet capable of serving as a development platform. The computer I’ve been borrowing for the last month or so is simply not sufficient to do much with, unfortunately, and all of my paying endeavors have suffered for it.

If you think this roadmap looks good, and want to accelerate the development of Bad Behavior, contribute financially and I’ll be able to devote more time to it, meaning version 2 comes closer to reality sooner.

And for those of you who are concerned about actually getting something back, my commitment to you is for every $25 contributed, whether from one person or multiple people, to complete one hour of development work within the seven days following the contribution, even if I have to put something else on the back burner temporarily. (Since I generally charge roughly double this amount — or more — for most paid development work, you’re getting something of a bargain.) I hope to have a feature-complete beta working on WordPress, MediaWiki and ExpressionEngine within 45 days, and a final 2.0 release within 75 days, and with your help, this may actually come to pass.

And by all means, if you think I left something out that should be in version 2, please let me know. And yes, I know a lot of you are flat broke, so even if you are unable to contribute financially, leave a comment. Say hi, or suggest changes, or something, just so that I know you’re there and you think I should continue this project.

Return to Bad Behavior

After many delays, technical difficulties, and much more, I’m finally pleased to announce that Bad Behavior 2.0 is taking shape and I have some downloadable code for you!
Read more »

Have you tried WordPress 2.0 yet? Did you find a bug? Leave a comment here and let me know what desperately needs to be fixed.

If you’re upgrading to WordPress 2.0 and use Google AdSense, there is something very important you need to know to ensure you continue to get well-targeted ads, and that Google doesn’t suspend your account for program violations.

One of the new features in WordPress 2.0 is a live post preview. If you scroll to the bottom of the page while editing a post, you’ll see a live preview of how your page will look once it’s published. This is a very nice addition to WordPress, but for AdSense publishers, and those using other context-targeted ad networks, it presents a serious problem.

When the post preview is rendered, it will try to fetch your Google ads!

And because the post hasn’t been published yet, when Google’s bot tries to crawl the page a few seconds later, it will receive a 404 error.

The best case scenario for this is that some of you (who don’t use permalinks) will receive very poorly targeted ads for up to two weeks after you publish your post.

And the worst case scenario, since Google prohibits displaying ads on 404 pages, is that you could get your account suspended.

WordPress 2.0 does provide a solution, though; it’s the new is_preview template tag. This new tag tells whether the post is being displayed in the post preview section while it’s being edited.

So all you need to do is to add in a check for this into your template code wherever you have placed AdSense, and the problem will be solved. Just add this code around your AdSense code:

<?php if (!is_preview()): ?>
// Paste your AdSense code here //
<?php endif; ?>

This way, the post preview will not try to show Google ads, and they will only be shown once your post is published. This will keep your AdSense account safe and your ads well-targeted.

Update: It’s come to my attention that is_preview() may be broken. If you find that’s the case, submit a bug report and post the ticket number in the comments below so we can track it.

Update: I’ve tested is_preview() and it seems to be working just fine. Like other template tags, it only works inside the loop, though.

Update: Since people frequently place ads outside the loop, there needs to be a way to test for this outside the loop. The following workaround worked for me:

<?php global $wp_query; if (!$wp_query->is_preview): ?>
// Paste your AdSense code here //
<?php endif; ?>

Update: Ticket 2188 is open for is_preview() acting strangely.

I’m finally making progress on getting Bad Behavior 2 debugged and in some sort of releasable shape. I was hoping to have it ready by now, but I had computer problems earlier in the week and spent most of a day and night working that out.

So I’ll be spending this weekend and probably Christmas Day working on Bad Behavior 2. Such is my life.

This is the third in a series of updates on the roadmap to Bad Behavior 2, the next major version of the Web’s premier link spam killer for PHP-based sites of all types.

The Bad Behavior API and callback layers are complete; the core code is now completely independent of the host application, which should make it much easier to port to other PHP-based systems. As proof of concept I’m developing an ExpressionEngine extension, in addition to the traditional WordPress plugin and MediaWiki extension. Other platforms should be able to get on board pretty quickly.

The first pre-release code should be out sometime this weekend, sleep and cash flow permitting. Those of you who are porting to other platforms will be able to work from this codebase with minimal or no changes through the final 2.0 release.

Remember, Bad Behavior is a user-driven project. If you feel that Bad Behavior has been useful to you and want to support its continued development, feel free to send along your holiday wishes. Yes, I know ’tis the season to max out the credit cards. Still, providing you with software that worries about spam so you don’t have to is what I do. And without your support, I’ll have to go do something else. (Thanks again to those of you who already contributed!)

This is the second of a series of updates on the roadmap to Bad Behavior 2, the next major version of the Web’s premier link spam killer.

I complained last time that I hadn’t yet received a copy of ExpressionEngine and wouldn’t be able to delay development of Bad Behavior much longer to wait for it. Well, this afternoon, the long-awaited beta copy of EE 1.4 arrived in my inbox, and so I’ll be spending the weekend looking at it and seeing what all needs to be done for a port.

Unfortunately, every host platform is somewhat different, and it’s becoming increasingly clear that installation for some platforms is going to be hairy at best, but compared to other plugins for those same platforms, about average.

I’ve had to drop the idea of using a PHP class for Bad Behavior’s tests, as it simply was too slow. Classes will still be used for certain host platforms that require them or where it will benefit, for the glue layer, but the core itself will be a set of callable functions.

I have been able to reduce the number of global variables used, though; the only thing global that’s particularly needed are the server globals anyway. Everything else can be passed around. This carries a slight performance penalty, but it’s much cleaner code, and you won’t notice a few microseconds here and there.

In other news, I’ve been busier than expected with trying to work on things that actually pay me money, of which Bad Behavior is only rarely such a thing. That also tends to delay development somewhat, as I generally don’t pull all-nighters anymore, for instance.

I’ve also been tossing around the idea of creating an alternative to Akismet, Matt Mullenweg’s commercial spam screening service. Because the API is open, anyone could (assuming the proper know-how) create their own service that talks to the Akismet plugin.

It isn’t all that hard to set up such a service, but as Matt found out, it does have to be funded somehow. So he went with a commercial model based on obtaining WordPress API keys… I can’t do that.

So if you’re interested in seeing a noncommercial Akismet replacement service, click here.

Anyway, I’m preparing to spend most of the weekend working on Bad Behavior. Feel free to leave your comments below. Nice holiday wishes are appreciated, too. After all, most of you missed my birthday…

About a month ago I posted a roadmap for the next major version of Bad Behavior, the PHP-based automated link spam killer. Now it’s time for an update.

First off, I mentioned in a comment on a prior post that I would be waiting to see the next version of ExpressionEngine before I went very far with the next version. Though I was told a beta would be available in November, I have yet to see it. If I don’t see it in the next few days, Bad Behavior will move forward, without support for ExpressionEngine.

Second, I have the basic structure of Bad Behavior laid out. It consists of two components: a core consisting of the test suite itself, and a glue component for each host platform. I’m also planning an administrative interface that will hook into each host platform, though I am not sure if this will be ready for all platforms at the time of release. Finally you’ll be able to configure Bad Behavior and view its activity within WordPress, MediaWiki, or whatever platform.

Third, the architecture is in place for Bad Behavior to show more informative error messages, each one including a unique key which either the user or the blog admin can look up to determine what went wrong and how to fix it. While all of the keys have been set, the documentation for each remains to be written. Bad Behavior will now serve errors such as 400 and 403, depending on the request, rather than 412.

And I’m experimenting with automated methods of detecting spam attack runs which may originate from dozens of different IP addresses and have somewhat different signatures. I may call for some assistance with this in the near future, and this isn’t likely to make it into 2.0, but it is in the works.

Finally, this post wouldn’t be complete without a mention of something strange that happened when I posted last month:

Without any further contributions to Bad Behavior development, I’ll work on it in my limited free time, and it’ll take somewhere around six months. If I were to receive, for instance, $500 in contributions, I could devote a significant amount of time to it, and complete it within the next month. Hey, don’t laugh, that’s only a few cents per user.

I didn’t expect to receive much of anything, and I had just picked the number out of thin air. The surprise was that I actually received $490! Clearly I didn’t complete it within a month, but that’s mostly due to my decision to wait for ExpressionEngine. I’m not waiting on them any longer, so you should expect an early Christmas present sometime within the next couple of weeks.

Be sure and review the roadmap and comment on it now, before I go too far and any necessary design changes become difficult or impossible.

And I wouldn’t mind if you want to contribute that last $10 either. It is my birthday, after all.

WordPress 2.0 Beta 1 was quietly released early Saturday morning. So quietly that this is probably the first post about it anywhere.

Previously known as 1.6, lead developer Matt Mullenweg decided that the changes to WordPress were extensive enough to warrant a major version number change.

I’ve been running it at Make Stupidity History for some time, and while there are certainly still a few bugs to iron out, it’s probably almost ready for prime time.

It is certainly ready for plugin authors to start working with and updating their plugins. Many plugin authors will find that their plugins are broken in 2.0 and need to be updated, due to the extensive internal changes.

If you need help with it, post in the Beta forum.

Download WordPress 2.0 Beta 1.

Update: I received word that several people posted about the beta before I did. But none of them offered nearly as much useful information about it.