Lunacy Unleashed

Notes from the field in the War on Spam

Bad Behavior 2 Roadmap

Update: Bad Behavior 2 development is on hold indefinitely. Find out why and how you can help.

Yesterday I said I was beginning work on Bad Behavior 2.0, the next generation of the Web’s premier link spam killer. And I did. I wrote some ten lines of code.

Before I go into the roadmap, I have to diverge a bit and explain something a lot of people may not be aware of.

Bad Behavior is open source software, released under the GNU General Public License, which you can find copies of all over the Internet, or included with the program. You don’t have to pay a cent to download or use it. However, developing it still costs me time and money, which is why it can go so long between minor releases. Unless (until) some cash comes in, it doesn’t get updated except in cases of dire emergency. Which only happens if I ship code with a typo in it, or Microsoft changes their search engine, or something like that.

I have hundreds of comments and trackback pings from users all over who have virtually eliminated their spam problems with Bad Behavior. And every so often, someone does click the nice PayPal button, to send a few bucks my way. Both are very much appreciated.

Killing blog spam has been mostly a labor of love, however, rather than cash, and as such, has to take a back seat to other more pressing concerns, like anything that generates revenue.

So what I’m going to do here is outline my roadmap for Bad Behavior 2.0, invite you to comment on it, and if you want to see it come about sooner rather than later, to vote with your dollars, pounds, euros, or whatever you have. The amount is blank, so fill in whatever you feel is appropriate.

And if you see any problems with it, or think it could be improved, you can comment on it as well.

First off, Bad Behavior needs to be even more modular than it is currently. Version 1 proved fairly easy to integrate into diverse PHP software packages with differing requirements for their plugins or modules, but it seems like each package requires something different. For version 2 I will have a structure put together to allow Bad Behavior to drop in much more easily into packages such as DotClear and Geeklog, where the plugin architecture is quite different than everything else. This will also have the side effect of opening Bad Behavior to porting to even more software packages. This will be the largest design change in version 2.

Second, Bad Behavior needs to deal with the database more intelligently. In version 1, I kept a log of requests which had been denied, expanded it to optionally include all requests, and expanded it again to include the reasons for denial. Then I started using the information in the log to make decisions. Version 2 will feature a complete redesign of the database table, and expansion into two tables, one strictly for logging (for you to stare at), one strictly for making decisions. I expect to gain significant performance improvements thereby, as well as being able to make more intelligent decisions on which requests should be allowed and which should not be.

Third, Bad Behavior’s API needs improvement. It started as a simple generic interface, and has already outgrown that interface. Version 2 will feature a completely redesigned API for integration into the host PHP program, offering more flexibility, and hopefully the ability for the host program to provide services to Bad Behavior, such as statistics and log viewing.

Fourth, that error page needs to be reworked. Most legitimate users unfortunate enough to see the page, have no idea what to do, even though the page does provide suggestions. It needs to be shortened, clarified and contain links to expanded information sources so that users can solve the problem on their own whenever possible. It should also customize the message based on the specific reasons for denial. Though the ideal is that Bad Behavior should never present the page to a legitimate user — only to spammers.

Along those lines, the 412 error code will be changed. Bad Behavior 2 will attempt to deliver the most appropriate error code to the denial circumstance. In some circumstances, version 2 will return a 403 error. In others, perhaps a 412. In one case, a 417 is appropriate.

Fifth, Bad Behavior needs to provide better tools for site administrators to search for and eliminate any false positives that may arise. While version 1 contains whitelisting capability, it’s not easy for a site owner to determine why a particular request was blocked, due to being unable to find it in the logs. Version 2 will provide a unique key to each denied request which the site owner can use to immediately find the problem, if any, and take any necessary corrective action.

Finally, Bad Behavior must continue to keep up with spammers as they attempt to adapt and find new ways to post their automated garbage. To date, this has been at most a minor issue, as there is only so much the spammers can do, while maintaining their high rates of spamming (10,000 or more posts in a single run is not unusual). Bad Behavior attempts to drive up the cost of link spamming, by blocking as many of those spammy requests as possible, forcing the spammers to resort to MUCH slower manual methods, or ideally, give up and find more honest work.

This is my vision for Bad Behavior 2. All things being as they are right now, the timeline for all this is anywhere from one to six months. How quickly it gets done depends on you.

Without any further contributions to Bad Behavior development, I’ll work on it in my limited free time, and it’ll take somewhere around six months. If I were to receive, for instance, $500 in contributions, I could devote a significant amount of time to it, and complete it within the next month. Hey, don’t laugh, that’s only a few cents per user.

If you think this roadmap looks good, and want to accelerate the development of Bad Behavior, contribute financially and I’ll be able to devote more time to it, meaning version 2 comes closer to reality sooner. And by all means, if you think I left something out that should be in version 2, please let me know. And yes, I know a lot of you are flat broke, so even if you are unable to contribute financially, leave a comment. Say hi, or suggest changes, or something, just so that I know you’re there and you think I should continue this project.

About these ads

October 25, 2005 - Posted by | Bad Behavior, Blog Spam, Spam, WordPress

32 Comments

  1. Place more links to donate why don’t you! :D

    Comment by VxJasonxV | October 25, 2005

  2. Make a Donation.

    Comment by Michael Hampton | October 25, 2005

  3. [...] Many of you are going to be familiar with Bad Behavior, the link spam killer for WordPress, MediaWiki, Drupal, Geeklog, DotClear, Pixelpost, and several other packages I’ve forgotten about. I’ve posted the roadmap for Bad Behavior 2 over at my other site, Lunacy Unleashed. Go read it and leave a comment over there. [...]

    Pingback by Bad Behavior 2 Roadmap - IO ERROR | October 25, 2005

  4. Well, you wanted comments…so here ya go =)

    Comment by Jonathan | October 25, 2005

  5. Excellent outline and road map. I wish more serious plugin and add on developers would be so clear about what needs doing and what it will “take” to get it done. Great work.

    While it is unfortunately we live in a world that makes a program like Bad Behavior necessary, Bad Behavior is revolutionizing the way bloggers work, eliminating most of the evil comment spam.

    Thanks for all the amazing hard work you do to make the blogging world safer for everyone. You are amazing.

    Comment by Lorelle VanFossen | October 25, 2005

  6. Ahh! It’s a minefield! I panicked and tried to close the window, but ended up stepping on a link and donating $10!

    Comment by Mark Jaquith | October 25, 2005

  7. [...] Also, like most WordPress Plugins, Bad Behavior is free, but it takes time, money, and energy to produce, so if you are a fan of it, then consider dontate to the cause to keep this awesome software going. [...]

    Pingback by Lorelle on WordPress » Bad Behavior - Latest Update | October 25, 2005

  8. I just found out about Bad Behavior a few days ago. I would love to use it, but I don’t have enough skill (yet!) with PHP or the time to make it work with Expression Engine.

    It would help me (and other EE users) a lot if you could take some of the comments in this thread into account during your rewrite.

    I hope you get all the contributions you need! It’s unfortunate that Bad Behavior is needed, but it is and I’m thrilled that you’re serious about making it good.

    Comment by Ed "What the" Heckman | October 25, 2005

  9. Good news.
    As for the error page, I would like it to include some “I’m a human damn it, let me access the page” link so that false positive cases can be dealt with nicely. And to me, 403 is just enough. I guess a spammer having an eye over his logs (if such a thing happens) just doesn’t know what a 412 is.

    Comment by Ozh | October 25, 2005

  10. I love you guys. WordPress definitely has the most amazing community, and I’m glad to be a part of it. :)

    Ed, I’m aware of the interest for ExpressionEngine, and it’s one of the things motivating the version 2 redesign. I may well do the EE port myself, just to prove that my new design works. :)

    Ozh, I’d originally chosen 403, but way back in the beginning, people complained because they couldn’t tell apart the spambots blocked by Bad Behavior, and those blocked by other measures, in their Apache server logs. So I changed it to 412 at almost the last minute, which had the side effect of confusing a couple of spambots enough to crash them. It also upset a couple of RFC purists who rightly pointed out that 412 is almost never the “right” error to return. Hopefully with better logging (and log viewing) capabilities, this won’t be an issue in the future, though I’ll miss 412.

    Comment by Michael Hampton | October 25, 2005

  11. [...] A little note from me, maybe you are interested to read the notes on Next Generation of Bad Behavior development. The author called it as Bad Behavior 2 Road Map. Please drop your comment there, and make a donation to the owner if you could. [...]

    Pingback by Firman Pribadi — Bad Behavior 1.2.3, and the NG Roadmap | October 25, 2005

  12. Michael: Consider yourself 5% closer to the goal.

    Comment by Geof F. Morris | October 25, 2005

  13. Support Bad Behavior 2!

    If, like me, you get a lot of utility out of Bad Behavior, support Bad Behavior 2 financially! Michael made a compelling argument for me to support him financially, which I’ve personally found is all people really need to support you. [That&#8…

    Trackback by The Indiana Jones School of Management | October 25, 2005

  14. [...] As I said yesterday, however, I remain committed to the development of Bad Behavior. It is still sorely needed as a first line of defense for WordPress, not to mention all of the other platforms on which it now runs. [...]

    Pingback by Lunacy Unleashed » Automattic Kismet | October 26, 2005

  15. Well, I must say I’m both honored and amused. It seems several people took my “for instance, $500″ offhand comment as a fundraising goal and contributed a specific percentage toward it. Thanks, guys.

    So after one day, the returns are more than I’d expected, less than I’d hoped.

    At the moment, the tally stands at $90, or 18%, of the $500 “goal.” Again, thank you.

    Most people don’t know that I used to work for MCI, and that they closed down the center where I had been working. Since that time, my primary sources of income (save unemployment, which is a rant for another blog, but suffice it to say I hate government assistance) have been WordPress consulting and Google ads. The ads right now supply less than 1/3 of the income I need to live, and it’s the occasional consulting work, and the even rarer PayPal donation, that fill in the large gap.

    So the $90 means a lot. The other $410 would mean a lot more.

    It would mean I could spend more time actually coding things you want, as shown above, and less time looking for someone who wants something done to their blog, who seem to be in fairly short supply lately.

    If you can contribute, please do. If you can’t, and you know someone who wants some work done to their blog, send them my way. If you don’t know any such person, leave a comment with your opinion of my roadmap.

    Comment by Michael Hampton | October 26, 2005

  16. You should’ve told me that, Michael.

    Comment by Geof F. Morris | October 26, 2005

  17. I finally read through the post and will post my comments!

    Ok, so, it all looks good, I’m all for more modulability, and all that jazz. Above and beyond all, I’m looking forward to each new release.

    One point that I wanted to touch on, is that you mentioned proxied/networked (i.e. corporate) environments.
    Do have you any plans to expand this, and attempt to work with properly configured proxies? (Do you already?)

    I’m sure you’re not going to play nice with bad proxies/programming :P.

    Comment by VxJasonxV | October 26, 2005

  18. Geof, which that? Like I said, I’m honored, but it’s still funny. :)

    Jason, any proxy in common use in an enterprise setting should pass through just fine, as long as it’s configured properly. That’s true even now. It’s the misconfigured ones, and the malicious ones, that are going to run into problems.

    Ideally for version 2 I’d like to be able to detect the specific proxy server software in use, when possible, and provide directions on how to fix the configuration. Humans hit by the block will see it and be able to take action, while spambots are only going to record the 4xx error and move on.

    Come to think of it, why don’t I just make the error code 404 instead of 412. (Before you HTTP purists complain, I invite you to read RFC 2616, sections 10.4.4 and 10.4.5.)

    Comment by Michael Hampton | October 26, 2005

  19. [...] With all the buzz around the net discussing Akismet, which, to me, is just another hype for wordpress.com, I cam across IOerror’s discussion of his roadmap for Bad Behaivor 2. Again, people are praising Akismet as the second coming of Christ, and if that is the case, then the combination of Spam Karma and Bad Behavior is the first coming. Along with the roadmap, IOError discusses the reality of what it will take for him to be able to knock out the new version, CA$H. Quite understandable, hell, this blogger had to go back to his prior profession for the exact same reason. But the case of Bad Behavior is simple, if 100 bloggers such as this one has already done and donated a mere $10, we would in essence be hiring a full time spam fighter. So let this be a challenge to other bloggers, jump on the bandwagon and use Kismet, or support the developer who set the standard in the first place. I’m from the school of “if it ain’t broke, don’t fix it”, so therefore I’m not disabling my two working anti-spam plugins to try the flavor of the week, I’d rather see the BB2 come down the pipe a month or so from now. It’s up to you now. akismet anti spam blogging Geek life my site spam wordpress Tags [...]

    Pingback by Miklb’s Mindless Ramblings » Bad Behavior 2 Roadmap | October 27, 2005

  20. 100 * $10 would be two times what you asked for.

    Comment by VxJasonxV | October 27, 2005

  21. Heh. Nobody said you had to stop at $500. :)

    After 72 hours, $320, or 64% of $500, has come in. It looks like we just might make it!

    Oh, and here’s an update. I registered for a developer copy of ExpressionEngine, so as to see about porting Bad Behavior to it, but after three days, I haven’t heard back from pMachine. If any of you know anyone there, whack them over the head for me. :)

    Comment by Michael Hampton | October 28, 2005

  22. [...] Michael has outlined his roadmap for Bad Behavior 2. [...]

    Pingback by Blogging according to Ajay » Put Bad Behavior 2 on the Road…. | October 30, 2005

  23. Hm, since Jason asked… as of about 15 minutes ago, the tally is up to $490! That’s 95% of the “$500 goal”! Anybody got a $10 out there? :)

    The ExpressionEngine folks got back to me, and make an excellent case that I should wait for the next release, version 1.4, and port to that, as they are going to introduce a new architecture called Extensions into that release, which will permit things like Bad Behavior to integrate much more easily into their code. I can’t wait to see it. :)

    Comment by Michael Hampton | October 31, 2005

  24. Hi Michael, I plan some fund raising end November when my blog members meet. I hope they’ll scrap together some Euro for WordPress, BB and other projects we use.

    I wanted to point out that I posted a small stats graph on BB’ error 412 here:

    http://www.nazioneindiana.com/wp-content/badbehavior-sett-ott.jpg

    Quite effective, right? The whole post is in italian and deals with our monthly website stats:

    http://www.nazioneindiana.com/2005/11/02/statistiche-di-ottobre/

    Comment by jan | November 2, 2005

  25. Hi,
    I don’t use Bad Behavior (i get very little spam), but i thought i might say that i admire your work anyway, and also your honesty about the cash.
    Keep up!

    Comment by eduardo | November 6, 2005

  26. [...] About a month ago I posted a roadmap for the next major version of Bad Behavior, the PHP-based automated link spam killer. Now it’s time for an update. [...]

    Pingback by Lunacy Unleashed » Bad Behavior 2 Roadmap Update | November 29, 2005

  27. [...] This is the second of a series of updates on the roadmap to Bad Behavior 2, the next major version of the Web’s premier link spam killer. [...]

    Pingback by Lunacy Unleashed » Bad Behavior 2 Roadmap Update | December 16, 2005

  28. [...] After many delays, technical difficulties, and much more, I’m finally pleased to announce that Bad Behavior 2.0 is taking shape and I have some downloadable code for you! There are some caveats, though. [...]

    Pingback by Lunacy Unleashed » Bad Behavior 2 Alpha 1 | December 31, 2005

  29. [...] As you might expect, this is going to put a big dent in further development of Bad Behavior. Not to mention everything else I do. [...]

    Pingback by Lunacy Unleashed » It’s dead, Jim | January 3, 2006

  30. [...] I’ve suffered a hardware failure on my computer and thus further updates to this site — and to Bad Behavior will be few and far between until it’s been resolved. I’ve posted further information for those of you interested in all the technical details. [...]

    Pingback by It’s dead, Jim - Homeland Security or Homeland Stupidity | January 4, 2006

  31. [...] This is the third in a series of updates on the roadmap to Bad Behavior 2, the next major version of the Web’s premier link spam killer for PHP-based sites of all types. [...]

    Pingback by Lunacy Unleashed » Bad Behavior 2 Update | January 7, 2006

  32. [...] A representative from a major open source project informed me that the project would be willing to contribute financially to Bad Behavior, but wanted to ensure that it would get something in return, and have a better idea of the timeframe of development. Thus I’m updating the previously posted roadmap. [...]

    Pingback by Lunacy Unleashed » Bad Behavior 2 Roadmap Update | February 2, 2006


Sorry, the comment form is closed at this time.

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: