comment spam: punish the victim?

Lately, I spend more time fighting comment spammers than doing productive work, specifically comment spammers that target movable type, which tends to bog down the server when attacked.

I have a lot of ideas about spam detection, some of it implemented in code. I hope to announce some open source software in the very near future.

My basic approach so far has been to:

• periodically kill all instances of mt-comments and mt-tb.cgi for the attacked site when the attack is in progress
• attempt to determine the ip address(es) of the attack, and block those addresses either at the firewall or apache level. [This isn't always as easy as it sounds, for various reasons, and is still a manual process]

And sometimes:

• temporarily turn off the victim *.cgi file (by running "chmod o-x" on it) until the spammers are blocked

With that last approach, I've always been careful to turn it back on later... But that just means that the next attack, from somewhere else will do the same thing.

Tonight for the first time, I turned off a script and left it off. I emailed the site owner, and of course I suspect it might make the site owner mad. But on the other hand, it helps keep everyone else on the server happy, and could motivate the site owner to take an interest in solving the problem.

I need feedback on this. I'm uneasy just declaring martial law. :)

What do you think? Is it good to shut down a part of one site to protect the community? What if it's YOUR site next?

Also: hydrogen seems to have run out of diskspace, which may have contributed to the database problem, and was also causing login issues with squirrelmail.

I have space alerts set up on all the other machines, but apparently never set one up for hydrogen, (it's usually farily empty). That's being monitored now, and I've deleted several gigs of junk.

The database on db.sabren.com was clogged with idle connections for much of last night and this morning, causing many of you to get "too many connection" errors.

I have a script that monitors the database server and kills off these idle connections (which pile up when people use php's ridiculous mysql_pconnect() function) every minute.

However, it, too was getting these errors, which as far as I know has never happened before. MySQL usually keeps one connection available for the root user to log in and kill off threads, and my script does log in as root, but it seems it was getting locked out anyway.

Anyway, the server is up and running again, and the script has been modified to just kill and restart mysql should that happen again.

On a related topic, if (and only if) your account is on mercury, scandium, or titanium AND you are connection to db.sabren.com, you should consider contacting me have your database moved to db2.sabren.com, which is in the same datacenter as these machines and will have a much faster response time for your queries.