request for comments: 30 second timeouts

The servers stay up, but is the cure worse than the disease?

Since I've been travelling for the holidays (not to mention sick in bed with the flu), I revived some of the older monitor rules for the script that watches over the servers.

One of the rules is the 30 second timeout. It says that any script run from cron or http will time out and be killed after 30 seconds. There is one hard coded exception: mt.cgi.

MT itself is slow but it requires a password and is only accessed by the site owner, so there's no problem letting it through.

But mt-comments.cgi and mt-trackback.cgi also suffer from horendous slowdowns as a site ages. And of course, when comment spammers attack, these slow, cpu-intensive scripts pile up and the server crashes.

For a long time I was simply renicing these "old" scripts, but that's no longer working. The reason that the servers are (mostly) once again running smoothly is that I started killing these scripts outright.

So: on the one hand, the servers run, and are safe from an mt comment spam attack.On the other: some people's sites are throwing 500 errors when readers post comments.

For the most part, this problem is specific to MT sites that have a lage number of posts and comments. The servers are VERY fast machines, and almost every script that runs for 30s+ is an MT script. The more stuff you put into MT, the slower it gets.

There are mature blogging tools such as wordpress available that do not have MT's scaling problems, so on the one hand, there's really no excuse for anyone to have to run such a slow site. It's no good for the server, and it's no good for *your* readers either.

But: a huge number of users here run movable type. I don't know how may sites have grown to the point that their comments run this slow, but I know it happens, eventually, to just about everyone who runs MT.

I don't know what to do here. Right now, I know I can't be watching the servers as closely as I normally would, and so I'm leaving the 30 second rule in place, even though its causing problems for some people. I'd rather lose a few comments than deal with a crash.

I'd like to come up with a better, permanent policy on this next week, but I don't know what the answer is.

What do you all think?