WP-Cache strangling WordPress 2.1.3 on Dreamhost?

Okay, first up: I apologize in advance if I have missed something obvious and this is stupid. I have a degree in English, so about all I’m good for is spotting typos at 50 paces. I have searched the forums and I don’t know how many blog posts but haven’t found a solution, so here we go. Get comfortable. I’ll give you all the info I have:

Here’s me: http://www.needcoffee.com

I’m running 2.1.3 of WP on a Dreamhost shared server account. I started running WP-Cache a while back (before 2.1.3) when DH suggested I needed to cache the site because I was smacking the server around more than I should. I get about 4000 pageviews a day for the WP portion of my site (the rest of my site is still in its pre-WP static HTML form, because I haven’t gotten around to converting all of it yet).

In the past few months, I’ve started getting 500 Internal Server Errors. I wish I could say what the line of demarcation was between everything working/everything not working was exactly, but all of that got clouded by the fact that I was attacked by so many comment spammers back in December that even with Bad Behavior, Spam Karma 2, and Akismet in place, the sheer number of them getting through and hammering at my wp-comments file was enough to shut me down. They weren’t able to post a comment at all, but their attempts were so often that it was killing me. I turned off comments and put a RedirectMatch gone for the comments files in my .htaccess and all was well.

But after a while, I thought the coast was clear–I renamed wp-comments and kept changing it every so often and so I was able to turn on comments again. Yet, they finally showed back up again, or at least enough of them that Dreamhost said they were back and I closed comments down.

But the 500s persisted. Especially when I would be saving/editing or publishing a post. That would 500 out and I’d lose the post completely. I kept hitting the PHP memory ceiling–a story I’ve seen repeated a lot. I didn’t think I should be hitting the ceiling–something didn’t add up. So I went through optimizing my theme. I pulled out all the plugins I felt I could live without, stripped some PHP calls from my theme, made sure I had the latest versions of everything (I think this is the point I upgraded from 2.1 or 2.1.1 to 2.1.3–I never downloaded 2.1.2). I even stripped out every single plugin one by one to see if any of them were causing the pain. Nothing. I moved all of my sites to other accounts on the server so that I would have only this one site with a PHP ceiling all to itself. Nothing. I tried going from PHP 4.4.2 to 5.2.1. Nothing. I had tried putting gzip compression in via my .htaccess and stripped that back out again. Nothing. I went through my access logs and cleaned up, banning some IP addresses that were doing weird things and using robots.txt to get the bots to calm down, since they were running about like mad weasels. Nothing. Optimized my database. Nada.

Those steps may not be in the proper order of how I did them. I’ve been trying all kinds of things, so forgive my memory, it’s sieve-like.

Okay, so here’s where it gets strange. I started playing with WP-Cache. I realized that while it was creating the cache, it wasn’t actually USING the cached pages. Nice, huh? All the overhead of storing the cache plus the overhead of writing the site anyway. So…that seemed like the solution. I stripped out WP-Cache, reinstalled it and everything was fine.

Well, no. Shortly thereafter, 500, 500. What the #@(@ now?

So more WP-Cache tinkering. I thought, well, to hell with it, whatever low amount I had it set on wasn’t high enough. So I kept raising it higher and higher. I saw some guy on his blog just keeps it at 999999999 or something, but I didn’t go that far. Still, it didn’t help. It would work fine for a short while, sometimes an hour, then 500, 500, 500.

After a while, I thought: hmmm. What if WP-Cache is too big? Now that doesn’t make a lot of sense, but it was probably 4am when I thought this and I’d been tearing my hair out. So I lowered it to something like 30K or 40K seconds.

Then everything worked fine…again, for an hour. Then 500 again. I tried lowering it further, because even then I was getting 4000 cached pages. It would work fine after the change, then, blam. Back to being broken after a short time. GAH.

So finally I turned off WP-Cache completely. Turned around and tried to do exactly what I had just done, the same thing that had caused the 500. It worked. What? No, that can’t be it. So I turned WP-Cache back on, but with some small cache amount, like 5000 seconds. It kept working. Great. But…eventually, it quit after about an hour or so. 500.

So basically I’m chasing WP-Cache around. I think the problem’s with WP-Cache because any time I fiddle with it, it works for a short amount of time, then the site closes up shop again within about an hour. But I couldn’t tell you for the life of me what’s causing it.

Now, and here’s the thing. Maybe I’ve outgrown my shared hosting. I’m fully prepared to accept that. I’m eyeing the VPS offering they have at hosting.com. But what I don’t want to do is pull up stakes and upgrade to something like that, only to find out that lack of resources isn’t what was doing me in, but something buggy that I could have fixed–and I have the same issues but now I’m paying more to have them.

WordPress is 2.1.3, WP-Cache is 2.1.1, PHP is (currently) 4.4.2, other plugins in the mix are Ad Rotator 1.0, Bad Behavior 2.0.10, Category Visibility-RH Rev 1.1.b9, Customizable Post Listings 1.1, Feedburner Feed Replacement 2.2, Full Text Feed 1.03, Google Sitemaps 3.0b6, Podpress 7.7, Post Templates by Category R1.3, UTW 3.1415…, and Unfancy Quote 2.0.

Any ideas? Thanks in advance.