[Plugin: WP Super Cache] scaling issues

I’d like to start by saying that I’m a long-time wp-cache user/fan, and I’ve been amazed by how awesome wp-super-cache is! Thanks donncha for your rocking supercache code! 🙂

I think I’m running into a scaling issue with the wp-cache part of wp-super-cache, and I’m wondering if anybody else is seeing the same issue? Our WordPress blog’s wp-cache does a lot of caching. The issue is that /wp-content/cache/ has loads of files in it. For instance, tonight at 9:30 PM, there are 1,228 wp-cache files in /wp-content/cache/. During peak, this number is much higher.

I think this causes Apache to slow down, because Apache needs to scan the folder to see if there’s an existing wp-cache file to serve to a given cookied user (and then serve that cached file, or generate a new one). This means that each pageview requires a scan of a directory with at least a thousand files.

This is just a hypothesis, but we’re running another site on a similar server, and it’s *so* much faster. I think the big difference is this directory-scan issue. (The new site has many many fewer wp-cache files.)

I wanted to post here for two reasons:

First, is anybody else seeing this sort of issue on a high-traffic blog?

Second, are there any workarounds to this issue? For instance, would it be possible to put wp-cache cache files into subdirectories, much as supercache does? For instance, the supercached cache file for a random blog page is located in this nested subdirectory: /wp-content/cache/supercache/www.sitename.com/2008/12/08/slug-slug/index.html. Using subdirectories would seem to eliminate the directory-scan issue. Would taking the supercache-nested-folder approach be relevant to solving this wpcache-directory-scan issue?

Thanks!

-Bob