Support » Plugin: WP Super Cache » Can I force files to be pre-cached on page edit?

  • I’m trying to prepare a site to handle sporadic, unexpected, very high loads (my target is ~1000 visitors per second). When the site is under this load, we also expect to have to make frequent content changes, and want them to be visible quickly (within 30-60 seconds of posting).

    WP Super Cache helps somewhat, but the load gets ridiculously high just after a page is edited. One of the Advanced options is “Clear all cache files when a post or page is published or updated.” That setting ensures that information is visible quickly, but occasionally causes Apache to crash, as it has to spin up several hundred php-cgi processes before the cached page is rebuilt and saved to the file system.

    The process is, roughly, this:

    • The page is cached, copies are served through mod_rewrite, all is happy
    • The page is updated and saved, and the cached copy of the page is removed
    • The first request comes in, there’s no cached copy of the page, so we launch PHP to serve the request and rebuild the page
    • While that’s happening, hundreds of additional requests come in, and all have to be handled in the same way
    • Finally, one of those many requests completes, a new copy of the cached page is saved to the file system, and future requests are handled purely by mod_rewrite
    • Meanwhile, those other few hundred requests that came in while there was no cached copy of the page, are served, and the cache may be rebuilt a few hundred times (not sure if WP Super Cache catches this or not)
    • Occasionally, in the middle of all this, Apache just plain crashes, presumably hitting a process limit or other major resource-contention issue

    We have basically the same problem with, say, unchecking that setting, and having a system cron job call wp-cron.php once a minute (with the super-cache freshness set to 60 seconds). wp-cron sees that the page is stale and removes it, but doesn’t force a rebuild; again, there’s a small window where there’s no cached version of the page, before a new one gets built and saved, a LOT of requests hit PHP, and it’s basically the same scenario as above.

    The “right” solution probably is to put the whole site behind something like a Varnish cache, or possibly have another server running wget every thirty seconds against a hidden WordPress backend, but because of Reasons that probably won’t happen. With that in mind, is there a setting I’m overlooking, to force WP Super Cache to build new pages before removing the old ones, to eliminate the window where no cached copy of a page exists?

    http://wordpress.org/extend/plugins/wp-super-cache/

Viewing 1 replies (of 1 total)
  • Thread Starter David E. Smith

    (@desmith)

    Anyone? I’d really like to not have a server crash, if I can avoid it…

Viewing 1 replies (of 1 total)
  • The topic ‘Can I force files to be pre-cached on page edit?’ is closed to new replies.