WP-Cron hogging CPU (and memory) on VPS

  • Rolf Allard van Hagen

    (@ravanh)


    9 years, 1 month ago

    Hi,

    Running a VPS with a WordPress Multisite 4.1.1, I notice sporadic bursts in very high (near 100%) CPU usage. Memory is also maxed out on PHP processes.

    These bursts seem to come without warning and last until I either set

    define('DISABLE_WP_CRON', true);

    in wp-config.php or simply ‘after a while’ or until the server comes to a grinding halt.

    Access logs show many (very many!) entries with

    "POST /wp-cron.php?doing_wp_cron=1425301287.7601780891418457031250 HTTP/1.0" 499 0

    where the response status code 499 is thrown by Nginx and stands for “client has closed connection”. In the meantime I see many “Connection reset by peer” errors in the error log. But this confuses me. How would I interpret that in the context of a wp-cron job?

    Are there any tools to find out what exactly is causing so many wp-crons running at the same time, what they are actually doing and why they are so slow?

    Thanks for any tips about debugging this 🙂

Viewing 8 replies - 1 through 8 (of 8 total)
  • Moderator Marius L. J.

    (@clorith)

    9 years, 1 month ago

    Hi,

    You can use a plugin like https://wordpress.org/plugins/wp-crontrol/ to look at what crons there are (this is per site though), it might be that some of your sites don’t get visited often and thus don’t trigger WP Cron at set intervals, you may want to have a look at https://wordpress.org/plugins/wp-cron-control/ and what they are doing for that

    Thread Starter Rolf Allard van Hagen

    (@ravanh)

    9 years, 1 month ago

    Hi Marius, thanks for the feedback 🙂

    … it might be that some of your sites don’t get visited often and thus don’t trigger WP Cron at set intervals…

    Interesting. The access log at times of CPU spikes did show cron calls from both low traffic and medium traffic sites but I’ll try to pay more attention to this possibility.

    Too bad Wp Crontrol is only per site and cannot see network wide. And only show scheduled jobs but cannot see what jobs are running real time. And I suppose it will never show any due jobs accumulating on low traffic sites because the visit to the cron admin page itself will trigger any due jobs…

    Hard to debug this cron system 🙁

    Moderator Marius L. J.

    (@clorith)

    9 years, 1 month ago

    True, debugging cron jobs isn’t a simple task, it takes some time unfortunately.

    What you’ll want to do is look at what cron jobs are scheduled, and how often they are scheduled to run. It sounds like it might be a corn that’s meant to run very often but isn’t triggered as often as it wants to.

    You could also look into running the cron job through real cron, as it sounds like you know your way around the VPS.
    Have a read at https://tommcfarlin.com/wordpress-cron-jobs/ about how to disable WP_Cron and replace it with a real cron, it might be that the rule that’s running is hitting execution timeouts or similar (which won’t happen if ran by real cron)

    Thread Starter Rolf Allard van Hagen

    (@ravanh)

    9 years, 1 month ago

    Hmmm, yes I contemplated the possibility of switching to cron to trigger wp-cron but the problem is I would have to set up cron jobs for each blog in the network and keep on adding jobs for each new blog in the future…

    Or I’ll have to write a network-aware wp-cron-triggering multisite plugin that would loop through and ping all sites in the network to fire up each wp-cron every so often. But never all at once, of course, or it will probably have the exact same rush effect that I’m looking to avoid. Shucks.

    Or does such a plugin exist already?

    Thinking about it, does the Jetpack Uptime Monitor module not use a ping? Does that trigger WP Cron?

    Moderator Marius L. J.

    (@clorith)

    9 years, 1 month ago

    I think jetpack just queries the headers, which wouldn’t trigger any behavior at all.

    The way you set up a real cron is that you make a cron job on your system hit the WP cron scheduler, you don’t need to make a new job on your server every time one of your sites adds a new task.
    They’ll still be ran “by WordPress”, your task just triggers the job from the command line at fixed intervals so you don’t miss any jobs and with more reliability to them actually finishing as they should.

    Thread Starter Rolf Allard van Hagen

    (@ravanh)

    9 years, 1 month ago

    The way you set up a real cron is that you make a cron job on your system hit the WP cron scheduler, you don’t need to make a new job on your server every time one of your sites adds a new task.

    Not for every task, no. But I’d need to set up a cron job for each site in the network and upon each new site in the future. One cron “to rule them all” for the primary site would trigger only WP_Cron on the primary site. Or am I mistaken?

    Moderator Marius L. J.

    (@clorith)

    9 years, 1 month ago

    Hi,

    That’s correct, you’d need one for each site unfortunately, unless you get creative and make the cron hit a script that fetches all the sites and then iterates over them for you.

    Thread Starter Rolf Allard van Hagen

    (@ravanh)

    9 years, 1 month ago

    I’ll have to think about such a plugin some more because (as I said) I don’t want to make something that — instead of fixing the problem — actually causes such a flood of cron jobs across the network. It should maybe itterate over all sites in batches…

Viewing 8 replies - 1 through 8 (of 8 total)
  • The topic ‘WP-Cron hogging CPU (and memory) on VPS’ is closed to new replies.