WordPress.org

Forums

wp-cron causing excessive processes (24 posts)

  1. mrarrow
    Member
    Posted 4 years ago #

    I run my own VPS (Apache 2.0.63, PHP 5, MySQL 5.091) and have quite a few multisite installs for various clients.

    One of these installs however regularly (twice a week?) generates the following warnings that get sent to me. At this point the server grinds to a halt for a couple of minutes until the processes clear.

    I've read in quite a few places that wp-cron.php can actually cause this problem but wondered whether anyone had a solution for it? The customer is using The Morning After theme which in itself doesn't seem to be doing anything unusual. Any ideas or thoughts?

    Time:          Mon Nov 22 19:08:31 2010 +0000
    Account:       <nameofuser>
    Process Count: 11 (Not killed)
    
    Process Information:
    
    User:<nameofuser> PID:1520 Run Time:57(secs) Memory:170088(kb) exe:/usr/bin/php cmd:/usr/bin/php /home/<nameofuser>/public_html/wp-cron.php
    User:<nameofuser> PID:1708 Run Time:50(secs) Memory:171648(kb) exe:/usr/bin/php cmd:/usr/bin/php /home/<nameofuser>/public_html/index.php
    User:<nameofuser> PID:3435 Run Time:37(secs) Memory:169832(kb) exe:/usr/bin/php cmd:/usr/bin/php /home/<nameofuser>/public_html/index.php
    User:<nameofuser> PID:3476 Run Time:36(secs) Memory:160624(kb) exe:/usr/bin/php cmd:/usr/bin/php /home/<nameofuser>/public_html/index.php
    User:<nameofuser> PID:3547 Run Time:33(secs) Memory:160880(kb) exe:/usr/bin/php cmd:/usr/bin/php /home/<nameofuser>/public_html/index.php
    User:<nameofuser> PID:20399 Run Time:17(secs) Memory:147880(kb) exe:/usr/bin/php cmd:/usr/bin/php /home/<nameofuser>/public_html/index.php
    User:<nameofuser> PID:20462 Run Time:15(secs) Memory:160344(kb) exe:/usr/bin/php cmd:/usr/bin/php /home/<nameofuser>/public_html/wp-cron.php
    User:<nameofuser> PID:24550 Run Time:0(secs) Memory:147880(kb) exe:/usr/bin/php cmd:/usr/bin/php /home/<nameofuser>/public_html/wp-cron.php
    User:<nameofuser> PID:32667 Run Time:65(secs) Memory:169836(kb) exe:/usr/bin/php cmd:/usr/bin/php /home/<nameofuser>/public_html/index.php
    User:<nameofuser> PID:32694 Run Time:63(secs) Memory:172200(kb) exe:/usr/bin/php cmd:/usr/bin/php /home/<nameofuser>/public_html/index.php
    User:<nameofuser> PID:32720 Run Time:63(secs) Memory:169832(kb) exe:/usr/bin/php cmd:/usr/bin/php /home/<nameofuser>/public_html/index.php
  2. mrarrow
    Member
    Posted 4 years ago #

    Anyone else experiencing this. Its happening on several of our accounts?

  3. I *think* this is fixed in trac.

  4. mrarrow
    Member
    Posted 4 years ago #

    OK - well I'm just upgrading each one to 3.0.2 via Subversion, so I'll see if that fixes it.

    Thanks

  5. Yeah, cron was firing for each blog in the network, instead of, you know, just for the install. :D

  6. mrarrow
    Member
    Posted 4 years ago #

    OK, well I've now upgraded all installs to WP 3.0.3 and I'm still getting these errors e.g.

    Time:          Sun Dec 12 02:23:19 2010 +0000
    Account:       <nameofuser>
    Process Count: 11 (Not killed)
    
    Process Information:
    
    User:<nameofuser> PID:9259 Run Time:216(secs) Memory:172296(kb) exe:/usr/bin/php cmd:/usr/bin/php /home/<nameofuser>/public_html/index.php
    User:<nameofuser> PID:9293 Run Time:213(secs) Memory:172288(kb) exe:/usr/bin/php cmd:/usr/bin/php /home/<nameofuser>/public_html/index.php
    User:<nameofuser> PID:9701 Run Time:206(secs) Memory:150268(kb) exe:/usr/bin/php cmd:/usr/bin/php /home/<nameofuser>/public_html/index.php
    User:<nameofuser> PID:17796 Run Time:67(secs) Memory:13876(kb) exe:/opt/suphp/sbin/suphp cmd:/opt/suphp/sbin/suphp
    User:<nameofuser> PID:17933 Run Time:55(secs) Memory:13876(kb) exe:/opt/suphp/sbin/suphp cmd:/opt/suphp/sbin/suphp
    User:<nameofuser> PID:19521 Run Time:51(secs) Memory:13876(kb) exe:/opt/suphp/sbin/suphp cmd:/opt/suphp/sbin/suphp
    User:<nameofuser> PID:19523 Run Time:51(secs) Memory:13876(kb) exe:/opt/suphp/sbin/suphp cmd:/opt/suphp/sbin/suphp
    User:<nameofuser> PID:19534 Run Time:50(secs) Memory:13876(kb) exe:/opt/suphp/sbin/suphp cmd:/opt/suphp/sbin/suphp
    User:<nameofuser> PID:19536 Run Time:50(secs) Memory:13876(kb) exe:/opt/suphp/sbin/suphp cmd:/opt/suphp/sbin/suphp
    User:<nameofuser> PID:19903 Run Time<nameofuser>:22(secs) Memory:13876(kb) exe:/opt/suphp/sbin/suphp cmd:/opt/suphp/sbin/suphp
    User:<nameofuser> PID:26372 Run Time:355(secs) Memory:171188(kb) exe:/usr/bin/php cmd:/usr/bin/php /home/<nameofuser>/public_html/index.php

    The files being reported can vary, but they're generally the index.php or cron.php with some su php mixed in for good measure.

    When several accounts are reporting these errors at the same time it then increases our server load immensely (and I mean 35/40-fold) which grinds everything to a halt. This normally happens during the night which I guess does indicate that its some kind of cron process running, as usage stats show that all of our sites generally don't get much external traffic during night hours.

    However, one time it happened during the day on an install that I was working on at the time (just simple CSS stuff). I found the server load just increasing exponentially for no apparent reason. When it was getting beyond a joke, I quit out of all my browser sessions (only a couple of tabs open for that site) and that seemed to immediately have an effect i.e. within a very short space of time, the server load had dropped back down to normal levels.

    Coincidence - I dunno? Something's odd and it also seems to be also happening on accounts that aren't actually live yet i.e. they're not having users generating content/logging in etc.

    Any ideas or thoughts?

  7. Well, the cron fix may not have been rolled in to recent releases. It may be for 3.1. (the recent two small releases were unplanned)

    though if you had quite a few tabs open all connected to your site.... yeah. That may do it if it's firing up new processes on the server & not killing them off.

  8. mrarrow
    Member
    Posted 4 years ago #

    What if a user doesn't actually log out of the WP backend, but simply shuts their laptop or switches off their PC? It _appears_ that the session still remains active at least as far as WP is concerned. Which in turn creates unnecessary loading on the server, especially because that session is no longer required.

    The reason I ask is that often this happens at night and on accounts where development work has been happening only during the day. This isn't just me, but also other clients on other installs on our server.

  9. No, it's your server configuration. If someone is logged in and does nothing, it should die off after a set amount of time. This is done by your server - not wordpress.

    Tho I do wanna know what kind of plugins you're running - especially ones that work when the index page is hit.

    yes, something is keeping the session alive, and yes, I have seen spikes like this. 90% of the time they were plugins. (the other was all related to pulling feeds back and forth across the site, or putting the user's own feed in the rss widget).

  10. mrarrow
    Member
    Posted 4 years ago #

    Well it seems to be happening to the installs that have had recent activity on them in terms of content being generated or edited (but as I say, not necessarily at that time - coz I'm getting spikes during the night too).

    By default I set them up as multisite (even though I may not immediately need the multisite capability - I come at it from WPMU!). I don't use multidomain installs now - I set each one up in separate accounts on the server.

    There's no common themes being used and the installs have varying amounts of traffic and user-generated activity.

    In general the only common plugins I use across all these sites are:

    Unfiltered MU 1.3.1
    Defensio 2.5.9

    I'm not sure if this helps!

  11. So you have multiple multisite installs? And not just one?

    What are the server specs (ram)?

  12. We're experiencing what might be a similar issue, and I've been looking into Cron much more recently as the culprit.

    One network has 191 sites, and 1 of them makes a full 50% of the calls to wp-cron.php, even though it has nowhere near that %age of the traffic.

    Even weirder.. that guy actually deactivated his site 2 weeks ago, and we're STILL getting 50% of our calls to wp-cron coming via the /deactivate-site/wp-cron.php?doing_wp_cron

    I just put in a ticket about deactivated sites returning 500 instead of 404 (or something else), which might be somewhat related, but yea, I also think that wp-cron might be taking down our server once or twice a week.

    Any ideas how to confirm this is the situation? (other than just disabling wp-cron)

    Does anyone have suggestions or experience with disabling the default wp-cron and using a custom one?

  13. mrarrow
    Member
    Posted 4 years ago #

    Its a VPS with:
    1024Mb RAM
    76Gb hard drive

    Viewing Top I generally run at about 0.25 to 1 most of the time. My server provider has said that anything up to about 3-6 is OK as long as its not constant. So I'm obviously well within boundaries at the moment. And I normally have at least 600mb of RAM free at any one time (according to Top).

    I get excessive resource usage messages everyday from at least one or two of the accounts, but there's no appreciable reduction in speed or response of any of the installs or server at this point.

    I then get load average alert warnings probably every 5 - 10 days (which can briefly reach 25 - 30 at its peak, before it settles down again). During this time, everything on the server grinds to a halt.

    Reading this through now, maybe the excessive resource usage isn't connected with the server load warnings. It certainly seems odd...

    Any ideas?

  14. Andrew Nacin
    Lead Developer
    Posted 4 years ago #

    In multisite, cron gets fired on each individual blog. Executing wp-cron.php via sys cron doesn't help much as it can't select which blog to run cron on.

    I'm not sure what your bug is here, but I would tend to guess it is the server configuration. You may want to set up a jobs system if you are scaling at high levels.

  15. mrarrow
    Member
    Posted 4 years ago #

    I think I may have got to the bottom of this (took a while!). First off, I noticed my global php.ini file had the following settings:

    CORE: memory_limit = 256Mb
    OPTIONS & INFORMATION: max_execution_time = 1000sec

    So I changed those back to 96Mb and 60secs respectively (I believe they are the default options), which immediately reduced the number of accounts reporting the error messages.

    However, the bigger traffic sites were still causing issues.

    So after a lot of investigation, I found out that wp-cron.php is actually run on EVERY page load (when logging into the backend or when a user views any frontend page).

    I was really surprised by this (and it does seem very inefficient), so I disabled wp-cron.php from firing in this way by adding the following line to my wp-config.php file:
    define('DISABLE_WP_CRON', true);

    and then added a cronjob in cPanel to run this file for each site (and any sub domains) at a predetermined time (say every 4hrs or so)

    wget http://domainname/wp-cron.php
    wget http://subdomain.domainname/wp-cron.php

    Doing this for each site that was reporting the errors seems to have fixed it and I've had nice steady loads every since - although you know what's gonna happen now I've said that...:-)

  16. mborella
    Member
    Posted 4 years ago #

    I've run into a similar issue with wp-cron.php processes bringing a server to its knees. it happened a few hours after I upgraded to 3.1. Anyone else seeing this?

  17. Could have been a plugin though.

  18. mrarrow
    Member
    Posted 4 years ago #

    Now that a suitable amount of time has elapsed, I can confirm that the following sorted my particular issues:

    Resetting my global PHP.ini file to following:
    CORE: memory_limit = 96Mb
    OPTIONS & INFORMATION: max_execution_time = 60secs

    For sites with any significant traffic, I also disabled wp-cron in the wp-config.php file and set up cron jobs in cPanel (for each site and subdomain) to run at regular intervals, generally 4-6hrs. All as detailed above.

    This then fixed the issues almost immediately.

    Further to that, I recently noticed that 1Gb of RAM had "dropped off" my VPS (should've had 2Gb. but it was only operating with 1Gb!!!!). So I politely requested my hosting provider find the extra RAM and reapply it...

    This actually may have been the catalyst for the problems in the first place, but clearly the first two actions fixed the server load before I noticed the missing RAM and WP leaner and meaner the first place.

    I'm currently running about 15-20 separate WP installs on the same VPS, most with multiple subdomains.

  19. Dang I wish we could like forum posts. ;)

  20. Ditto.

    Have you checked out 3.1 yet? IIRC is has some cron helpage (ONLY the main site kicks off certain jobs, iirc).

  21. mrarrow
    Member
    Posted 4 years ago #

    No I've not gone to WP 3.1 yet, as the sites that were causing me grief are all using TDO Mini Forms and from what I understand, that plugin no longer works with 3.1. Grrrr....

    However, on the other sites, I'm maintaining the installs via Subversion, so upgrading those shouldn't be a problem, right?

  22. DrLightman
    Member
    Posted 4 years ago #

    Thanks mrarrow for your cron's adventures feedback :)

    I'm trying myself to shed some light upon missing scheduled posts in a network of 100+ blogs of a WP3 multisite installation, WP-super-cache enabled.

    I've pretty much followed the instructions listed here to crontab a custom wp-cron-all.php script, but authors of some blogs report that their scheduled posts doesn't get published, dashboard reports "missing schedule".

    wp-cron-all.php is crontabbed to run every 15 minutes.

    This file contains among other things a loop made this way:

    foreach ($blogs as $blog) {
    	$cronStartTime = microtime(true);
    
    	echo date('Y/m/d H:i:s', $cronStartTime) . " Starting wp-cron for $blog->domain"."$blog->path...";
    	$cron_url = $blog->domain . $blog->path . 'wp-cron.php?doing_wp_cron';
    	wp_remote_post( $cron_url, array('timeout' => 0.01, 'blocking' => false, 'sslverify' => false) );
    
    	$cronEndTime = microtime(true);
    	$duration = $cronEndTime - $cronStartTime;
    	echo " (completed in ". $duration ." seconds)<br />\r\n";
    }

    The log.txt that is output indicates that every call to blogXXX/wp-cron.php?doing_wp_cron is executed correctly. (I don't know what "doing_wp_cron" is for).

    But if I run a-blog/wp.cron.php manually from a browser, it loads and loads and loads, never ends, this makes me think.

    Trying to re-enable default WP cron, removing the constant from wp-config.php, caused the server to freeze, and needed to reboot.

  23. Mariano Blaya
    Member
    Posted 3 years ago #

    I also have had some issues with many wp-cron processes running and causing CPU throttling.
    As I knew which plugin was running, rather than disabling wp-cron on wp-config.php (I am not brave enough .), I came up with a few lines of code using API_transient that make the trick.

    $prefix = .... // something to help you searching your stuff on wp_options table
    $name = ..... // whatever identifies your function
    $mins = .... // maximum frequency
    $trname = substr( $prefix . sprintf( "%u", crc32($name)), 0, 45 );
    if( get_transient($trname) )
    return "-- not yet";
    set_transient( $trname, date('c'), $mins*60 );
    // your program ....

  24. Harish Chouhan
    Member
    Posted 3 years ago #

    Hello,

    I have a server with lot of WordPress websites. I often get such notification emails.

    Few things I have experimented are:

    1. Sometimes this is caused due to plugins. So make sure to disable all plugins and then enable each one every day.
    2. If the site has lot of posts and some buggy sitemap generator plugin or some backup plugin, then this can also cause the issue.
    3. I disabled WP-Cron from the functions.php file. I then setup a CRON using the hosting's control panel (in my case Cpanel) and setup a cron job every minute. (+1 to @mrarrow)

    For WordPress, always make sure to have a cache plugin. It reduced lot of load. Try reducing the number of queries in the themes too. It has a big impact in overall speed.

Topic Closed

This topic has been closed to new replies.

About this Topic