Support » Plugin: Jetpack by WordPress.com » Jetpack Sync cron jobs overloading server at WPEngine

  • Resolved adaptistration

    (@adaptistration)


    I have about 55 installs on an account at WPEngine and this afternoon, jetpack.full.sync.cron was overloading the server and causing all of the installs to throw 502/504 errors. All of the installs use JetPack so we had to deactivate it across all of them in order to get the server load back down.

    I noticed at https://wordpress.org/support/topic/jetpack-sync-killing-the-db/ this appears to be a known issue but I don’t see anything about it in any of the subsequent changelogs. Do you know if this has been addressed and if not, is there a temp fix I can put in place so as to be able to reactivate JetPack across my installs?

    Some of the higher traffic installs include:

    adaptistration.com
    morikami.org
    townhallseattle.org
    fourarts.org
    creationtech.com
    insidethearts.com

    Jetpack has been deactivated across all of those.

    Many thanks!

Viewing 15 replies - 1 through 15 (of 27 total)
  • Plugin Author Jeremy Herve

    (@jeherve)

    Jetpack Mechanic 🚀

    Sorry for all the trouble!

    The problems should indeed have been addressed in Jetpcak 4.3.2. I assume you were running the latest version of Jetpack on those sites?

    Could you tell me more about your setup? Do you use Core’s WP Cron system, or do you load your own alternate Cron system?

    Could you also try to add the following to a functionality plugin, and let me know if it helps?

    
    add_filter( 'jetpack_sync_listener_should_load', '__return_false' );
    add_filter( 'jetpack_sync_sender_should_load', '__return_false' );
    

    Thanks!

    • This reply was modified 2 years, 10 months ago by  Jeremy Herve.

    Hi Jeremy, many thanks for the reply. Yes, I’m running the core cron system. I did flip over to WPEngine’s replacement cron but it too was causing their server to run hot so we switched about about four months ago.

    I put those functions into the functions.php file in the child theme at adaptistration.com but still see both jetpack_sync_full_cron and jetpack_sync_cron listed with their default 1min recurrence rate: http://screencast.com/t/zTLvJEh0Mh

    Or did the function effectively prevent them from running even though they are still listed?

    And sorry, forgot to confirm: yes, all sites running 4.3.2.

    Plugin Author Jeremy Herve

    (@jeherve)

    Jetpack Mechanic 🚀

    Thanks for the extra details!

    WordPress itself has locks that prevent it from running more than once process at once, and prevent it (by default) from starting a new cron more than once every few minutes. I assume that’s not helping you because you have multiple high-traffic sites on the same server, all triggering Cron and thus overloading things.

    To solve this issue, I think you’ll need to play with Jetpack’s Cron interval, thanks to these 2 filters:
    jetpack_sync_incremental_sync_interval
    jetpack_sync_full_sync_interval

    As you can see, the default is 1min, but you could set it to 10min, and see if it helps.

    Let me know how it goes.

    I’m dealing with the same issues with my WP Engine servers. I came up with the following code. See any issues with it? It seems to work when using a cron viewer. I have not tested on a production server yet.

    // Jetpack custom cron schedule to reduce cron load
    add_filter( 'jetpack_sync_incremental_sync_interval', function() { return '10min'; } );
    add_filter( 'jetpack_sync_full_sync_interval', function() { return '10min'; } );
    add_filter( 'cron_schedules', 'jetpack_custom_cron_schedule' );
    
    function jetpack_custom_cron_schedule( $schedules ) {
        if( ! isset( $schedules["10min"] ) ) {
            $schedules["10min"] = array(
                'interval' => 600,
                'display' => __( 'Every 10 minutes' )
            );
        }
        return $schedules;
    }
    Plugin Author Jeremy Herve

    (@jeherve)

    Jetpack Mechanic 🚀

    Yep, that should help!

    Let me know how it goes!

    Jeremy, following the latest jetpack updates, all of these problems came roaring back and are even worse than before. Is there no way to completely disable jetpack crons?

    I’ve been round and round with this via the techs at WPE and there’s nothing more they can do either.

    Plugin Author Jeremy Herve

    (@jeherve)

    Jetpack Mechanic 🚀

    @adaptistration Sorry to hear that! Do you currently use the snippets I posted above?

    Would you mind contacting us via this contact form and mention this thread? I’d like to run some more tests with you on your site.

    Thanks!

    I’ll reply on that other thread, thanks

    Just following up, the above script has been successfully in reduce load on my WP Engine servers. I will say Jetpack did not caused my high load issues. Whenever a server get’s overcapacity any usage will destabilize the server. Jetpack is easy to blame as it’s the most prominent in the accesslogs and cron jobs. I would consider the use of the above script a temporary workaround. With WP Engine the typically permanent fix is to reduce the number of installs on a particular server.

    That’s an interesting perspective but it doesn’t take into account providers with a wide range of users; micro clients with nothing more than brochureware and a few hundred hits a day to much larger groups with thousands of hits, ecommerce, etc.

    The system resources for those smaller users are, at most, a fraction of what the larger users are but then you run into something like the cron jobs, which are blind to those distinctions. So with what you’ve proposed, the only solution is to sharply increase operating costs and have to pass that along to the end user.

    From a static perspective, that solution makes complete sense, however, from the other dynamic side of that coin, providers need solutions that don’t shove the cost of doing business down the line to the end user.

    In the end, I’m hopeful that scenarios like this help illuminate the needs for each perspective in the symbiotic relationship that are otherwise difficult to anticipate. The pros here at Jetpack have always been very helpful and, it seems to me at least, interested in this point of view.

    Hi, I’m using this simple code (and my wordpress is now consuming much less resources):

    // Jetpack custom cron schedule to reduce cron load
    add_filter( 'jetpack_sync_incremental_sync_interval', function() { return 'hourly'; } );
    add_filter( 'jetpack_sync_full_sync_interval', function() { return 'daily'; } );
    

    This means: incremental sync every hour and full sync once a day.
    For me it’s ok because I post only once or twice a day.

    (Note: I didn’t use Jetpack Publicize, so I don’t know if this code may have some side effects in that case)

    Plugin Author Jeremy Herve

    (@jeherve)

    Jetpack Mechanic 🚀

    I have good news! We’ve made some more changes to the sync process in the past few weeks, and the next Jetpack release will include a fix to the problems you’ve been having on WP Engine or when using tools like Cavalcade to manage Cron.

    Jetpack 4.5 should be available within a week or so.

    Many thanks Jeremy Herve, are you by chance aware of the solution we rolled out at my WPE installs with the assistance of Brandon from Jetpack?

    Consequently, it would be all kinds of helpful if we were able to determine if the impending changes might interfere with or knowingly make that solution moot.

    Drew

    Plugin Author Jeremy Herve

    (@jeherve)

    Jetpack Mechanic 🚀

    @adaptistration That would depend on what Brandon had you try. Did you add something like this to your theme’s functions.php file or to a functionality plugin?

    add_filter( 'pre_option_jetpack_sync_settings_sync_via_cron', '__return_zero' );

    If so, you can keep this filter, it will keep working once Jetpack 4.5 is released.

Viewing 15 replies - 1 through 15 (of 27 total)
  • The topic ‘Jetpack Sync cron jobs overloading server at WPEngine’ is closed to new replies.