WordPress.org

Ready to get started?Download WordPress

Forums

Memcached Object Cache
[closed] switch_to_blog() causing memory leak (10 posts)

  1. Joe Hoyle
    Human Made
    Posted 1 year ago #

    I am seeing memory leaks with switch_to_blog(), the issue seems to be this:

    1. wp_cache_init() is called, GLOBALS['wp_object_cache'] in 2MB in size, total memory usage is 22MB.
    2. Switch to blog is called
    3. wp_cache_init() is called (as wp_cache_switch_to_blog() is not implemented), $GLOBALS['wp_object_cache'] in 0.5MB in size, total memory is 22.5MB

    At this point, total memory should be 21.5MB, however the first WP_Object_cache object is not being garbage collected when the reference in $GLOBALS['wp_object_cache'] is set to the new instance of WP_Object_Cache.

    Doing this:

    for ( $i = 1; $i < 100; $i++ ) { switch_to_blog( 15 ); restore_current_blog(); }

    raises memory usage to circa 120MB.

    I have tried using xdebug_debug_zval to count references to the first instance of WP_Object_Cache, however that is appearing to be only 1 - so I am not sure why this is not being garbage collected.

    If anyone can replicate this, any advice would be much appreciated!

    http://wordpress.org/extend/plugins/memcached/

  2. Mike Little
    Member
    Posted 1 year ago #

    I'm seeing my requests running out of memory (even with a ridiculous 512M allocated) on a system that does a lot of blog switching.

    Seems to have started with WP 3.5, I'm now seeing it with 2.0.1 of this plugin too. Whereas the plugin was definitely ok earlier.

    Right now I have to remove the plugin, which is killing performance.

  3. Joe Hoyle
    Human Made
    Posted 1 year ago #

    I have not managed to fix the references issue, however when I implemented wp_cache_switch_to_blog things somewhat improve. Doing this means the object cache size (in php memory) will increase whenever switching to a _new_ blog, but restoring to older blogs has no overhead. So this happens:

    1. wp_cache_init on blog 1 - memory used: 2mb
    2. switch_to_blog( 2 ) - has an object cache of 1.5m - memory used: 3.5mb
    3. restore current blog - memory used 3.5 (as blog 1 was already in the object cache)

    If switching between a small amount of blogs this is acceptable (and better than this bug), however memory will still increase with the more blogs you switch to this way.

    My implementation of wp_cache_switch_to_blog:

    function wp_cache_switch_to_blog() {
    
    	global $blog_id, $table_prefix, $wp_object_cache;
    	$wp_object_cache->blog_prefix = ( is_multisite() ? $blog_id : $table_prefix ) . ':';
    }
  4. Joe Hoyle
    Human Made
    Posted 1 year ago #

    Mike: which version of PHP are you running? I am using 5.4, haven't got round to debugging this under 5.3 yet.

  5. Mike Little
    Member
    Posted 1 year ago #

    I'm running 5.3.2.

    Unfortunately, the site in question does a lot of switch_to_blog, so I don't know if your solution would work.

    And currently the site is having an 'event', which means 25 - 50K visits per day over a 8 hour period, much of that live interaction, so I can't do anything that might destabilize the site!

    I'll see whether I can reproduce the issue on my test rig, and then whether your solution would fix things.

  6. Mike Little
    Member
    Posted 1 year ago #

    Is this a bug in switch_to_blog()? I know it was changed recently... Or has the change triggered a bug in the plugin?

  7. Joe Hoyle
    Human Made
    Posted 1 year ago #

    Ok, I have a fix!

    Not sure what triggered this issue, I know switch_to_blog() did change, but looking at what changed I don't see why this would have broken.

    Anyhow, working on my above function, the key is to blow away the in-memory cache everytime switch to blog is called, this lets the memory footprint remain low no matter how many times you switch to different blogs:

    if ( ! function_exists( 'wp_cache_switch_to_blog' ) ) :
    function wp_cache_switch_to_blog() {
    
    	global $blog_id, $table_prefix, $wp_object_cache;
    	$wp_object_cache->cache = array();
    	$wp_object_cache->blog_prefix = ( is_multisite() ? $blog_id : $table_prefix ) . ':';
    }
    endif;

    To test this, I have a helper function to eat up memory:

    function use_memory() {
    	for( $i = 0; $i < 100; $i ++ ) {
    		wp_cache_add( $i, str_pad( 'a', 1014 * 1014, 'a') );
    	}
    }

    So to test (in init hook):

    use_memory();
    echo memory_get_usage() / 1024 / 1024; // should be about 120MB
    switch_to_blog( 2 );
    echo memory_get_usage() / 1024 / 1024; // should be about 25MB
    restore_current_blog();
    echo memory_get_usage() / 1024 / 1024; // should be about 120MB

    You need to flush memcached every time running the test, so wp_cache_add() works

    I am running the above function in production which fixed memory issues I was having, can now switch_blog_blog() 20+ times without issue

  8. Andrew Nacin
    Lead Developer
    Plugin Author

    Posted 1 year ago #

    This was a deliberate change in core.

    wp_cache_reset() used to be used during the switch_to_blog(), which would obliterate the existing object cache. So if you switched from site A, to site B, back to site A, all existing local cache for site A would be gone, and you'd have to re-query everything (notably, options). It was really silly and actually made switch_to_blog() more expensive, especially if you were doing something like get_blog_option() repeatedly, which now does a switch() inside it.

    The caching layer was changed in 3.5 to also key storage by site. So if you switched from A to B to C all the way to Z, we'd store anything in local cache for when you switched back.

    This isn't a "memory leak" as much as an overall performance benefit (most of the time).

    Of course, there are situations where a way to "free" memory would be pretty useful. We deprecated wp_cache_reset(), but could bring back its functionality for when the situation requires it.

  9. Ryan Boren
    WordPress Dev
    Plugin Contributor

    Posted 1 year ago #

    Background: http://core.trac.wordpress.org/ticket/21434

    I wouldn't mind bringing back wp_cache_reset() with lots of phpdoc explaining how and when to use it in a plugin. I think the current behavior should remain the default, however, since it saves lots of queries.

  10. Joe Hoyle
    Human Made
    Posted 1 year ago #

    nacin: Yeah, I can see how blowing away the object cache for the not-current-site would be a bad idea, if you don't have persistant object caching. But doesn't this change mean it's pretty much not possible to switch_to_blog() more than say, 20 times:

    Presuming I have 100 blogs on an ms install, all with ~3mb object caches, switching between them in a loop is going to load 300MB of data into that process's memory - if I am using Memcached, I would be better off throwing it away from the current process on switch_to_blog and pull it back from Memcached when a blog is next switched to.

    That's presuming the overhead of pulling everything from Memcached is essentially "free".

Topic Closed

This topic has been closed to new replies.

About this Plugin

About this Topic