WordPress.org

Forums

Memcached Object Cache
[closed] switch_to_blog() causing memory leak (10 posts)

  1. Joe Hoyle
    Human Made
    Posted 2 years ago #

    I am seeing memory leaks with switch_to_blog(), the issue seems to be this:

    1. wp_cache_init() is called, GLOBALS['wp_object_cache'] in 2MB in size, total memory usage is 22MB.
    2. Switch to blog is called
    3. wp_cache_init() is called (as wp_cache_switch_to_blog() is not implemented), $GLOBALS['wp_object_cache'] in 0.5MB in size, total memory is 22.5MB

    At this point, total memory should be 21.5MB, however the first WP_Object_cache object is not being garbage collected when the reference in $GLOBALS['wp_object_cache'] is set to the new instance of WP_Object_Cache.

    Doing this:

    for ( $i = 1; $i < 100; $i++ ) { switch_to_blog( 15 ); restore_current_blog(); }

    raises memory usage to circa 120MB.

    I have tried using xdebug_debug_zval to count references to the first instance of WP_Object_Cache, however that is appearing to be only 1 - so I am not sure why this is not being garbage collected.

    If anyone can replicate this, any advice would be much appreciated!

    http://wordpress.org/extend/plugins/memcached/

  2. Mike Little
    Member
    Posted 2 years ago #

    I'm seeing my requests running out of memory (even with a ridiculous 512M allocated) on a system that does a lot of blog switching.

    Seems to have started with WP 3.5, I'm now seeing it with 2.0.1 of this plugin too. Whereas the plugin was definitely ok earlier.

    Right now I have to remove the plugin, which is killing performance.

  3. Joe Hoyle
    Human Made
    Posted 2 years ago #

    I have not managed to fix the references issue, however when I implemented wp_cache_switch_to_blog things somewhat improve. Doing this means the object cache size (in php memory) will increase whenever switching to a _new_ blog, but restoring to older blogs has no overhead. So this happens:

    1. wp_cache_init on blog 1 - memory used: 2mb
    2. switch_to_blog( 2 ) - has an object cache of 1.5m - memory used: 3.5mb
    3. restore current blog - memory used 3.5 (as blog 1 was already in the object cache)

    If switching between a small amount of blogs this is acceptable (and better than this bug), however memory will still increase with the more blogs you switch to this way.

    My implementation of wp_cache_switch_to_blog:

    function wp_cache_switch_to_blog() {
    
    	global $blog_id, $table_prefix, $wp_object_cache;
    	$wp_object_cache->blog_prefix = ( is_multisite() ? $blog_id : $table_prefix ) . ':';
    }
  4. Joe Hoyle
    Human Made
    Posted 2 years ago #

    Mike: which version of PHP are you running? I am using 5.4, haven't got round to debugging this under 5.3 yet.

  5. Mike Little
    Member
    Posted 2 years ago #

    I'm running 5.3.2.

    Unfortunately, the site in question does a lot of switch_to_blog, so I don't know if your solution would work.

    And currently the site is having an 'event', which means 25 - 50K visits per day over a 8 hour period, much of that live interaction, so I can't do anything that might destabilize the site!

    I'll see whether I can reproduce the issue on my test rig, and then whether your solution would fix things.

  6. Mike Little
    Member
    Posted 2 years ago #

    Is this a bug in switch_to_blog()? I know it was changed recently... Or has the change triggered a bug in the plugin?

  7. Joe Hoyle
    Human Made
    Posted 2 years ago #

    Ok, I have a fix!

    Not sure what triggered this issue, I know switch_to_blog() did change, but looking at what changed I don't see why this would have broken.

    Anyhow, working on my above function, the key is to blow away the in-memory cache everytime switch to blog is called, this lets the memory footprint remain low no matter how many times you switch to different blogs:

    if ( ! function_exists( 'wp_cache_switch_to_blog' ) ) :
    function wp_cache_switch_to_blog() {
    
    	global $blog_id, $table_prefix, $wp_object_cache;
    	$wp_object_cache->cache = array();
    	$wp_object_cache->blog_prefix = ( is_multisite() ? $blog_id : $table_prefix ) . ':';
    }
    endif;

    To test this, I have a helper function to eat up memory:

    function use_memory() {
    	for( $i = 0; $i < 100; $i ++ ) {
    		wp_cache_add( $i, str_pad( 'a', 1014 * 1014, 'a') );
    	}
    }

    So to test (in init hook):

    use_memory();
    echo memory_get_usage() / 1024 / 1024; // should be about 120MB
    switch_to_blog( 2 );
    echo memory_get_usage() / 1024 / 1024; // should be about 25MB
    restore_current_blog();
    echo memory_get_usage() / 1024 / 1024; // should be about 120MB

    You need to flush memcached every time running the test, so wp_cache_add() works

    I am running the above function in production which fixed memory issues I was having, can now switch_blog_blog() 20+ times without issue

  8. Andrew Nacin
    Lead Developer
    Plugin Author

    Posted 2 years ago #

    This was a deliberate change in core.

    wp_cache_reset() used to be used during the switch_to_blog(), which would obliterate the existing object cache. So if you switched from site A, to site B, back to site A, all existing local cache for site A would be gone, and you'd have to re-query everything (notably, options). It was really silly and actually made switch_to_blog() more expensive, especially if you were doing something like get_blog_option() repeatedly, which now does a switch() inside it.

    The caching layer was changed in 3.5 to also key storage by site. So if you switched from A to B to C all the way to Z, we'd store anything in local cache for when you switched back.

    This isn't a "memory leak" as much as an overall performance benefit (most of the time).

    Of course, there are situations where a way to "free" memory would be pretty useful. We deprecated wp_cache_reset(), but could bring back its functionality for when the situation requires it.

  9. Ryan Boren
    WordPress Dev
    Plugin Contributor

    Posted 2 years ago #

    Background: http://core.trac.wordpress.org/ticket/21434

    I wouldn't mind bringing back wp_cache_reset() with lots of phpdoc explaining how and when to use it in a plugin. I think the current behavior should remain the default, however, since it saves lots of queries.

  10. Joe Hoyle
    Human Made
    Posted 2 years ago #

    nacin: Yeah, I can see how blowing away the object cache for the not-current-site would be a bad idea, if you don't have persistant object caching. But doesn't this change mean it's pretty much not possible to switch_to_blog() more than say, 20 times:

    Presuming I have 100 blogs on an ms install, all with ~3mb object caches, switching between them in a loop is going to load 300MB of data into that process's memory - if I am using Memcached, I would be better off throwing it away from the current process on switch_to_blog and pull it back from Memcached when a blog is next switched to.

    That's presuming the overhead of pulling everything from Memcached is essentially "free".

Topic Closed

This topic has been closed to new replies.

About this Plugin

  • Memcached Object Cache
  • Frequently Asked Questions
  • Support Threads
  • Reviews

About this Topic