Key eviction best practices

  • Resolved Jason

    (@galapogos01)


    3 weeks ago

    Hi guys,

    I run a fairly large WooCommerce site. I have configured redis with the widely recommended allkeys-lfu eviction strategy to avoid memory overflows, which has created some peculiar side effects with plugins & themes which do not correctly handle some of their transients being purged (eg. Enfold / Avia Theme Builder, Facebook for WooCommerce).

    My cache overflows regularly, seemingly caused by scrapers which crawl every category and attribute page on site for every permutation of WooCommerce filter_{attribute}using residential proxies with rolling UAs and IPs. These requests flood the cache and cause legitimate entries to be purged (even with lfu).

    My questions are —

    1. Do you have any tools/scripts for analysing cache content/usage and identifying any particular page type or query that could be blowing out the cache more than necessary?
    2. Is there any way to tell this plugin to exclude certain request types / pages (ie. don’t cache any wp_query if the $_GET contains filter_%?
    3. Any other suggestions to tune the cache (eg. volatile_lfu?), without adding infinite RAM 😅?

    Thanks,
    Jason

Viewing 8 replies - 1 through 8 (of 8 total)
  • Plugin Author Till Krüss

    (@tillkruess)

    3 weeks ago

    The easiest way would be to exclude large cache groups. But only Object Cache Pro has a tool to scan all groups and display its byte size. But I’m sure Claude can write you one as well.

    I’d really just say give Redis/Valkey more memory if possible.

    Thread Starter Jason

    (@galapogos01)

    3 weeks ago

    Thanks for the reply Till!

    Do you have any advice on volatile-lfu vs allkeys-lfu with a WordPress setup? I don’t understand how this plugin manages/sets the TTL well enough to know if this would help.

    The biggest groups appear to be post-queries and term-queries which would not be excludable as a whole.

    More memory is not really an option – it would just make the problem bigger.

    Plugin Author Till Krüss

    (@tillkruess)

    2 weeks, 6 days ago

    It’s mostly themes/plugins that dictate the TTL, but you can limit it using WP_REDIS_MAXTTL.

    You can certainly exclude *-queries groups, but adding more memory would result in better performance than firing more SQL queries.

    Either of the lfu eviction policies are good, I tend to stick with volatile-*.

    manos2000

    (@manos2000)

    1 week, 1 day ago

    I made some tests at traficky sites (not woocommerce) allkeys-lru was the best.

    Anyway the key as Till wrote is redis_maxttl, every site has a different optimal value depending on theme, traffic etc, so need to test

    • This reply was modified 1 week, 1 day ago by manos2000.
    Thread Starter Jason

    (@galapogos01)

    1 week ago

    Thanks @manos2000 . I think the issue is that volatile-lfu would only evict keys with a TTL; is there a way to set a “default TTL”?

    tbh there must be something crazy going on with cache keys being reused/rewritten as the problematic keys that are being evicted are being used, so I can’t really understand why they are being evicted with allkeys-lfu. Some metrics on key hits would probably help with diagnosis.

    Plugin Author Till Krüss

    (@tillkruess)

    1 week ago

    Yeah, volatile-lfu if you set WP_REDS_MAXTTL or allkeys-lfu if you don’t.

    manos2000

    (@manos2000)

    1 week ago

    @galapogos01

    at wp-config you put the TTL in seconds, eg for 3 hours :

    define( ‘WP_REDIS_MAXTTL’, 10800 );

    Thread Starter Jason

    (@galapogos01)

    3 days, 14 hours ago

    Thanks!

    I have written a small python utility which extracts redis session queries with MONITOR then summarises them by key & activity (read vs write).

    It appears that WordPress is repeatedly (multiple times per request) writing wp:posts results to the cache for the same nav_menu_item post ID, but never reads the cache. As my menu is over 100 items, there are over a hundred of these entries being written every during a single request. They occur even with a default theme. My current assumption is that WP core is very poorly optimised leading to cache thrashing. I will update the thread if I find anything else.

Viewing 8 replies - 1 through 8 (of 8 total)

You must be logged in to reply to this topic.