Support » Plugin: The SEO Framework » Strange increase in description transients

  • Resolved fpmsummer

    (@fpmsummer)


    Hello, I have questions 🙂

    On a couple of sites with SEO Framework installed (sites also using Genesis), I noticed that there are now thousands (yes, thousands) of added records in the options table. But this explosion of data is not happening with all the sites I have it installed on, and yes, the latest version is operational on all of those sites.

    The records seem to be three per post, named as:

    _transient_tsf_desc_noa_1_post_208_1_en_us
    _transient_timeout_tsf_1_011_ldjs_post_208_1_en_us
    _transient_tsf_1_011_ldjs_post_208_1_en_us

    And there are also some like this, my guess is for monthly archive data, which I have turned off on most of the sites:

    _transient_timeout_the_seo_f7_011_ldjs_month_12_09_1_en_us
    _transient_the_seo_f7_011_ldjs_month_12_09_1_en_us

    What are these records for, why are so many of these records being added to the options table, can they be deleted, and why do they only show up on some of the sites with the plugin and not others?

Viewing 8 replies - 1 through 8 (of 8 total)
  • Plugin Author Sybre Waaijer

    (@cybr)

    Hi @fpmsummer,

    Your advanced question will receive an elaborate answer, I hope that you can fully grasp its content as it will explain the basics of WordPress’ cache 🙂

    What is transient cache?
    The word transient is literally explained as an adjective: lasting only for a short time; impermanent.
    This literally means that transient cache (the entries you see in your database) only last for a short time.

    There are two prominent keys used in The SEO Framework:
    The first is tsf_desc_*, this is for the description.
    The second is tsf_ldjs_*, this is for the Schema.org data.

    The * part is automatically generated, based on the cache version and type (f7, noa_1, 011, etc.), the post type (post, page, month, year, etc.), ID (208, 1, 2, etc.) and locale (en_us, en_ca, nl_be, etc.).

    Both are maintained for at most 604800 seconds (1 week), after which the transient cache entry will be removed automatically by WordPress.
    The data is saved in two parts: The first part is the value which is found (description, schema.org data, etc.), and the second part is the timeout (a.k.a. expiration time-stamp).
    This is why each entry has two copies, one of which is aptly named.

    A developer could choose to automatically overwrite these entries on specific actions. For instance, if you change the SEO settings of a page, the transient will be removed to make room for a new one.

    Why use it?
    Sometimes, complex calculations need to take place that require either a lot of memory or a lot of processing power. Either way; on a shared hosting server these calculations should be limited as much as possible.
    For example, the generation of Schema.org data is very resource intensive because it loads other “linked” posts, pages and archives which are of no use to the actual visitor, but are great for Search Engines. Recalculating the outcome is a waste of your server’s resources, and the outcome is extremely persistent (unless you’d like to live in the past).

    These complex calculations can slow your website down; saving the result of the calculations in the database (or even better: memory) can greatly improve your website’s performance.

    This shouldn’t be applied to all code, as a database call is actually very slow. Nevertheless, in my calculations the usage of the transient cache far outperformed just a single regeneration.

    In effect, your whole server can maintain more users, and a faster website is also good for SEO.

    Great! Now, why do some sites have more entries?
    As stated earlier, the transient cache key entries are stored for 604800 seconds. After that, they will be cleaned up automatically.
    4 database entries are added for each and every single post, page, archive, author, etc. page where The SEO Framework runs: Two are for the description and schema.org values, and two for both their timeouts.
    These entries are only created on request. So, if for example http://example.com/this-page-is-never-visited/ is never visited, no entry will likely be present.

    These transients are generated on both the front-end and the back-end.

    So if you have loads of posts and view 20 entries in the admin area, 40 new keys are generated (description only) by having the SEO Bar active alone.

    On the front end, when these 20 posts are visited again, 40 more keys will be generated for the Schema.org entries, as the Schema.org entries aren’t rendered in the admin area.

    In conclusion, more content over time with consistent requests equal more transient keys.

    So many keys! Does it slow my website down?
    Databases are built for data, vasts amounts of data. Database daemons like MariaDB and MySQL have optimized everything they could to handle this data. Storing a million new entries shouldn’t affect performance.

    Also, WordPress auto-loads all options (with a certain key active) from the database. Because of this, adding new options through the WordPress Option API is a very fast item.
    However, as it does this, it won’t touch the transient cache (with some exceptions).

    The transient cache entries are only loaded when they have to be loaded. This is why there are so many different keys: The caching data is spread out.

    Can I manually delete these entries from the database?
    The database entries will automatically be deleted by WordPress if the timeout passes, even if the plugin is no longer active.
    But sure, you can safely delete these entries. They will however come back, so it’s quite a waste of time and deleting them is actually only useful in development cycles.

    I don’t want to see The SEO Framework’s transients any longer!
    If you run a private dedicated HTTP server with a highly optimized PHP handler with an optimized caching handler, plus your SQL server is located on another instance; I can truly understand you’d rather have these kind of transients disabled.
    I could also understand that you’d like to test the performance of it being disabled.

    You can do so by defining the following constant; you should add the code below to your wp-config.php file or a (mu-)plugin before action plugins_loaded priority 5:

    define( 'THE_SEO_FRAMEWORK_DISABLE_TRANSIENTS', true );
    

    This will effectively disable these kinds of transients. A few transients are kept, however; even with the constant defined. This is done for several reasons; like for consistent passive interaction with the Title Fix extension.

    Please also note that the use of this constant will disable the transient for the Sitemap as well. Depending on the amount of entries, this can greatly affect performance.

    Footnotes:

    1. The “month, year, day” names in the transient cache key names reflect the WordPress archive type; these could just as well be named “movies, actors, directors”. month_12_09 means December 2009.
    2. The disabling of transients within The SEO Framework won’t remove these transients from the database directly.
    3. WordPress cleans up transient cache entries on a Cron job (AFAICR).
    4. The caching performance can greatly differ from server to server.
    5. Caching plugins can overwrite the default transient behavior. I do condemn using such feature as it generally leads to unwanted artifacts.
    6. Crawlers (like Googlebot) can induce generation of these caching entries. The explosion could mean that there’s (finally) some good interaction.
    7. I’ve left Easter eggs in the transient entries for boolean or unexpected code-only values.

    Resources:
    Key generation code and Why have you disabled breadcrumbs?

    I hope this all clears things up! 🙂 Do let me know if you still have questions. Cheers!

    Makes perfect sense, thank you!

    A little confused about why the transients for archives are there at all, since I have those turned off in most places (for monthly and for categories), so why those archive types are creating transients when they are disabled was a question… you’re saying it’s not a matter if they’re disabled for indexing, the fact that a crawler visits them is enough to create that transient?

    I’m about to move one site that has a lot of these, and I want the database to be as clean as possible before moving (security issues testing).

    Plugin Author Sybre Waaijer

    (@cybr)

    Hi @fpmsummer,

    If I understand it correctly, with “Turning off the archive” you mean “applying noindex to the archive”.
    Because the archives are most likely not blocked by the actual robots.txt file (which is good!), crawlers like Googlebot will still visit those pages, even if it has noindex applied.

    The noindex option has no effect on the cache, so if anything visits the archive or any page, a new cache entry will still be created.
    These cache entries can also be created by merely visiting the archive or post overview in the admin area, as most SEO data is generated there as well in order to output the SEO Bar.

    So, whatever the case, all these keys will likely be generated anyway.

    Transients and security
    If you wish to test for security issues, I can assure you that transients shouldn’t affect this.
    In fact, although the transients do carry over with most backup plugins and through a manual database transfer, they do not hold any significant data that could be accessed anymore directly than WordPress’ options themselves.
    This is because WordPress’ Transient API makes use of the Options API.

    Also, if you wish to test for security issues, these keys will then also be generated; and if it’s an actual copy of the main site, they should be quite identical (apart from the timeout’s time-stamp).

    By default, if there’s a security issue, it should be correlated closer to WordPress or the actual Database than anything that correctly utilizes its features. Of course, this is a very broad aspect; then again, I’m not quite sure what you wish to test 🙂

    The SEO Framework, since 2.7.0, is linted against many common security issues (and even more so) through WordPress.com VIP Code Review‘s standards.

    That said, however, there is a plugin available to delete all transients on request: Transient Cleaner.

    Footnote: What bots crawl
    Google, like many other Search Engines, will still crawl “noindex” pages and search for links within, so be sure not to block them in robots.txt. So having these pages still accessible can improve your SERP.
    Having “noindex” enabled does help prevent duplicated content issues and does prevent popular Search Engines like Google from showing the pages on their Results Pages.
    Also, not every Search Engine crawler abides to the robots rules, they’re considered “bad bots”.

    Plugin Author Sybre Waaijer

    (@cybr)

    Hi @fpmsummer,

    Great news! From The SEO Framework 2.8.0 a new settings box will be introduced.
    Within that box you can choose to disable certain transients.

    Performance Settings TSF 2.8.0

    Cheers 🙂

    That’s great Sybre! As I read it, the Transient Cache is recommended and from the screenshot it looks like its enabled as default. That’s great. I know one should go through all the settings to see if they are optimal to the sites, but sometimes we just need some guidance when it comes to advanced stuff (and transient cache does sound advanced to a non-developer I guess).

    Once again – great work, and thanks for explaining this topic 🙂

    Plugin Author Sybre Waaijer

    (@cybr)

    Hi Bjarne,

    Anytime! 🙂

    I believe the descriptions aren’t quite clear enough on that topic, so I will expect some questions on this one. As you can already see, a few huge comments on this matter have already been posted within this very topic already.

    But it pretty much leads to the following, and I’ll keep this comment ready for the future:
    1. Disable Sitemap Cache if (any):

    • your website uses proxy cache (static CDN, Varnish);

    2. Disable Description Cache if (any):

    • your website uses proxy cache (static CDN, Varnish);
    • your website uses object cache (from a plugin);
    • your website uses page cache;
    • most of your posts’ and pages’ content is shorter than 2500 words;
    • your website contains thousands of posts;
    • your database caching acts up;

    3. Disable Schema.org Cache if (any):

    • your website uses proxy cache (static CDN, Varnish);
    • your website uses object cache (from a plugin);
    • your website uses page cache;
    • your website only has one or two categories;
    • your website contains thousands of posts;
    • your database caching acts up;

    The SEO Framework outputs its calculation time in the HTML source code of the page at the bottom of the SEO meta tags. So it’s up to the users to find out what works best :).

    The transients won’t vanish upon disabling one of the options. They’ll just become inactive and won’t be overwritten anymore. They’ll be cleaned up by WordPress core automatically after they expire and they have a lifetime of one week.

    Plugin Author Sybre Waaijer

    (@cybr)

    Hi @fpmsummer,

    TSF 2.8.0 has just been released. With that, the new options are available.

    You can safely remove the definition (if implemented) that I’ve suggested earlier in favor of the new options.

    Enjoy! 🙂

    Excellent! Can’t wait to check that out 🙂

Viewing 8 replies - 1 through 8 (of 8 total)
  • The topic ‘Strange increase in description transients’ is closed to new replies.