• We noticed that the following query repeatedly causes a high volume of temporary tables in our MySQL server environment:

    (Reference: Line 2126 of powerpress.php latest version)

            if( powerpress_is_custom_podcast_feed() || get_query_var('feed') === 'podcast' )
            {
                    global $wpdb;
                    $join .= " INNER JOIN {$wpdb->postmeta} AS pp_{$wpdb->postmeta} ";
                    $join .= " ON {$wpdb->posts}.ID = pp_{$wpdb->postmeta}.post_id ";
            }
    

    The query is generated as:

    SELECT wp_posts . * FROM wp_posts INNER JOIN wp_postmeta AS pp_wp_postmeta ON wp_posts . ID = pp_wp_postmeta . post_id WHERE ? = ? AND wp_posts . post_name = ? AND wp_posts . post_type = ? AND ( pp_wp_postmeta . meta_key = ? ) GROUP BY wp_posts . ID ORDER BY wp_posts . post_date DESC

    If I am reading it correctly, the root cause appears to be that MySQL is generating a temporary table to handle the sort:

    SELECT wp_posts . * FROM wp ...wp_posts.post_date` DESC

    We’ve added an index on the post_date field in wp_posts to try and mitigate the full row scan via temp table.

    Do you have any recommendations or improvements that can be made to this plugin’s query to prevent the operation from creating a high volume of temporary tables on disk?

    Statistics: Of the half dozen or so customers that use this plugin regularly on this hosting node, we observe about 13k temporary tables per hour generated by this query.

Viewing 9 replies - 1 through 9 (of 9 total)
  • Thread Starter maplegrovehosting

    (@maplegrovehosting)

    Here is a copy of the explain plan as well:

    https://maplegrovepartners.com/public/powerpress_query_explain.jpg

    Plugin Support Shawn

    (@shawnogordo)

    The Blubrry dev team has been made aware of this post. A response will be posted here as soon as they’ve had a chance to take a look.

    Thread Starter maplegrovehosting

    (@maplegrovehosting)

    Thank you. Just to add to this, confirming that the order by gets added starting line 179 of powerpress-playlist.php, and a lot of queries I see in the query log the result set is already filtered down to one row making the ORDER BY unnecessary in that instance.

    For example, if the build query contains: wp_posts.post_name = “my post title”
    you should always expect a single result, and there for not require the order by to be appended.

    Hoping we can review the conditionals of when the ORDER BY needs to be appended to the query to minimize its use accordingly to when its actually relevant to the result set.

    Thread Starter maplegrovehosting

    (@maplegrovehosting)

    Did significant more digging into the code base and research of the issue. Looks like the powerpress.php uses WP query filters to build the requisite query. I still have not been able to reproduce/trace fully what action causes the query in the original post to be generated. I attempted adding the following patch as a hacky mitigation to test the theory:

    
    function powerpress_posts_orderby($orderby) {
            global $wp_query;
            error_log($wp_query->request);
            if(strpos($wp_query->request, 'post_name ='))
            {
                    $orderby = '';
            }
    
            return $orderby;
    }
    add_filter('posts_orderby', 'powerpress_posts_orderby');
    

    Still seeing the query getting run after this hotpatch. When I log out this filter I’ve added to powerpress.php it doesn’t ever log the query that mysql is noting in its statement analysis logs (where we originally discovered the issue).

    Hoping this jogs enough technical inspiration to help root cause when looked at as plugin dev team can probably shoot this one a bit quicker :–)

    Thread Starter maplegrovehosting

    (@maplegrovehosting)

    I now have a working root cause and a partial solution.

    Root cause: When a /feed/ is generated for an individual post, powerpress does not alter the ORDERBY clause (as there is no filter) to remove the default sort by post_date. Given that its a single post, we can be confident an order by is not required, and we can us the built-in wp API to check for this condition.

    
    function powerpress_posts_orderby($orderby) {
            if(is_single())
            {
                    $orderby = '';
            }
            return $orderby;
    }
    add_filter('posts_orderby', 'powerpress_posts_orderby');
    

    in powerpress.php, line 2188

    We’re still seeing the temporary tables getting logged with post_name = ? in the WHERE clause, so there is likely more conditional logic needed in the if to handle other cases that the plugin uses that I haven’t considered yet. The case above seemed representative of the issue at first glance.

    Thread Starter maplegrovehosting

    (@maplegrovehosting)

    “SELECT SQL_CALC_FOUND_ROWS wp_posts.ID FROM wp_posts INNER JOIN wp_postmeta AS pp_wp_postmeta ON wp_posts.ID = pp_wp_postmeta.post_id WHERE 1=1 AND wp_posts.post_type = ‘post’ AND (wp_posts.post_status = ‘publish’) AND ( pp_wp_postmeta.meta_key = ‘enclosure’ ) GROUP BY wp_posts.ID ORDER BY wp_posts.post_date DESC LIMIT 0, 100”

    Is another additional query that’s running very frequently in our environment when I debug via PHP vs MySQL, but it does not consume a temporary table to disk like the original post does.

    Thread Starter maplegrovehosting

    (@maplegrovehosting)

    As an update while waiting for a developer response, we mitigated impact by leveraging the MySQL Rewriter Plugin to rewrite the queries as follows:

    SELECT wp_posts . * FROM wp_posts INNER JOIN wp_postmeta AS pp_wp_postmeta ON wp_posts . ID = pp_wp_postmeta . post_id WHERE ? = ? AND wp_posts . post_name = ? AND wp_posts . post_type = ? AND ( pp_wp_postmeta . meta_key = ? ) GROUP BY wp_posts . ID ORDER BY wp_posts . post_date DESC

    TO

    SELECT wp_posts . * FROM wp_posts INNER JOIN wp_postmeta AS pp_wp_postmeta ON wp_posts . ID = pp_wp_postmeta . post_id WHERE ? = ? AND wp_posts . post_name = ? AND wp_posts . post_type = ? AND ( pp_wp_postmeta . meta_key = ? )

    and

    SELECT SQL_CALC_FOUND_ROWS wp_posts . ID FROM wp_posts INNER JOIN wp_postmeta AS pp_wp_postmeta ON wp_posts . ID = pp_wp_postmeta . post_id WHERE ? = ? AND wp_posts . post_name = ? AND wp_posts . post_type = ? AND ( pp_wp_postmeta . meta_key = ? ) GROUP BY wp_posts . ID ORDER BY wp_posts . post_date DESC LIMIT ?,?

    TO

    SELECT SQL_CALC_FOUND_ROWS wp_posts . ID FROM wp_posts INNER JOIN wp_postmeta AS pp_wp_postmeta ON wp_posts . ID = pp_wp_postmeta . post_id WHERE ? = ? AND wp_posts . post_name = ? AND wp_posts . post_type = ? AND ( pp_wp_postmeta . meta_key = ? )

    Plugin Author benbeecroft

    (@benbeecroft)

    Hello, and thank you for using PowerPress! We (the Blubrry Dev team) are looking into this. Thank you for your feedback and proposed solution.

    Plugin Author Angelo Mandato

    (@amandato)

    Hello @maplegrovehosting,

    with your posts and post_meta tables, is the ID / post_id column in each table indexed?

    What is the values in the query for LIMIT? Your first post did not include the “LIMIT X, Y” parameter, which WordPress adds to the query. Without this, it will scan the entire posts table, which is most likely why the results then need to be placed in a temp table to group by then sort.

    So we can reproduce the same results, please give us a list of plugins and the theme name that is currently activated. Also if you can help us further with this, disable all plugins other than PowerPress and use a stock theme such as twentyeightneen, this will help us know if it is only a problem with PowerPress or a problem with a combination of PowerPress and another theme/plugin.

    Thanks,
    Angelo

Viewing 9 replies - 1 through 9 (of 9 total)
  • The topic ‘Custom Podcast Feed INNER JOIN Causes Large Volume of Temporary Tables’ is closed to new replies.