Support » Plugin: WooCommerce » After 7.2.1 update action scheduler and debug log have gone crazy

  • Resolved Krzysztof Płaneta

    (@chrisplaneta)


    Action scheduler actions and logs don’t clear and get millions of rows in the DB. It produces about 2MB of data a minute.

    Debug log increases about 1GB per 20 mins with millions of these errors.

    [21-Dec-2022 09:40:09 UTC] WordPress database error Unknown column ‘wp_postmeta.post_id’ in ‘where clause’ for query SELECT meta_id FROM m31w_postmeta,
                        (SELECT DISTINCT post_id FROM m31w_postmeta
                        WHERE (meta_key = ‘_billing_country’ OR meta_key=’_shipping_country’) AND meta_value=’NZ’)
                        AS states_in_country
                    WHERE (meta_key=’_billing_state’ OR meta_key=’_shipping_state’)
                    AND meta_value=’NL’
                    AND wp_postmeta.post_id = states_in_country.post_id
                    LIMIT 100 made by do_action_ref_array(‘action_scheduler_run_queue’), WP_Hook->do_action, WP_Hook->apply_filters, ActionScheduler_QueueRunner->run, ActionScheduler_QueueRunner->do_batch, ActionScheduler_Abstract_QueueRunner->process_action, ActionScheduler_Action->execute, do_action_ref_array(‘woocommerce_run_update_callback’), WP_Hook->do_action, WP_Hook->apply_filters, WC_Install::run_update_callback, wc_update_721_adjust_new_zealand_states, Automattic\WooCommerce\Database\Migrations\MigrationHelper::migrate_country_states, Automattic\WooCommerce\Database\Migrations\MigrationHelper::migrate_country_states_for_orders

Viewing 15 replies - 1 through 15 (of 21 total)
  • I was just informed that my entire disk was full, and it was due to the log file being full of these type of errors.

    This is a critical error, so I hope someone will look at this ASAP!

    • This reply was modified 1 month, 1 week ago by nipter.
    • This reply was modified 1 month, 1 week ago by nipter.
    Lisa Clarke

    (@polkadotcreations)

    I am having the exact same problem. I deleted my 16GB error_log to get my disk space back down under my limit, but that is obviously a temporary solution.

    Same thing happened to me this morning. After deleting a 26 GB Apache error log file to free up some disk space, I managed to transfer the previous day’s (merely 12 GB!) Apache error log file to my local system to investigate, and I discovered it was full of millions (seriously) of lines like this:

    [Wed Dec 21 00:00:26.821990 2022] [php:notice] [pid 237087] [client REDACTED:47642] WordPress database error Unknown column 'wp_postmeta.post_id' in 'where clause' for query SELECT meta_id FROM REDACTED_wp_postmeta,\n\t\t\t\t\t(SELECT DISTINCT post_id FROM REDACTED_wp_postmeta\n\t\t\t\t\tWHERE (meta_key = '_billing_country' OR meta_key='_shipping_country') AND meta_value='NZ')\n\t\t\t\t\tAS states_in_country\n\t\t\t\tWHERE (meta_key='_billing_state' OR meta_key='_shipping_state')\n\t\t\t\tAND meta_value='SL'\n\t\t\t\tAND wp_postmeta.post_id = states_in_country.post_id\n\t\t\t\tLIMIT 100 made by do_action_ref_array('action_scheduler_run_queue'), WP_Hook->do_action, WP_Hook->apply_filters, ActionScheduler_QueueRunner->run, ActionScheduler_QueueRunner->do_batch, ActionScheduler_Abstract_QueueRunner->process_action, ActionScheduler_Action->execute, do_action_ref_array('woocommerce_run_update_callback'), WP_Hook->do_action, WP_Hook->apply_filters, WC_Install::run_update_callback, wc_update_721_adjust_new_zealand_states, Automattic\WooCommerce\Database\Migrations\MigrationHelper::migrate_country_states, Automattic\WooCommerce\Database\Migrations\MigrationHelper::migrate_country_states_for_orders, referer: https://REDACTED/wp-cron.php?doing_wp_cron=1671580796.8379271030426025390625

    I have redacted my site name and WP database table prefix here, but this is clearly the issue. There’s a SQL query in something ActionScheduler is doing that is hardcoded to the wp_postmeta table, rather than using the site’s table prefix variable.

    Oops.

    • This reply was modified 1 month, 1 week ago by room34. Reason: Fixed formatting of log code excerpt
    Thread Starter Krzysztof Płaneta

    (@chrisplaneta)

    [Solution] I have a solution (I hope) but it is risky. So far, no errors on my site, but it is your call if you will go this way. This is what you should do:

    1. Download and install Advanced Database Cleaner plugin
    2. Go to Tools > WP DB Cleaner > Tables > find “actionscheduler”
    3. mark “actionscheduler_actions” and “actionscheduler_logs” and choose “empty rows” from the dropdown above the table
    4. find and enable some “maintenance mode” plugin
    5. Now, go over your whole list of plugins and deactivate them and reactivate them one by one (WooCommerce too)

    What this will do is all your scheduled actions should be restarted. In my case I am no longer experiencing the issue described in this post.

    Plugin Support Niall a11n

    (@wpniall)

    Hi everyone! The WooCommerce core developers are aware of this issue and have a PR ready to fix the issue.

    It should be released in WooCommerce 7.2.2 sometime soon. I’m checking if we can get a more specific timeframe for when that might be released.

    Plugin Support Niall a11n

    (@wpniall)

    There’s also a separate PR based off the previous one I shared that should fix any pending bad requests after the 7.2.1 update: Re-run the 7.2.1 state migrations when merchants update to 7.2.2. #36119

    Thank you @chrisplaneta !

    And thank you, @wpniall ! Will 7.2.2 clean up current failing jobs, so that @chrisplaneta fix won’t be necessary to do?

    @wpniall – and there I see you have already answered this question. That is great to know. Thanks!

    @chrisplaneta – Would you please be able to elaborate what is needed in point 4: “find and enable some “maintenance mode” plugin“. Is it a specific plugin you need to install? Or, do you have an example of such a plugin?

    UPDATE: I believe I get you. You need a plugin that puts your site into maintenance mode, for when you restart every plugin.

    • This reply was modified 1 month, 1 week ago by nipter.

    Please see this thread/post for more information.

    @chrisplaneta – I followed your solution guide, and luckily it worked well on my side as well. Thank you! and first it looked like it was working. However, the wpcc_actionscheduler_actionsand wpcc_actionscheduler_logs tables, plus the .php.error.log file started filling up again. It seems like I need to wait for 7.2.2.

    • This reply was modified 1 month, 1 week ago by nipter.
    • This reply was modified 1 month, 1 week ago by nipter.
    • This reply was modified 1 month, 1 week ago by nipter.

    UPDATE:
    I followed the above mentioned procedure, without re-activating WooCommerce. Then, the log-o-maniac stopped, and this site still works fine. Looking forward to version 7.2.2, where this should be fixed completely.

    • This reply was modified 1 month, 1 week ago by nipter.
    • This reply was modified 1 month, 1 week ago by nipter.

    Our solution was to simply download 7.2.0 and reinstall it. (Upload Plugin -> Replace Current with Uploaded)

    Wondering about clearing out the ActionScheduler data tables (mine have 2.6 million entries after this… situation).

    For those of us comfortable with direct database actions, is it fine to just truncate the wp_actionscheduler_actions and wp_actionscheduler_logs (prefixes removed) tables? I’d rather not install a random database cleanup plugin I’ve never heard of before to do something this simple, if that’s all there is to it.

    (Update: OK, well… I just decided to go ahead and do it after doing a mysqldump of the database as a precaution.)

    • This reply was modified 1 month, 1 week ago by room34.
Viewing 15 replies - 1 through 15 (of 21 total)
  • You must be logged in to reply to this topic.