The plugin slows down the site by littering the wp_options table
-
For synchronization jobs, the plugin uses the wp_options table. As a result, after a while I got more than 75,000 entries of the form wc_facebook_background_product_sync_job_* in the wp_options table. It’s not normal, is it? It is wrong to store something in wp_options besides site settings, plugins and template.
It also slows down the product list page in the admin area, because a separate SQL query for get synchronization status is executed for each product (each row).
I suggest storing synchronization jobs in a separate table in which can configure the appropriate indexing and cleaning by creation time or status.
-
As a result, after a while I got more than 75,000 entries of the form wc_facebook_background_product_sync_job_* in the wp_options table. It’s not normal, is it?
No, this is not normal. Background processes that have completed are cleaned up daily. For you to have this many left in your database means one of two things:
- Your jobs aren’t being marked as
completed
, and therefore are not being cleaned up. - Your background process isn’t running daily as it should.
If you look in the database at the
option_value
column, you should see some JSON for the job. Would you be able to paste the JSON here from some of the oldest jobs?There are not so many of them with failed.
{"requests":{"\u0432\u0438\u043d\u043d\u044b\u0439\u041c1\u043a\u043e\u043c\u0444\u043e\u0440\u0442\u041a\u0440\u0443\u04226_10455":"DELETE","p-10455":"UPDATE"},"id":"863abc633168a273085314521000c335","created_at":"2021-05-04 14:05:03","created_by":0,"status":"failed","started_processing_at":"2021-05-04 14:05:44","updated_at":"2021-05-04 14:05:45","total":2,"progress":2,"handles":["AcxJMYkTfvHDueZ9tCI-q9rGSKXsnrUQqpE0ZxZT6e9My8aIvMevtx2dz9M_xxqbir48q8-aYjk-qs5kF1auKfCeUBaqA95awS4MbABbHlTNEQ"],"completed_at":"2021-05-04 14:05:45","failed_at":"2021-05-04 14:05:47","failure_reason":"Uncaught Exception: Job data key \"requests\" not set in \/var\/www\/domain.com\/htdocs\/wp-content\/plugins\/facebook-for-woocommerce\/includes\/Products\/Sync\/Background.php:65\nStack trace:\n#0 \/var\/www\/domain.com\/htdocs\/wp-content\/plugins\/facebook-for-woocommerce\/vendor\/skyverge\/wc-plugin-framework\/woocommerce\/utilities\/class-sv-wp-background-job-handler.php(588): SkyVerge\\WooCommerce\\Facebook\\Products\\Sync\\Background->process_job()\n#1 \/var\/www\/domain.com\/htdocs\/wp-content\/plugins\/facebook-for-woocommerce\/vendor\/skyverge\/wc-plugin-framework\/woocommerce\/utilities\/class-sv-wp-background-job-handler.php(168): SkyVerge\\WooCommerce\\PluginFramework\\v5_10_0\\SV_WP_Background_Job_Handler->handle()\n#2 \/var\/www\/domain.com\/htdocs\/wp-includes\/class-wp-hook.php(292): SkyVerge\\WooCommerce\\PluginFramework\\v5_10_0\\SV_WP_Background_Job_Handler->maybe_handle()\n#3 \/var\/www\/domain.com\/htdocs\/wp-includes\/class-wp-hook.php(316): WP_Hook->apply_filters()\n#4 \/var\/www\/domain.com\/htdocs\/wp-includes\/plugin.php(484): WP_Hook->do_action()\n#5 \/var\/w"}
Most tasks with the status “completed”. For example:
{"requests":{"\u043a\u043e\u0440\u0433\u0438\u041c1+\u04221_18660":"DELETE","p-18660":"UPDATE","p-17072":"UPDATE"},"id":"bbbf6746b5eeae8e95a7c0b8035a2e19","created_at":"2021-09-26 15:54:24","created_by":0,"status":"completed","started_processing_at":"2021-09-26 16:06:36","updated_at":"2021-09-26 16:06:47","total":3,"progress":3,"handles":["AcxFcCvticu_mOsu1kXhUOu5Ch5LnJI2d2KEn3bqbv4DD4M3my0r14nm-s-OY4BcYh7-5adTl1H9F9n-8SkKFn8Q"],"completed_at":"2021-09-26 16:06:47"}
The plugin is always the latest version, WordPress and WC too.- This reply was modified 3 years, 1 month ago by glashkoff.
Thanks for the logs @glashkoff,
@jpry I’ll skip this one for you in case you have a clue about what may be causing this!
Just in case, I will clarify why I wrote that storing a job in wp_options is an undesirable solution. This can be useful if someone decides to repeat the same thing that this plugin does. When data is stored this way, site cannot make a simple and fastest request to MySQL like
DELETE FROM "wp_options" WHERE created_at < DATE_SUB(NOW(), INTERVAL 1 DAY) AND status = "completed"
Instead, the plugin does an unpredictably resource-intensive operation.
First, the plugin needs to request all entries with wc_facebook_background_product_sync_job_*. It can be a long time if something goes wrong and the records stop being cleared, or there are a lot of products in the store.
Then the plugin needs to parse the data in JSON format inside them in order to get the status of the task and perform manipulations with it (sort, delete or change). There may not be enough time for this as part of the background execution of tasks if the site uses the standard cron WP. And even if such site background jobs are run through the wp-cli, there may not be enough server RAM to perform these tasks. Or it just prevents the site from being responsive when background tasks are running.
The above problem (not from the previous post, but about potential interrupting jobs) is observed by me on many sites that have more than ~100 products. Each of them can have hundreds of variations, so the total number of jobs required for processing can be even 10,000 or more. I dream that someday the plugin will work more efficiently. Thanks for your work anyway, devs.I also have same problem, with about 1600 rows, but problem is that some have option_value size of like 400 000 characters, for example :
{“requests”:{“p-143606″:”UPDATE”,”p-143597″:”UPDATE”,”p-143435
and so on for 400 000 characters long.Some other database rows are more “normal” size. EDIT: I also have pretty many products and variations, about 7000 products, with a lot more variations.
- This reply was modified 3 years, 1 month ago by bjorngus.
@bjorngus Please start your own thread instead of posting in this one.
Just in case, I will clarify why I wrote that storing a job in wp_options is an undesirable solution.
Thank you for your explanation. We do understand why it is desirable to avoid using the
wp_options
table, for all of the reasons you specified. Unfortunately, reworking the database storage is not something we’re able to do in the near future. But since this plugin is open-source, you’re welcome to submit an issue and/or pull request on the public repository.There are not so many of them with failed. Most tasks with the status “completed”.
It sounds like the action to clean up the old rows is either not running at all, or it is failing when it does run. If you go to the Admin Dashboard, then WooCommerce > Status > Scheduled Actions, you can use the search field to search for
facebook_for_woocommerce_daily_heartbeat
. Does it seem like many of these are either “Pending” or “Failed”?@jpry yes, it is. All old events have the status “Failed”. Their log shows that they run from 6 to 15 minutes. Well, actually, running tasks through the wp-cli helped to cope with this.
The second problem (it is impossible to open /wp-admin/edit.php?post_type=product) I managed to solve by forcibly deleting the “Facebook sync” column in the plugin code. Unfortunately, not all sites can turn off this column manually through the admin interface, because the list of products does not open due to the expiration of the PHP script timeout.
About the contribution to open source: I see a lot of messages in Issues about the inability to synchronize products. Maybe it has to do with storing the job in wp_options. Right now I don’t have enough time to understand the project and create a quality solution for pull request. But I will try to return to this issue when I deal with the elimination of the consequences that this plugin and others brought to my clients websites.All old events have the status “Failed”. Their log shows that they run from 6 to 15 minutes.
Typically these log entries should also include an error message when a failure occurs. This will be in the far right column labeled “Log”. Could you copy the log data from that column here so we can see what is happening?
The second problem (it is impossible to open /wp-admin/edit.php?post_type=product) I managed to solve by forcibly deleting the “Facebook sync” column in the plugin code. Unfortunately, not all sites can turn off this column manually through the admin interface, because the list of products does not open due to the expiration of the PHP script timeout.
If you (or anyone else finding this later) needs a way to turn this off in the future, here is a link to a method for disabling these columns without being able to access the screen in question: https://gist.github.com/JPry/a701f30f3d99d5b5479801788bad415e.
Typically these log entries should also include an error message when a failure occurs. This will be in the far right column labeled “Log”. Could you copy the log data from that column here so we can see what is happening?
I can’t provide logs right now, sorry. Unfortunately, at the time of your message, I had already thoroughly cleaned all the logs to reduce the size of the databases. I will wait for new errors to appear.
- Your jobs aren’t being marked as
- The topic ‘The plugin slows down the site by littering the wp_options table’ is closed to new replies.