• I have 2 separate multisite installations both with their own buckets.

    Site 1 after upgrade, stuck at:

    Running Attachments Update (60.77% Complete) — We’re going through all the offloaded Media Library items and updating the metadata with the sizes of files that have been removed from the server. This will allow us to serve the correct size for media items and the total space used in Multisite subsites. This will be done quietly in the background, processing a small batch of Media Library items every 2 minutes. There should be no noticeable impact on your server’s performance

    (i have access to change settings)

    Site 2 after upgrade, stuck at:

    Running Attachments Update (80.75% Complete) — We’re going through all the offloaded Media Library items and updating the metadata with the sizes of files that have been removed from the server. This will allow us to serve the correct size for media items and the total space used in Multisite subsites. This will be done quietly in the background, processing a small batch of Media Library items every 2 minutes. There should be no noticeable impact on your server’s performance. Pause Update

    Settings Locked Temporarily — You can’t change any of your settings until the “Metadata Update” has completed.

    (I have no access to change settings)

    I will pause update and restart but nothing seems to change.

    So moving forward is there any command line operations I could perform to see whats going on, I am unclear how to debug this.

    I am NOT running the EWWW plugin

    Thanks guys, really appreciate the plugin.

Viewing 13 replies - 1 through 13 (of 13 total)
  • Plugin Contributor ianmjones

    (@ianmjones)

    You should have “Settings Locked Temporarily” in both cases, so on site 1 you possibly have issues with Object Caching, cron, loopback requests, or any combination of them, or maybe something else. ¯\_(ツ)_/¯

    Either way, you need to check the Health Status of both site 1 and 2 to ensure all tests are passing.

    We have a doc that may help you debug background processes.

    https://deliciousbrains.com/wp-offload-media/doc/background-processes-not-completing/

    You can also try pausing the update, enable debugging with WP_DEBUG and WP_DEBUG_LOG set to true and WP_DEBUG_DISPLAY set to false, resume the update and then check the wp-content/debug.log file after a few minutes.

    @ianmjones

    Yes have looked at that document, and I will test WP_DEBUG_LOG, WP_DEBUG_DISPLAY

    Some other notes I should have mentioned. There is a 3rd multisite that upgraded without issue.

    The 2 that are struggling have been using your plugin for a longer time (3+ years I think now) while the site that had no issues had only been using the plugin for a month.

    All sites are using WPMUDEV smush. All sites are set to delete the local file. All sites are set with cname and using the cloudlflare with webp compression via cloudflare.

    Have you guys tested against these plugins, and what is the process its getting stuck at you think? Does it have to bring the file locally or is it simply scanning the s3 bucket for what files exist?

    Thanks

    Follow up question after reviewing more closely.:

    https://deliciousbrains.com/wp-offload-media/doc/background-processes-not-completing/

    Basic Authentication
    Where would that code snipped be best added for a multisite situation? Should i add it into your plugin’s file: wordpress-s3.php and when the update completes is it necessary to have that code?

    Hostname Resolution
    I have my dns via cloudflare and have some other security protocols instituted. My websites have no problems being visible via web browsers, but pinging my site for the first time responds with: unknown host

    So I am guessing this is a result of cloudflare security measures?

    And as further side note I am now testing with a max execution time of 300 (was previously 30) and have set the cloudflare account to development mode.

    Another note, after turning on the debug logging, I see this related item referenced:

    AS3CF: Total file size for the attachment is 0: 131

    I’m not sure what to make of that or why issue that would cause.

    so in the logs AS3CF: Total file size for the attachment is 0: 131 keeps repeating.

    I went into your code and changed the line printing this statement to instead print to the log: print_r($attachment, true) which reveals the file that perhaps keeps hanging your plugin…possible bug that it could get hung up on one file?

    It points to a site that is very old and unused. I inspected the file and it exists remotely

    AS3CF: Total file size for the attachment is 0: 131

    [22-Jul-2020 16:50:44 UTC] AS3CF: Total file size for the attachment is 0: stdClass Object
    (
        [ID] => 131
        [provider_object] => a:3:{s:6:"bucket";s:18:"my.cname.bucket100";s:3:"key";s:69:"content/uploads/sites/70/2014/10/08170557/khurley1-myfilename1000.jpg";s:6:"region";s:9:"us-west-1";}
    )

    I will next try to delete the file and see if that makes a change, but would be good to report how this file could somehow break everything.

    I deleted that file and restarted, the next file its getting stuck on is a .mov file its getting stuck on is a .mov file that does not exist and is referenced from an older blog id path of a blog that was deleted.

    If some other updates didn’t properly finish on previous version for whatever reason from sites that have been using your plugin for a longer time, then perhaps some error checking needs to be done for that and that was overlooked? Thanks Ian, looking forward to your thoughts.

    More updates: searching the database for the file name, i found that the .mov file is referenced from another blog in the their media library (created in 2014) which is strange. I deleted it. And now it seems to be making progress with the % of completion changing slowly. In the logs, I am seeing the following notice:

    PHP Notice: Trying to get property ‘attachment_id’ of non-object in /mypath/wp-content/plugins/amazon-s3-and-cloudfront/classes/as3cf-filter.php on line 116

    But it doesn’t seem to stop the progress indicator % rising (slowly)

    Thanks for the debug tips, fingers crossed and I will keep this thread updated. If you have idea about that notice and if I should be diving into that notice deeper please let me know. Thanks

    Another update and revealing a possible bug(i replaced my bucketname cname with something generic for this post:

    WordPress database error You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 's-Road%' OR post_content LIKE '%//my.cname.bucket100/content/uploads/sites/13/20' at line 1 for query SELECT ID FROM wp_13_posts WHERE ID > 0 AND ID <= 27 AND (post_content LIKE '%//my.cname.bucket100/content/uploads/sites/13/2017/03/29182625/SPC-Painting-St-Mary's-Road%' OR post_content LIKE '%//my.cname.bucket100/content/uploads/sites/13/2017/03/29182625/SPC-Painting-St-Mary%27s-Road%') made by do_action_ref_array('as3cf_cron_update_replace_provider_urls'), WP_Hook->do_action, WP_Hook->apply_filters, DeliciousBrains\WP_Offload_Media\Upgrades\Upgrade->do_upgrade, DeliciousBrains\WP_Offload_Media\Upgrades\Upgrade->run_upgrade, DeliciousBrains\WP_Offload_Media\Upgrades\Upgrade_Content_Replace_URLs->upgrade_blog, DeliciousBrains\WP_Offload_Media\Upgrades\Upgrade->upgrade_blog, DeliciousBrains\WP_Offload_Media\Upgrades\Upgrade_Filter_Post->upgrade_item, DeliciousBrains\WP_Offload_Media\Upgrades\Upgrade_Filter_Post->find_and_replace_attachment_urls, DeliciousBrains\WP_Offload_Media\Upgrades\Upgrade_Filter_Post->process_pair_replacement

    and

    [22-Jul-2020 21:53:36 UTC] WordPress database error You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘s-Road-(2)%’ OR items.original_source_path LIKE ‘2017/04/SPC-Painting-St-Mary’s-‘ at line 4 for query
    SELECT DISTINCT items.*
    FROM wp_13_as3cf_items AS items USE INDEX (uidx_source_path, uidx_original_source_path)
    WHERE items.source_type = ‘media-library’
    AND items.source_id NOT IN (1957) AND (items.source_path LIKE ‘2017/04/SPC-Painting-St-Mary’s-Road-(2)%’ OR items.original_source_path LIKE ‘2017/04/SPC-Painting-St-Mary’s-Road-(2)%’) made by do_action(‘wp_ajax_delete-post’), WP_Hook->do_action, WP_Hook->apply_filters, wp_ajax_delete_post, wp_delete_post, wp_delete_attachment, do_action(‘delete_attachment’), WP_Hook->do_action, WP_Hook->apply_filters, Amazon_S3_And_CloudFront->delete_attachment, Amazon_S3_And_CloudFront->remove_attachment_files_from_provider, DeliciousBrains\WP_Offload_Media\Items\Media_Library_Item::get_by_source_path
    [22-Jul-2020 21:53:41 UTC] WordPress database error You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘s-Road%’ OR items.original_source_path LIKE ‘2017/03/SPC-Painting-St-Mary’s-Road’ at line 4 for query
    SELECT DISTINCT items.*
    FROM wp_13_as3cf_items AS items USE INDEX (uidx_source_path, uidx_original_source_path)
    WHERE items.source_type = ‘media-library’
    AND items.source_id NOT IN (1910) AND (items.source_path LIKE ‘2017/03/SPC-Painting-St-Mary’s-Road%’ OR items.original_source_path LIKE ‘2017/03/SPC-Painting-St-Mary’s-Road%’) made by do_action(‘wp_ajax_delete-post’), WP_Hook->do_action, WP_Hook->apply_filters, wp_ajax_delete_post, wp_delete_post, wp_delete_attachment, do_action(‘delete_attachment’), WP_Hook->do_action, WP_Hook->apply_filters, Amazon_S3_And_CloudFront->delete_attachment, Amazon_S3_And_CloudFront->remove_attachment_files_from_provider, DeliciousBrains\WP_Offload_Media\Items\Media_Library_Item::get_by_source_path

    • This reply was modified 4 months, 2 weeks ago by jiggaman.

    So since that error it got to: Running Excerpts Upgrade (76.98% Complete) without any explicit s3 messaging of errors…ideas?

    Success! Using the the filters as3cf_update_replace_s3_urls_batch_size, as3cf_update_replace_s3_urls_time_limit

    I increased the time, and then lowered max attachments to per batch. That got it “unstuck” and then it complete I think. No notification that it did complete but did show controls again. GREAT. Network one completed….now for network 2.

    It’s stuck on 0. Its a bigger network and I have beefier settings:
    PHP Memory Limit: 256M
    WP Memory Limit: 256M
    WP Max Upload Size: 200 MB
    PHP Time Limit: 300
    Media Files: 11,454
    Offloaded Media Files: 3,404
    Not Offloaded Media Files: 8,050

    OME Metadata Version: 1

    Whats differeent is the Not Offloaded Media Files seems higher and the OME Metatdata version is on 1 while the other one now says 8, any ideas on how to get this one moving as well?

    I haven’t heard back…so I’m guessing you guys are working on these bugs I brought up? I still love you guys! Let us know whats up. And further update I got this network completely updated with your suggestions of debugging.

    That said, my second larger network I can’t get past 0% and i am not able to see any debugging messages that I can fix related to your plugin.

    jiggaman

    (@jiggaman)

    Just another heads up to the developers:
    classes/upgrades/upgrade-filter-post.php

    abstract class Upgrade_Filter_Post extends Upgrade {
    
            /**
             * @var int Time limit in seconds.
             */
            protected $time_limit = 10;
    
            /**
             * @var int Batch size limit for this request session.
             */
            protected $size_limit = 50;

    These numbers aren’t getting over written by mu plugin suggestion for upping the limits.

Viewing 13 replies - 1 through 13 (of 13 total)
  • You must be logged in to reply to this topic.