WordPress.org

Ready to get started?Download WordPress

Forums

EWWW Image Optimizer
[resolved] Bulk optimize does not work (24 posts)

  1. James Kraus
    Member
    Posted 6 months ago #

    The bulk upload feature is displaying an empty screen for the site I'm working on. I'm using the cloud version right now, but the regular version didn't work either.

    Setting shows all oks:

    Cloud API Key: Verified, used 0 of 1000, usage will reset in 27 days.
    
    Only need one of these: finfo: OK  getimagesize(): OK  mime_content_type(): OK

    Debug log:

    Debug Log
    
    www-data
    WP version: 3.8.1
    EWWW IO version: 182
    ewww_image_optimizer_admin_init()
    ewww_image_optimizer_init()
    ewww_image_optimizer_cloud_verify()
    verification success via: 146.185.173.199
    verification body contents: great
    ewww_image_optimizer_disable_tools()
    ewww_image_optimizer_options()
    ewww_image_optimizer_cloud_verify()
    verification success via: 192.241.202.158
    verification body contents: great
    ewww_image_optimizer_cloud_quota()
    quota data retrieved: 1000 0 27
    ewww_image_optimizer_jpeg_background()
    ewww_image_optimizer_jpeg_background()
    ewww_image_optimizer_jpg_quality()

    And only this shows up in the logs:

    [Mon Feb 24 16:52:39 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:52:42 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:52:42 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:52:42 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:52:42 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:52:42 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:52:43 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:52:43 2014] [debug] mod_deflate.c(615): [client 198.137.20.180] Zlib: Compressed 242871 to 63731 : URL /wp-admin/load-styles.php, referer: http://wpdev.domain.com/wp-admin/upload.php?page=ewww-image-optimizer-bulk
    [Mon Feb 24 16:52:43 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:52:43 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:52:43 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:52:43 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:52:43 2014] [debug] mod_deflate.c(615): [client 198.137.20.180] Zlib: Compressed 1153 to 444 : URL /index.php, referer: http://wpdev.domain.com/wp-admin/upload.php?page=ewww-image-optimizer-bulk
    [Mon Feb 24 16:52:46 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:53:14 2014] [debug] mod_deflate.c(615): [client 198.137.20.180] Zlib: Compressed 746891 to 163162 : URL /wp-admin/upload.php, referer: http://wpdev.domain.com/wp-admin/options-general.php?page=ewww-image-optimizer-cloud/ewww-image-optimizer-cloud.php
    [Mon Feb 24 16:53:14 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:53:14 2014] [debug] mod_deflate.c(615): [client 198.137.20.180] Zlib: Compressed 172768 to 55623 : URL /wp-admin/load-scripts.php, referer: http://wpdev.domain.com/wp-admin/upload.php?page=ewww-image-optimizer-bulk
    [Mon Feb 24 16:53:14 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:53:14 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:53:14 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:53:14 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:53:14 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:53:14 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:53:14 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:53:14 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:53:14 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:53:14 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:53:14 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:53:14 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()
    [Mon Feb 24 16:53:14 2014] [debug] mod_headers.c(756): headers: ap_headers_output_filter()

    https://wordpress.org/plugins/ewww-image-optimizer/

  2. nosilver4u
    Member
    Plugin Author

    Posted 6 months ago #

    when you say it displays an empty screen, do you mean that it displays the left-hand wordpress menu, but the content area is blank?

    If so, how many images are in your media library?

  3. DLT95
    Member
    Posted 6 months ago #

    I have the same problem. The left-hand WP menu displays and the right-hand content stays blank. The page tries to load for about 15 seconds then stops.

    Our site has about 10,000 images.

    I should point out that this is the bulk optimise page, not the bulk upload page mentioned by the OP.

  4. nosilver4u
    Member
    Plugin Author

    Posted 6 months ago #

    I think the OP meant the bulk optimize as well, since there is no page titled bulk upload. The issue has been fixed in the dev version, release is due Saturday at the latest.

  5. DLT95
    Member
    Posted 6 months ago #

    Thanks, and thanks for a very useful plug-in.

  6. DLT95
    Member
    Posted 6 months ago #

    Installed the 1.8.3 update just now. Bulk optimise page content loads with a message about importing images into the table. I click on that button and get a twirling icon that stays for several minutes.

    Then I get this:

    Warning: Illegal string offset 'file' in xxxxxxxxxxxxxx/wp-content/plugins/ewww-image-optimizer/common.php on line 1297

  7. nosilver4u
    Member
    Plugin Author

    Posted 6 months ago #

    Sounds like the import is stalling because your max_execution_time is too low. Increase it via your php.ini (ask your webhost for help if you need to. The warning won't hurt anything, it just means one of your images doesn't have a filename stored in the post_meta table. If you get a lot of those, it might be worth taking a closer look at your media library. You can turn on debugging, and then look at the EWWW IO column in the Media Library to see what metadata is actually stored in the database for those particular images.

  8. nosilver4u
    Member
    Plugin Author

    Posted 6 months ago #

    DLT95, if you can hold on for a couple days, I have some ideas that could make the Import significantly faster. Just need to learn how to do multi-row inserts now...

  9. DLT95
    Member
    Posted 5 months ago #

    Thanks for the suggestions. I will look at them.

    It's not really a problem for me as nearly all our images have been optimised by your plug-in at the time of uploading anyway. I just wanted to run the bulk tool to see what it would find.

  10. nosilver4u
    Member
    Plugin Author

    Posted 5 months ago #

    problem should be fixed as Import is about 50x faster than it used to be.

  11. James Kraus
    Member
    Posted 5 months ago #

    Problem is not fixed.

    I updated and the Bulk Optimize script was erroring at 256MB of memory, so after increasing memory to 512MB I finally managed to get it to run for a few minutes. I stopped it, then navigated around for a bit.

    Now when I load the Bulk Optimize page I get the same blank screen, no errors. I'm pretty sure the script is just timing out. I've tried both optimizers (cloud and non-cloud) and it happens to both.

  12. James Kraus
    Member
    Posted 5 months ago #

    Selecting "Bulk Optimize" from the media library works fine, just using the link in the left sidebar doesn't work.

  13. nosilver4u
    Member
    Plugin Author

    Posted 5 months ago #

    Sorry, you never answered any of my questions, and the issue reported by DLT95 (which is apparently different than your problem) was fixed, that's why I marked it resolved. Deactivate all other plugins, and try loading the Bulk Optimize, as this sounds like a plugin conflict. Can't say for sure that it is, but I've not seen memory consumption issues on my test site, and it has 27,000 images.
    Please check any server error logs for clues, and turn on debugging in wp-config.php and see if that shows any errors or warnings.

  14. DLT95
    Member
    Posted 5 months ago #

    With 1.8.4 I get a memory allocation problem about which I have just commented in another support thread with a similar title to this one. I probably should have made the comment here instead. Sorry about that.

  15. nosilver4u
    Member
    Plugin Author

    Posted 5 months ago #

    James, you might also try taking a look at the ewwwio_images table in your database and make sure there aren't ludicrous amounts of images in there. You could even use the Empty Table button to see if that fixes anything. If that still doesn't help, the WP_DEBUG setting and checking the server error logs are a MUST.

  16. nosilver4u
    Member
    Plugin Author

    Posted 5 months ago #

    I've got some ideas on making this better, but it's going to take some time to implement.

  17. nosilver4u
    Member
    Plugin Author

    Posted 5 months ago #

    James, improved Import is in 'dev', although I don't think that has anything to do with the current problem, which I'm still not sure what the problem is since you haven't reported any further error messages or provided any debugging output that would pinpoint the issue.

  18. nosilver4u
    Member
    Plugin Author

    Posted 5 months ago #

    James, if I don't hear anything further by tomorrow, I'll have to assume the issue has been resolved.
    If you would like to troubleshoot further, please answer these questions:
    1. how many images are in your media library?
    2. have you increased max_execution_time in your php configuration?
    3. have you checked your error logs? I know you posted some logging info originally, but you did not specify if that was from access logs or error logs.
    4. Have you tried turning on debugging in wp-config.php? This is totally different than turning on debugging for the plugin itself.

  19. nosilver4u
    Member
    Plugin Author

    Posted 5 months ago #

    Ok, scratch all that. I think I've narrowed it down to a memory issue. And you don't get any errors on a memory failure. I inserted some memory stats into my local dev copy, and found this:

    When loading the Bulk Optimize (from the left-hand menu, so it will process ALL media library images), memory spikes by about 40 MB with 26,000 images in the Media Library. This could be somewhat higher if your WP install is generating more than 3-4 resizes (I've seen some that do 9 or more), and if you have other plugins utilizing the attachment metadata.

    When loading the Bulk Optimize page via the drop-down actions on the Media Library (to do smaller batches of images), memory usage barely increases at all.

    So, increasing the memory limit yet again could alleviate the issues you are seeing. I think it is worth noting that memory usage on my site with 26,000 images never went above 72 MB, so the fact that your site is consuming up to (and possibly exceeding) 512 MB would be cause to take a serious look at what plugins are eating up all that memory.

    I AM going to take a look through that portion of the code and see if there is any way to alleviate the memory spike, but I don't know if I'll be able to do much.

  20. nosilver4u
    Member
    Plugin Author

    Posted 5 months ago #

    Where there's a will, there's a way...

    I knew the memory usage was caused by loading ALL the image metadata from the database at once, but I wasn't sure if there was a way to make that better without sacrificing speed. When we were doing 1 image at a time, it took waaaay too long, and caused server timeouts (plus, it isn't very nice to your db server).

    So I decided to try a compromise, and load 1000 items at a time. It reduced the memory of usage of the function to a a very manageable amount. It doesn't actually appear to use ANY memory for that function, but I'm sure that isn't entirely accurate, perhaps it pre-allocates the memory because it knows how many results we are expecting (1000).

    The shocking part is that it actually got quicker too. I don't know how that is possible, but it cut the processing time for the 'image count' function by about 70%.

    So, if you haven't already resolved your own issues, try out the 'dev' version and see how you fare: http://downloads.wordpress.org/plugin/ewww-image-optimizer.zip

    I'm calling this case closed, as I'm almost positive this is what you were seeing problems with. Of course feel free to correct me if you still get blank screens on the bulk optimize after installing the dev version. Make sure it shows up as version 1.8.4.9, that will tell you you're using the dev release.

  21. nosilver4u
    Member
    Plugin Author

    Posted 5 months ago #

    one last thing, and then I'm done...

    Just so you are aware, if you resume a Bulk Optimize, it will not be near as quick, because it has to do images 1 at a time. On my test site, it was around 24 seconds to prep the bulk optimize for a resume. I think there are some ways to improve that also, but that'll be a project for another day.

  22. James Kraus
    Member
    Posted 5 months ago #

    Thanks Nosilver, I'll try out the Dev version and report back if it solves our issues!

  23. nosilver4u
    Member
    Plugin Author

    Posted 5 months ago #

    I was thinking about the problems you described on the drive home, and it got me to thinking that the initial problem was the memory issue, which you resolved by increasing your memory limit. I've pretty much solved the memory issues in the code as well, but the issue you had afterward wasn't making sense to me, where you described it this way:

    I stopped it, then navigated around for a bit.

    Now when I load the Bulk Optimize page I get the same blank screen, no errors. I'm pretty sure the script is just timing out. I've tried both optimizers (cloud and non-cloud) and it happens to both.

    As I'm re-reading that, it exactly describes what I saw when I did the same thing. Mine didn't time out, because I have a dedicated test box with fast hardware, and potentially fewer images (although I'm curious how many you are working with). However, it did still take significantly longer to load, because it then had to parse around 25,000 images one at a time.

    At any rate, let me know how it goes with the dev version.

  24. DLT95
    Member
    Posted 5 months ago #

    V 1.85 now imports our 10,000+ images into the table correctly ready for bulk optimising. The import takes a while but that's no problem.

Reply

You must log in to post.

About this Plugin

About this Topic