Support » Plugin: EWWW Image Optimizer » [Plugin: EWWW Image Optimizer] Optimizing 12-30,000 Images

  • I noticed there was a similar thread asking about doing XX images at a time, though there was no final response to the inquiry, so I thought I’d start a new thread before killing the server on a multi-thousand image run :-).

    Is the plugin capable of pausing between runs, or does it try to optimize all of the images in one shot? If the latter, would it be possible to implement a pause? For those of us that have clients with 20-50GB’s of images, trying to run this in a one-shot deal would cripple the server or lead to a time-out due to the sheer number of images being processed.

    Ideally, it’d be nice to see a setting where we can choose how long to pause and how many images are included in each run. For the 12-30,000 mentioned, there’s no way a 10 second pause would do as some of the images are rather large in comparison to your everyday featured image uploads (each post consists of 4-10 images, all high-quality).

    Thanks in advance!

Viewing 3 replies - 1 through 3 (of 3 total)
  • Plugin Author nosilver4u


    Lots of questions there, so hopefully I don’t miss anything…

    It sounds like you’re looking for an auto-sleep function where it does 10 images, sleeps, does 10 more, sleeps, does 10 more, sleeps, etc. I don’t imagine this would be difficult to implement, but I’ve generally tried to avoid making the bulk process take longer than necessary. The resume capability has made that less of an issue, so I’m open to revisiting this again.

    That all said, there is a way to do this manually that you may not be aware of:
    You can start the bulk optimize, then hit the stop button in your browser. Then go into the bulk optimize again, and it will now give you a resume button. rinse, repeat…

    Additionally, there are a couple things on the performance side to clarify:
    1. If you are doing high-quality images, I assume you are working mostly with jpg images, and those process much quicker than png images, although with the quantity you are looking at it is still going to take a while. With a very large 18 MB jpg on my very under-powered test server, it only takes 13 seconds. A smaller 4 MB file took around 3 seconds. You don’t need to worry about timeouts as much (because of the ‘resume’ capability), but that leads me to the issue you mention with crippling/overloading the server.
    2. None of the utilities used are capable of multi-threading, so worse-case scenario, you are looking at monopolizing a single core for the duration, and your web server should keep happily chirping along. Of course, if it doesn’t, you can hit the stop button like I mentioned and let it catch up for a bit.

    If you STILL feel the need for an auto-sleep function, let me know, and I will schedule it for inclusion in the upcoming 1.2 release (looking to push that out this weekend or early next week). You should also be aware that adding such a delay will involve an increased risk of timeouts. I believe most of the issue is on the apache side, but php also has timeout limits that should be increased should you choose to go this route.

    Thanks for the quick response!

    The biggest issue that I’ve seen with that many images is the consistent hit the CPU takes due to non-existent multi-threading capabilities (which has nothing to do with the plugin). On a slightly busy site, taxing the CPU for hours while processing that many images would be a huge drain of resources, especially for those on a VPS with limited resources.

    With most VPS’s only providing 2-4 CPU cores, which are often shared among other users on the host node, using a full core + the remaining cores for traffic would likely lead to their account being suspended, or the user receiving a rather nasty e-mail from their provider (of course, worst case scenario, though 30-50GB is a huge chunk :)).

    If we had something such as:

    1). The ability to set the time between each subsequent run. Most providers allow 30-60 seconds for a PHP script to either finish or terminate and on custom solutions, this limit could be increased, so I don’t think time-outs would be an issue as long as the end-user kept within those ranges. Of course, with Apache there are limitations too, though even at a max of 30 seconds between runs, that’d allow a “cool-down” period which would prevent over-taxing the CPU. On a dedicated server, this wouldn’t be that much of an issue as more than likely you’d have plenty of resources to go around :-).

    2). The ability to pick up where the last run left off in the event of a time-out (which would require a log log to be maintained I would imagine – though a CRON could be set to clean/delete this log every XX day/week/month).

    There’s no major rush to get either implemented right now as we’re just testing the waters, though so far, the plugin works perfectly. The above would simply improve upon an already functioning concept and allow for a little more control :-).

    Thanks again!

    Plugin Author nosilver4u


    I’m going to tackle these backwards…

    #2 has been a highly requested feature, and is already implemented, which is why I suggested using it to implement a sort of manual pause & resume routine, but perhaps I wasn’t clear about that.

    Regarding #1, I need to be clear that without using javascript and almost completely rewriting the bulk optimize function (for what you have describe as a ‘little more control’), it could still timeout as soon as it hits the php, apache, or webhost restrictions on your server. Which is why I’ve implemented the ‘resume’ function as I mentioned. It is technically impossible to guarantee that the Bulk Optimize function will go on forever unless you use javascript, and even then you have Chrome & Firefox nagging you about a ‘long running script’.

    In my pondering here, I thought of one other thing that could help avoid nasty messages from your webhost. It would involve using the ‘nice’ command to make the tools play nice with other processes, basically setting them at the lowest priority possible so that other users on the same server aren’t affected by it (as much).

    So, if you would STILL like me to try and implement the method using php, I can do that without too much trouble within the next week or so. If you want a javascript solution, you’ll have to find someone to contribute the code.

Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘[Plugin: EWWW Image Optimizer] Optimizing 12-30,000 Images’ is closed to new replies.