Title: Bulk Optimize Fails – Use Cloud API instead? Last modified: February 8, 2017 --- # Bulk Optimize Fails – Use Cloud API instead? * Resolved [hightechdad](https://wordpress.org/support/users/hightechdad/) * (@hightechdad) * [9 years, 3 months ago](https://wordpress.org/support/topic/bulk-optimize-fails-use-cloud-api-instead/) * I have close to 11,000 images that need to be optimized. Regular upload optimization works fine but when I do the bulk optimize, it times out and eventually tells me to increase my PHP max_execution_time. I just changed it to 180 but the bulk still times out. * Yes, lots of images, my blog is over 10 years old. * So, if I do the cloud API route, instead of local server, would that offload all of the optimization processes to the cloud? * Can I use this plugin or should I disable and shift over to the Cloud version of EWWW? (I tried having both simultaneously but got a fatal error.) * What are your recommendations on which plugin to use? * Also, I use a CDN for my images. Will I need to purge the cache after I go through a bulk optimization? * Thanks for this great plugin! I just want to get it to work! Viewing 15 replies - 1 through 15 (of 18 total) 1 [2](https://wordpress.org/support/topic/bulk-optimize-fails-use-cloud-api-instead/page/2/?output_format=md) [→](https://wordpress.org/support/topic/bulk-optimize-fails-use-cloud-api-instead/page/2/?output_format=md) * Plugin Author [nosilver4u](https://wordpress.org/support/users/nosilver4u/) * (@nosilver4u) * [9 years, 3 months ago](https://wordpress.org/support/topic/bulk-optimize-fails-use-cloud-api-instead/#post-8782583) * 11,000 images is no big deal, I routinely run the scanner on 40k on my dev box 🙂 * Try increasing the memory_limit and see if that helps with the scan issues. * Using the API does indeed offload the compression to the “cloud”, which is really a cluster of servers dedicated to compressing your images. * You can use either plugin, but if you decide to use the API, I recommend using the Cloud version, because it is much smaller, so it updates much quicker. Not that it’s too big of a deal, but I use it on most of my own sites now for that reason. * After you optimize everything, you’ll definitely want to purge your CDN cache to make sure they grab fresh copies of everything. * If the memory_limit thing doesn’t work, hit me up here: [https://ewww.io/contact-us/](https://ewww.io/contact-us/) * Thread Starter [hightechdad](https://wordpress.org/support/users/hightechdad/) * (@hightechdad) * [9 years, 3 months ago](https://wordpress.org/support/topic/bulk-optimize-fails-use-cloud-api-instead/#post-8782671) * Thanks for the quick response. I’m on DreamPress (the DreamHost WordPress service). They have a dynamic memory allocation I believe and while I can ask to up the memory allocation, they might not do it. * So, if I were to switch over to the cloud version of the plugin, should I fully delete the server version (what I have now) or just disable? * And, moving forward, once I bulk optimize using the Cloud version and get everything optized (and purged from my CDN), would I then move back to the server version( which seems to work fine for small, article-related uploads)? * Thanks! * Plugin Author [nosilver4u](https://wordpress.org/support/users/nosilver4u/) * (@nosilver4u) * [9 years, 3 months ago](https://wordpress.org/support/topic/bulk-optimize-fails-use-cloud-api-instead/#post-8782795) * You might as well delete the server version if you switch to the cloud version. As for going back to the server version later, of course I’d rather you didn’t 🙂 But the main thing is what level of compression you use. If you’re just using lossless, then it is probably fine to use the server version for small uploads here and there. But if you take advantage of the huge savings that come with Max Lossy mode, there’s really no going back, it’s pretty fantastic stuff. You can run a few test images at tinyjpg.com to try it out and see for yourself. * As for the scan issue, I’m going through some debug information from someone else, but if I strike out there, I’ll send you the instructions for further debugging on that. It really should NOT be a memory issue anymore, but I figured if you had the ability to try that real quick, it would be worth checking out, just in case. * Moderator [Ipstenu (Mika Epstein)](https://wordpress.org/support/users/ipstenu/) * (@ipstenu) * 🏳️‍🌈 Advisor and Activist * [9 years, 3 months ago](https://wordpress.org/support/topic/bulk-optimize-fails-use-cloud-api-instead/#post-8782832) * Hi, I work for DreamHost (and I’m on the DreamPress team specifically). * What memory limits do you recommend for this? PHP limits can be adjusted within reason, of course, but I know even our shared starts at 90M and DP is higher. * The max execution time is also adjustable (within reason 😉 ) but in general we don’t like raising that forever since it means EVERY process can run for a really long time, and that gets messy with some plugins. * Plugin Author [nosilver4u](https://wordpress.org/support/users/nosilver4u/) * (@nosilver4u) * [9 years, 3 months ago](https://wordpress.org/support/topic/bulk-optimize-fails-use-cloud-api-instead/#post-8782896) * Hey Mika, it depends on how large the library is, but as I told [@hightechdad](https://wordpress.org/support/users/hightechdad/), I’m not sure it really is a memory issue. I built in extensive memory checks in the last couple releases, to try and prevent EWWW from exceeding the memory limit (with buffers of 2-4MB to be safe). On my local box I was able to scan a site with 40k attachments using only 16MB of memory. * As for time limits, the scan should almost never bail because of that, as it has a cap of 15 seconds for each batch of images. Every 100 image attachments, it checks both the memory usage and the time elapsed. So it will short-circuit the scan when it detects a resource constraint, and then resumes the scan on a subsequent AJAX call. * So, on the memory limit, if you could increase his settings by about 50%, I think that would allow us to determine for sure if it is a memory issue, or something else. * Also, if you know any other tricks for getting error output, I’d appreciate it. There seems to be some odd glitch we’re hitting, and I’ve had folks enable WP_DEBUG, and check error logs to no avail. I’ve resorted to adding extra logging calls within the scan functions to try and pin it down, but haven’t quite nailed it yet (and hopefully we aren’t still dealing with multiple issues causing the same 500 server error). * Thread Starter [hightechdad](https://wordpress.org/support/users/hightechdad/) * (@hightechdad) * [9 years, 3 months ago](https://wordpress.org/support/topic/bulk-optimize-fails-use-cloud-api-instead/#post-8782976) * Awesome to have Mika on this thread as well. I just switched over from Server to Cloud version (purchased APIs). * First, I didn’t delete the Server version, I just disabled it. When activating the Cloud version, much of my same settings were active (should I have deleted Server version?). Second, I believe DreamHost/Press updated my max_execution_time. I will let [@ipstenu](https://wordpress.org/support/users/ipstenu/) address the memory allocation. Third, just tried running the bulk optimize. It’s still hanging(“ did not complete, will try again”). * So, should I fully delete both plugins and start again from scratch? Should I try a “force reoptimize”? * Looking forward to getting this working! * Plugin Author [nosilver4u](https://wordpress.org/support/users/nosilver4u/) * (@nosilver4u) * [9 years, 3 months ago](https://wordpress.org/support/topic/bulk-optimize-fails-use-cloud-api-instead/#post-8783019) * The two plugins share settings, that shouldn’t really be a problem, and it certainly doesn’t affect the scan either way. I’ve been helping 4 different people with the same problem, so I apologize if I repeat things, but can you let the scan run until it stops trying (or succeeds)? * Then once Mika bumps the memory, you can try it again just to see what happens. Some failures are not entirely unexpected with a library that size. It can happen if an image has corrupted/missing metadata, and EWWW will try to rebuild it, which has the side effect of regenerating thumbs, and potentially triggering whatever broke the metadata in the first place. When that happens, EWWW will throw an error, but it will reattempt the scan, and mark that attachment as ‘ bad’ for future scan attempts. * If that (or something else) kills the scan 10 times in a row, then it will stop trying, but you’ll get a different error message when that happens. * Thread Starter [hightechdad](https://wordpress.org/support/users/hightechdad/) * (@hightechdad) * [9 years, 3 months ago](https://wordpress.org/support/topic/bulk-optimize-fails-use-cloud-api-instead/#post-8783036) * Thanks. I let it run and got “Operation timed out, you may need to increase the max_execution_time for PHP” (which was already done). It’s set at 180 secs. * My memory_limit is 512MB currently * So, I can perhaps open a ticket with DreamHost (or maybe [@ipstenu](https://wordpress.org/support/users/ipstenu/) can just do it for me – grin). * Moderator [Ipstenu (Mika Epstein)](https://wordpress.org/support/users/ipstenu/) * (@ipstenu) * 🏳️‍🌈 Advisor and Activist * [9 years, 3 months ago](https://wordpress.org/support/topic/bulk-optimize-fails-use-cloud-api-instead/#post-8783041) * If 512 isn’t enough, then … yikes. [@hightechdad](https://wordpress.org/support/users/hightechdad/), what’s your domain? I can go look and see if it’s actually something else. Like our process watcher may be drunk again… * Plugin Author [nosilver4u](https://wordpress.org/support/users/nosilver4u/) * (@nosilver4u) * [9 years, 3 months ago](https://wordpress.org/support/topic/bulk-optimize-fails-use-cloud-api-instead/#post-8783054) * Yeah, 512 should be plenty. I can run the scan full-tilt locally with 256M and 40k attachments, so it’s got to be something else. * Thread Starter [hightechdad](https://wordpress.org/support/users/hightechdad/) * (@hightechdad) * [9 years, 3 months ago](https://wordpress.org/support/topic/bulk-optimize-fails-use-cloud-api-instead/#post-8783058) * Thanks [@ipstenu](https://wordpress.org/support/users/ipstenu/) – yeah, well it could be the 20 million plugins I run. The site is [http://www.hightechdad.com](http://www.hightechdad.com). Site is old (but current) and has mushroomed over time. 60-70k visitors/month. Lots of ads, plugins, images, you name it. It’s a bit of a beast so I have been trying to do some optimization – CDN, CloudFlare, minification, optimization, etc. Any help/advice is appreciated! * Thread Starter [hightechdad](https://wordpress.org/support/users/hightechdad/) * (@hightechdad) * [9 years, 3 months ago](https://wordpress.org/support/topic/bulk-optimize-fails-use-cloud-api-instead/#post-8783301) * Ok, I have another thought for [@ipstenu](https://wordpress.org/support/users/ipstenu/). I was looking at my error logs and there are quite a few instances of paired errors (different IPs) related to ModSecurity and eWWW. * Here are a couple: * `Feb 08 12:38:24 2017] [error] [client OneIP] Premature end of script headers: admin-ajax.php, referer: [https://www.hightechdad.com/wp-admin/upload.php?page=ewww-image-optimizer-bulk](https://www.hightechdad.com/wp-admin/upload.php?page=ewww-image-optimizer-bulk) [ Wed Feb 08 12:42:20 2017] [error] [client TwoIP] ModSecurity: Access denied with code 418 (phase 2). Matched phrase “WQGP” at ARGS:rmfilter. [file “/dh/apache2/ template/etc/mod_sec2/99_dreamhost_rules.conf”] [line “151”] [id “1990070”] [ msg “Common known arguments for backdoor shell present in ARGS:rmfilter”] [hostname“ www.hightechdad.com”] [uri “/wp-load.php”] [unique_id “WJuCrNBhke4AABs3GDAAAAAF”]` * So, I’m wondering if the DreamPress firewall is viewing the upload.php call to the eWWW cloud as something as a security issue and is preventing the execution. I could be way off base here but I thought I would float it out there. * Moderator [Ipstenu (Mika Epstein)](https://wordpress.org/support/users/ipstenu/) * (@ipstenu) * 🏳️‍🌈 Advisor and Activist * [9 years, 3 months ago](https://wordpress.org/support/topic/bulk-optimize-fails-use-cloud-api-instead/#post-8783318) * It’s only 86 plugins 😀 * So here’s your error! * `Premature end of script headers: admin-ajax.php, referer: https://www.hightechdad. com/wp-admin/upload.php?page=ewww-image-optimizer-bulk` * Technically yes, it’s PHP running out of spoons to run the process. But in this case it’s not memory. I mean, it’s not the 512M. I think you’re right that it’s the 86 plugins. * Edited to add: The reason you see modsec at the same time is that the failure it causing a weird thing to pass to modsec so it ‘looks’ like a hack * Looking at them, I would disable (for the purposes of this process only!) * * all-in-one-wp-security-and-firewall * blogvault-real-time-backup * regenerate- thumbnails * wp-dbmanager * wp-serverinfo * And you probably want to remove these: * * x-custom-404 (this is active!) * x-disqus-comments * x-email-mailchimp (this is active!) * x-smooth-scroll * x-typekit * Unless those are all special plugins for your X theme. * I saw you told WP to use 512M but not PHP, so I changed that for you and bounced Apache to clear up some hung processes. That MAY be enough? I hope… Otherwise this may be a ‘Turn off ALL the plugins’ because while I’m not an advocate of the ‘ZOMG! too many plugins!’ I’m familiar with some of those plugins and they can be a little ‘heavy’ to run. - This reply was modified 9 years, 3 months ago by [Ipstenu (Mika Epstein)](https://wordpress.org/support/users/ipstenu/). * Plugin Author [nosilver4u](https://wordpress.org/support/users/nosilver4u/) * (@nosilver4u) * [9 years, 3 months ago](https://wordpress.org/support/topic/bulk-optimize-fails-use-cloud-api-instead/#post-8783326) * That could indeed be it right there. I’m not quite sure I understand what it is saying though. [@ipstenu](https://wordpress.org/support/users/ipstenu/), if you’d prefer to explain it via private email, I believe you have my address, otherwise [https://ewww.io/contact-us/](https://ewww.io/contact-us/) so we can figure out how to avoid tripping that rule inadvertently. * Moderator [Ipstenu (Mika Epstein)](https://wordpress.org/support/users/ipstenu/) * (@ipstenu) * 🏳️‍🌈 Advisor and Activist * [9 years, 3 months ago](https://wordpress.org/support/topic/bulk-optimize-fails-use-cloud-api-instead/#post-8783332) * We posted at the same time 🙂 * So you can test by disabling it in panel: [https://panel.dreamhost.com/index.cgi?tree=domain.manage&](https://panel.dreamhost.com/index.cgi?tree=domain.manage&); * Click on the edit and turn off ‘Extra Web Protection.’ * But I don’t think this is the case seeing as we’re getting “Premature end of script headers” at the same time. If it was Mod Security, you’d get a 418 error. Viewing 15 replies - 1 through 15 (of 18 total) 1 [2](https://wordpress.org/support/topic/bulk-optimize-fails-use-cloud-api-instead/page/2/?output_format=md) [→](https://wordpress.org/support/topic/bulk-optimize-fails-use-cloud-api-instead/page/2/?output_format=md) The topic ‘Bulk Optimize Fails – Use Cloud API instead?’ is closed to new replies. * ![](https://ps.w.org/ewww-image-optimizer/assets/icon-256x256.png?rev=1582276) * [EWWW Image Optimizer](https://wordpress.org/plugins/ewww-image-optimizer/) * [Frequently Asked Questions](https://wordpress.org/plugins/ewww-image-optimizer/#faq) * [Support Threads](https://wordpress.org/support/plugin/ewww-image-optimizer/) * [Active Topics](https://wordpress.org/support/plugin/ewww-image-optimizer/active/) * [Unresolved Topics](https://wordpress.org/support/plugin/ewww-image-optimizer/unresolved/) * [Reviews](https://wordpress.org/support/plugin/ewww-image-optimizer/reviews/) * 18 replies * 3 participants * Last reply from: [Ipstenu (Mika Epstein)](https://wordpress.org/support/users/ipstenu/) * Last activity: [9 years, 3 months ago](https://wordpress.org/support/topic/bulk-optimize-fails-use-cloud-api-instead/page/2/#post-8783439) * Status: resolved