adamh007
Forum Replies Created
-
Forum: Plugins
In reply to: [WP Super Cache] Cache rebuild does not work – nginx preload modeIt’s not as bad as you think it is
Following up on this, @donncha , based on the chart below, try to guess when I fixed the bugs, that were “not as bad”
Forum: Plugins
In reply to: [WP Super Cache] Cache rebuild does not work – nginx preload modeIt’s not as bad as you think it is
It’s actually much worse than you think it is. It is severely bugged, leading to massive spikes in response times. Look at the chart below. This is your bug, refusing to fix, causing spikes up to 4 seconds (!) everywhere around the world.
EDIT: images are bugged out in the post, here is a link to them: https://imgur.com/a/V0s2vCF
And another bug..
I found another bug, terrifyingly. GC runs anyway, regardless of preload mode. This is likely due to setting cache_max_time somewhere to preload mode time.
My settings:
My log:
wp_cron_preload_cache: doing taxonomy preload.
15:57:10 75004 /wp-cron.php?doing_wp_cron=1732895829.8098969459533691406250 wp_cron_preload_cache: no more posts to get. Limit (58) reached.
15:57:10 75004 /wp-cron.php?doing_wp_cron=1732895829.8098969459533691406250 wp_cron_preload_cache: no more posts. scheduling next preload in 600 minutes.
15:57:10 75004 /wp-cron.php?doing_wp_cron=1732895829.8098969459533691406250 wp_cron_preload_cache: clean expired cache files older than 36000 seconds.This is clearly not what the code was intended to do in wp-cache.php:
if ( $cache_max_time > 0 ) { // GC is NOT disabled
wp_cache_debug( "wp_cron_preload_cache: clean expired cache files older than $cache_max_time seconds.", 5 );
wp_cache_phase2_clean_expired( $file_prefix, true ); // force cleanup of old files.
}But in the same file you made a mistake and set cache_max_time to nonzero:
$cache_max_time = (int)$wp_cache_preload_interval * 60; // fool the GC into expiring really old filesI understand you are hesitant to touch these files, contaning over 4k lines each*, but this is so buggy and dysfunctional that I find your position indefensible.
*how did you even get to this point? Methods several pages long, files thousands of lines long. I understand it grew organically, but this is not a hobby project, it’s a for-profit organization.
The fixes you’d need to implement
- Don’t delete home index after every post-batch – the line I mentioned in my previous post can be safely deleted, no reason to wipe out home index after a preload run, it’s clearly a bug
- Don’t set cache_max_time to preload time – the code already caters for disabled GC – it’s also clearly a bug
If in doubt, again, please do look at the chart above – your cache plugin causing massive spikes, despite every setting on the plugin indicating this would never happen.
- This reply was modified 1 year, 6 months ago by adamh007.
Forum: Plugins
In reply to: [WP Super Cache] Cache rebuild does not work – nginx preload modedigging into the code further, @donncha it seems like the bug is due to a call to
wpsc_delete_files( get_supercache_dir() );
This happens after every batch of 10 posts is processed (WPSC_PRELOAD_POST_COUNT), and the preload reschedules itself. However, incorrectly, deletes the root folder after every batch.
Making the matter worse, wp-cron will not fire unless there is a visitor to the side, so the preload process just hangs, waiting for someone to hit the uncached home page – then the preload cron will slow down generating the cachefile, adding extra delay to serving the home index.
This seems a rather big problem in wpsc, not sure how this has never been noticed before 🤔
Forum: Plugins
In reply to: [WP Super Cache] Cache rebuild does not work – nginx preload modehi @donncha thanks for your patience working through this. I enabled debug logging, which pointed to a different(?) problem, namely the home index file gets trashed several times during preload (which I might have perceived first as files are getting timed-out despite preload).
Given the preload takes several minutes with reasonable CPU load on the server, and the index gets deleted several, users get a huge spike in response time when trying to access the non-existent home during preload.
I checked the source, and it does indeed call an actual deletion, no further checks or skips: wp-super-cache/wp-cache-phase2.php at 961ab356684208c40065464ea0ac331ac1f9f829 · Automattic/wp-super-cache
Here are the snippets from the debug log, showing the repeated deletion of root file. I’m hoping I stripped all sensitive data from it…
The wp-cache-phase2 file is 3.5k lines long -eeeek- so I don’t pretend to fully understand yet why this behaviour exists. However, this does not make sense, neither on face value, nor thinking about it more – could you please confirm this is a bug?
17:01:40 66527 /wp-cron.php?doing_wp_cron=1732813277.6925950050354003906250 wp_cron_preload_cache: fetched https://myexample.com/blog/currentpage1
17:01:41 66527 /wp-cron.php?doing_wp_cron=1732813277.6925950050354003906250 wp_cron_preload_cache: scheduling the next preload in 3 seconds.
17:01:41 66527 /wp-cron.php?doing_wp_cron=1732813277.6925950050354003906250 wpsc_delete_files: deleting ABSPATH/wp-content/cache/supercache/myexample.com/
17:01:41 66527 /wp-cron.php?doing_wp_cron=1732813277.6925950050354003906250 wpsc_delete_files: reading files: pagesample
17:01:41 66527 /wp-cron.php?doing_wp_cron=1732813277.6925950050354003906250 wpsc_delete_files: reading files: about
17:01:41 66527 /wp-cron.php?doing_wp_cron=1732813277.6925950050354003906250 wpsc_delete_files: reading files: faq
17:01:41 66527 /wp-cron.php?doing_wp_cron=1732813277.6925950050354003906250 wpsc_delete_files: reading files: sitemap_index.xml
17:01:41 66527 /wp-cron.php?doing_wp_cron=1732813277.6925950050354003906250 wpsc_delete_files: reading files: feedbackpageisgoodforyou
17:01:41 66527 /wp-cron.php?doing_wp_cron=1732813277.6925950050354003906250 wpsc_delete_files: reading files: terms
17:01:41 66527 /wp-cron.php?doing_wp_cron=1732813277.6925950050354003906250 wpsc_delete_files: reading files: pagesample2
17:01:41 66527 /wp-cron.php?doing_wp_cron=1732813277.6925950050354003906250 wpsc_delete_files: reading files: services
17:01:41 66527 /wp-cron.php?doing_wp_cron=1732813277.6925950050354003906250 wpsc_delete_files: reading files: index-https.html
17:01:41 66527 /wp-cron.php?doing_wp_cron=1732813277.6925950050354003906250 wpsc_delete_files: deleting ABSPATH/wp-content/cache/supercache/myexample.com/index-https.html
17:01:41 66527 /wp-cron.php?doing_wp_cron=1732813277.6925950050354003906250 wpsc_delete_files: reading files: resources
17:01:41 66527 /wp-cron.php?doing_wp_cron=1732813277.6925950050354003906250 wpsc_delete_files: reading files: anotherpage
17:01:41 66527 /wp-cron.php?doing_wp_cron=1732813277.6925950050354003906250 wpsc_delete_files: reading files: ..
17:01:41 66527 /wp-cron.php?doing_wp_cron=1732813277.6925950050354003906250 wpsc_delete_files: reading files: blog
17:01:41 66527 /wp-cron.php?doing_wp_cron=1732813277.6925950050354003906250 wpsc_delete_files: reading files: .
17:01:41 66527 /wp-cron.php?doing_wp_cron=1732813277.6925950050354003906250 wpsc_delete_files: reading files: contact
17:01:41 66527 /wp-cron.php?doing_wp_cron=1732813277.6925950050354003906250 wpsc_delete_files: reading files: somanypages
17:01:41 66527 /wp-cron.php?doing_wp_cron=1732813277.6925950050354003906250 wpsc_delete_files: remove directory ABSPATH/wp-content/cache/supercache/myexample.com/
…
17:06:44 66390 /wp-cron.php?doing_wp_cron=1732813590.5101430416107177734375 wp_cron_preload_cache: fetched https://myexample.com/terms
17:06:45 66390 /wp-cron.php?doing_wp_cron=1732813590.5101430416107177734375 wp_cron_preload_cache: scheduling the next preload in 3 seconds.
17:06:45 66390 /wp-cron.php?doing_wp_cron=1732813590.5101430416107177734375 wpsc_delete_files: deleting ABSPATH/wp-content/cache/supercache/myexample.com/
17:06:45 66390 /wp-cron.php?doing_wp_cron=1732813590.5101430416107177734375 wpsc_delete_files: reading files: pagesample
17:06:45 66390 /wp-cron.php?doing_wp_cron=1732813590.5101430416107177734375 wpsc_delete_files: reading files: about
17:06:45 66390 /wp-cron.php?doing_wp_cron=1732813590.5101430416107177734375 wpsc_delete_files: reading files: faq
17:06:45 66390 /wp-cron.php?doing_wp_cron=1732813590.5101430416107177734375 wpsc_delete_files: reading files: sitemap_index.xml
17:06:45 66390 /wp-cron.php?doing_wp_cron=1732813590.5101430416107177734375 wpsc_delete_files: reading files: feedbackpageisgoodforyou
17:06:45 66390 /wp-cron.php?doing_wp_cron=1732813590.5101430416107177734375 wpsc_delete_files: reading files: terms
17:06:45 66390 /wp-cron.php?doing_wp_cron=1732813590.5101430416107177734375 wpsc_delete_files: reading files: pagesample2
17:06:45 66390 /wp-cron.php?doing_wp_cron=1732813590.5101430416107177734375 wpsc_delete_files: reading files: services
17:06:45 66390 /wp-cron.php?doing_wp_cron=1732813590.5101430416107177734375 wpsc_delete_files: reading files: index-https.html
17:06:45 66390 /wp-cron.php?doing_wp_cron=1732813590.5101430416107177734375 wpsc_delete_files: deleting ABSPATH/wp-content/cache/supercache/myexample.com/index-https.html
17:06:45 66390 /wp-cron.php?doing_wp_cron=1732813590.5101430416107177734375 wpsc_delete_files: reading files: resources
17:06:45 66390 /wp-cron.php?doing_wp_cron=1732813590.5101430416107177734375 wpsc_delete_files: reading files: anotherpage
17:06:45 66390 /wp-cron.php?doing_wp_cron=1732813590.5101430416107177734375 wpsc_delete_files: reading files: ..
17:06:45 66390 /wp-cron.php?doing_wp_cron=1732813590.5101430416107177734375 wpsc_delete_files: reading files: blog
17:06:45 66390 /wp-cron.php?doing_wp_cron=1732813590.5101430416107177734375 wpsc_delete_files: reading files: .
17:06:45 66390 /wp-cron.php?doing_wp_cron=1732813590.5101430416107177734375 wpsc_delete_files: reading files: contact
17:06:45 66390 /wp-cron.php?doing_wp_cron=1732813590.5101430416107177734375 wpsc_delete_files: reading files: somanypages
17:06:45 66390 /wp-cron.php?doing_wp_cron=1732813590.5101430416107177734375 wpsc_delete_files: remove directory ABSPATH/wp-content/cache/supercache/myexample.com/
…
17:09:28 67725 /wp-cron.php?doing_wp_cron=1732813753.8266520500183105468750 wp_cron_preload_cache: fetched https://myexample.com/blog/currentpage2
17:09:29 67725 /wp-cron.php?doing_wp_cron=1732813753.8266520500183105468750 wp_cron_preload_cache: scheduling the next preload in 3 seconds.
17:09:29 67725 /wp-cron.php?doing_wp_cron=1732813753.8266520500183105468750 wpsc_delete_files: deleting ABSPATH/wp-content/cache/supercache/myexample.com/
17:09:29 67725 /wp-cron.php?doing_wp_cron=1732813753.8266520500183105468750 wpsc_delete_files: reading files: pagesample
17:09:29 67725 /wp-cron.php?doing_wp_cron=1732813753.8266520500183105468750 wpsc_delete_files: reading files: about
17:09:29 67725 /wp-cron.php?doing_wp_cron=1732813753.8266520500183105468750 wpsc_delete_files: reading files: faq
17:09:29 67725 /wp-cron.php?doing_wp_cron=1732813753.8266520500183105468750 wpsc_delete_files: reading files: sitemap_index.xml
17:09:29 67725 /wp-cron.php?doing_wp_cron=1732813753.8266520500183105468750 wpsc_delete_files: reading files: feedbackpageisgoodforyou
17:09:29 67725 /wp-cron.php?doing_wp_cron=1732813753.8266520500183105468750 wpsc_delete_files: reading files: terms
17:09:29 67725 /wp-cron.php?doing_wp_cron=1732813753.8266520500183105468750 wpsc_delete_files: reading files: pagesample2
17:09:29 67725 /wp-cron.php?doing_wp_cron=1732813753.8266520500183105468750 wpsc_delete_files: reading files: services
17:09:29 67725 /wp-cron.php?doing_wp_cron=1732813753.8266520500183105468750 wpsc_delete_files: reading files: index-https.html
17:09:29 67725 /wp-cron.php?doing_wp_cron=1732813753.8266520500183105468750 wpsc_delete_files: deleting ABSPATH/wp-content/cache/supercache/myexample.com/index-https.html
17:09:29 67725 /wp-cron.php?doing_wp_cron=1732813753.8266520500183105468750 wpsc_delete_files: reading files: resources
17:09:29 67725 /wp-cron.php?doing_wp_cron=1732813753.8266520500183105468750 wpsc_delete_files: reading files: anotherpage
17:09:29 67725 /wp-cron.php?doing_wp_cron=1732813753.8266520500183105468750 wpsc_delete_files: reading files: ..
17:09:29 67725 /wp-cron.php?doing_wp_cron=1732813753.8266520500183105468750 wpsc_delete_files: reading files: blog
17:09:29 67725 /wp-cron.php?doing_wp_cron=1732813753.8266520500183105468750 wpsc_delete_files: reading files: .
17:09:29 67725 /wp-cron.php?doing_wp_cron=1732813753.8266520500183105468750 wpsc_delete_files: reading files: contact
17:09:29 67725 /wp-cron.php?doing_wp_cron=1732813753.8266520500183105468750 wpsc_delete_files: reading files: somanypages
17:09:29 67725 /wp-cron.php?doing_wp_cron=1732813753.8266520500183105468750 wpsc_delete_files: remove directory ABSPATH/wp-content/cache/supercache/myexample.com/
…
17:11:10 66527 /wp-cron.php?doing_wp_cron=1732813855.2376370429992675781250 wp_cron_preload_cache: fetched https://myexample.com/services/unicornpetting
17:11:11 66527 /wp-cron.php?doing_wp_cron=1732813855.2376370429992675781250 wp_cron_preload_cache: scheduling the next preload in 3 seconds.
17:11:11 66527 /wp-cron.php?doing_wp_cron=1732813855.2376370429992675781250 wpsc_delete_files: deleting ABSPATH/wp-content/cache/supercache/myexample.com/
17:11:11 66527 /wp-cron.php?doing_wp_cron=1732813855.2376370429992675781250 wpsc_delete_files: reading files: pagesample
17:11:11 66527 /wp-cron.php?doing_wp_cron=1732813855.2376370429992675781250 wpsc_delete_files: reading files: about
17:11:11 66527 /wp-cron.php?doing_wp_cron=1732813855.2376370429992675781250 wpsc_delete_files: reading files: faq
17:11:11 66527 /wp-cron.php?doing_wp_cron=1732813855.2376370429992675781250 wpsc_delete_files: reading files: sitemap_index.xml
17:11:11 66527 /wp-cron.php?doing_wp_cron=1732813855.2376370429992675781250 wpsc_delete_files: reading files: feedbackpageisgoodforyou
17:11:11 66527 /wp-cron.php?doing_wp_cron=1732813855.2376370429992675781250 wpsc_delete_files: reading files: terms
17:11:11 66527 /wp-cron.php?doing_wp_cron=1732813855.2376370429992675781250 wpsc_delete_files: reading files: pagesample2
17:11:11 66527 /wp-cron.php?doing_wp_cron=1732813855.2376370429992675781250 wpsc_delete_files: reading files: services
17:11:11 66527 /wp-cron.php?doing_wp_cron=1732813855.2376370429992675781250 wpsc_delete_files: reading files: index-https.html
17:11:11 66527 /wp-cron.php?doing_wp_cron=1732813855.2376370429992675781250 wpsc_delete_files: deleting ABSPATH/wp-content/cache/supercache/myexample.com/index-https.html
17:11:11 66527 /wp-cron.php?doing_wp_cron=1732813855.2376370429992675781250 wpsc_delete_files: reading files: resources
17:11:11 66527 /wp-cron.php?doing_wp_cron=1732813855.2376370429992675781250 wpsc_delete_files: reading files: anotherpage
17:11:11 66527 /wp-cron.php?doing_wp_cron=1732813855.2376370429992675781250 wpsc_delete_files: reading files: ..
17:11:11 66527 /wp-cron.php?doing_wp_cron=1732813855.2376370429992675781250 wpsc_delete_files: reading files: blog
17:11:11 66527 /wp-cron.php?doing_wp_cron=1732813855.2376370429992675781250 wpsc_delete_files: reading files: .
17:11:11 66527 /wp-cron.php?doing_wp_cron=1732813855.2376370429992675781250 wpsc_delete_files: reading files: contact
17:11:11 66527 /wp-cron.php?doing_wp_cron=1732813855.2376370429992675781250 wpsc_delete_files: reading files: somanypages
17:11:11 66527 /wp-cron.php?doing_wp_cron=1732813855.2376370429992675781250 wpsc_delete_files: remove directory ABSPATH/wp-content/cache/supercache/myexample.com/
…
17:13:12 67962 /wp-cron.php?doing_wp_cron=1732813983.6998479366302490234375 wp_cron_preload_cache: fetched https://myexample.com/about
17:13:13 67962 /wp-cron.php?doing_wp_cron=1732813983.6998479366302490234375 wp_cron_preload_cache: scheduling the next preload in 3 seconds.
17:13:13 67962 /wp-cron.php?doing_wp_cron=1732813983.6998479366302490234375 wpsc_delete_files: deleting ABSPATH/wp-content/cache/supercache/myexample.com/
17:13:13 67962 /wp-cron.php?doing_wp_cron=1732813983.6998479366302490234375 wpsc_delete_files: reading files: pagesample
17:13:13 67962 /wp-cron.php?doing_wp_cron=1732813983.6998479366302490234375 wpsc_delete_files: reading files: about
17:13:13 67962 /wp-cron.php?doing_wp_cron=1732813983.6998479366302490234375 wpsc_delete_files: reading files: faq
17:13:13 67962 /wp-cron.php?doing_wp_cron=1732813983.6998479366302490234375 wpsc_delete_files: reading files: sitemap_index.xml
17:13:13 67962 /wp-cron.php?doing_wp_cron=1732813983.6998479366302490234375 wpsc_delete_files: reading files: feedbackpageisgoodforyou
17:13:13 67962 /wp-cron.php?doing_wp_cron=1732813983.6998479366302490234375 wpsc_delete_files: reading files: terms
17:13:13 67962 /wp-cron.php?doing_wp_cron=1732813983.6998479366302490234375 wpsc_delete_files: reading files: pagesample2
17:13:13 67962 /wp-cron.php?doing_wp_cron=1732813983.6998479366302490234375 wpsc_delete_files: reading files: services
17:13:13 67962 /wp-cron.php?doing_wp_cron=1732813983.6998479366302490234375 wpsc_delete_files: reading files: index-https.html
17:13:13 67962 /wp-cron.php?doing_wp_cron=1732813983.6998479366302490234375 wpsc_delete_files: deleting ABSPATH/wp-content/cache/supercache/myexample.com/index-https.html
17:13:13 67962 /wp-cron.php?doing_wp_cron=1732813983.6998479366302490234375 wpsc_delete_files: reading files: resources
17:13:13 67962 /wp-cron.php?doing_wp_cron=1732813983.6998479366302490234375 wpsc_delete_files: reading files: anotherpage
17:13:13 67962 /wp-cron.php?doing_wp_cron=1732813983.6998479366302490234375 wpsc_delete_files: reading files: ..
17:13:13 67962 /wp-cron.php?doing_wp_cron=1732813983.6998479366302490234375 wpsc_delete_files: reading files: blog
17:13:13 67962 /wp-cron.php?doing_wp_cron=1732813983.6998479366302490234375 wpsc_delete_files: reading files: .
17:13:13 67962 /wp-cron.php?doing_wp_cron=1732813983.6998479366302490234375 wpsc_delete_files: reading files: contact
17:13:13 67962 /wp-cron.php?doing_wp_cron=1732813983.6998479366302490234375 wpsc_delete_files: reading files: somanypages
17:13:13 67962 /wp-cron.php?doing_wp_cron=1732813983.6998479366302490234375 wpsc_delete_files: remove directory ABSPATH/wp-content/cache/supercache/myexample.com/Forum: Plugins
In reply to: [WP Super Cache] Cache rebuild does not work – nginx preload modeHi @donncha thanks for checking this and responding. What you say makes sense, but that’s not what the plugin is telling me:
Cache rebuild. Serve a supercache file to anonymous users while a new file is being generated. (Recommended)
From this most people would assume, including me, that a stale file can be served while the new one is being generated. I can see based on your answer this is not the case, but this is not trivial and caused a lot of unexpected delays in responses for us.
We since then increased the cache timeout below to the pregen time, but I would really expect the “Cache rebuild” to work as it is stated.
I’m wondering, under what circumstances is it better not to serve a stale file when I asked to “Serve a supercache […] while a new file is being generated”?
Alternatively, what’s the point of async pregen every x hours if the cache timeout is less than x hours and all of the pregen files would be thrown away?
I’m hoping you can see the two timeouts and the checkbox creating a very non-intuitive situation, where the server is not serving cached files, even though I asked it to do so, and I even run a periodic pregen process.
Thank you again for looking into this!
I haven’t installed / reviewed the code, but this approach is fine.
I’m not sure I understand your concern re slowing down the plugin, as the performance of the settings page is largely irrelevant, and after the images are converted the plugin doesn’t actually do anything (apart from redirecting the GET to the webp version based on header).
One thing I’d like to ask for though is to display what values the plugin is currently using on the slider – debugging WP is super awkward and functions.php is sometimes overwritten by themes (without telling me so). If you don’t display what values the plugin is using, I can never be sure if my settings were taken into account.
If you display the values on the slider, it would help users to verify their settings are working as expected.
Cheers
ps.: if this goes live, please document this on the plugin’s page – plugins without docs are so hard to work with 🙂
- This reply was modified 2 years, 1 month ago by adamh007.
I appreciate and agree with all of the above, but our conclusions are different: allowing to override the suggested defaults is important.
What solution do you propose
Complete solution
On the advanced tab, add a manual quality setting (numerical input box).
If you enter a value there, it would override the selector on the first tab, and the slider would display a “Custom” tab on the leftmost position.
The “Custom” tab is important as in other places (e.g., Media Library conversion dropdown) currently displays the same values as the slider. If someone enters a manual value in the advanced tab, these dropdowns would show a “Custom” item as well.
If not too hard work, the slider and the dropdown label would include the actual quality settings. This would help both current and future more technical people to understand what is “Lossy” (75), and what “Custom (50)” is when they entered the value in the Advanced tab.
Complete solution v2
As above, but add an “Advanced” checkbox next to the slider on the settings page, which turns it into a 10-99 numerical slider for anyone to set any values easily.
MVP
Add a num input on the advanced tab, which would override the slider value directly in the stored settings. This would work for batch convert and upload, but would not work for individual images in the media library.
- This reply was modified 2 years, 1 month ago by adamh007.
I don’t want to sound ungrateful, but I’m a little bit puzzled by your response.
- I did not say they do not lose quality – by definition, they do. It just doesn’t matter to the average visitor.
- Is it possible you have not tried yourself to compress typical images (illustrations) that often appear on websites with lower quality images?
- I’m not asking to optimise the plugin for a sample of my specific image.
Instead, I’m asking not to hardcode an arbitrary value of 75 as the lowest quality setting, as this may be a correct choice for some, but incorrect for others. It might be easy for you to agree with that it is not up to plugin authors to decide what settings people should use on their websites.
In any case, here is a typical sample illustration with the original 80-90 (284kb), 50 (154kb), and 25 (100kb) quality settings. I would be happy to put any of them on my site, especially for mobile users.
The link expires in 6 days.