Crawler never gives a green hit, only blue and not warming the cache

  • Resolved nemovich

    (@nemovich)


    1 year, 1 month ago

    I’m facing an issue with Litespeed Cache on my website. I’m using a temporary domain from Hostingr (looks like https://pantiq-carbou-9496399.hostingersite.com/), and I’ve set up the crawler. However, I consistently get a blue status for cache, never green. When I visit my site in a private browser, the first load is slow, but the reload is fast, indicating that the cache warming isn’t working.

    Here are the steps I’ve taken based on online solutions:

    1. Checked the purge log (see attached).
    2. Disabled all purge hooks and unchecked auto purge rules.
    3. Ran the crawler twice in a row without making any changes (still no green).
    4. Confirmed with my hosting provider that there are no server issues from their end.
    5. Left the website overnight, and it’s still not giving a green status, even though it’s re-crawling every 15 minutes and no edit/update actions.

    I’ve spent hours searching for a solution on support forums, but I haven’t found a resolution yet. Can you please assist me in resolving this cache warming issue with the crawler?

    Thank you for your help.

    Report number: FKMEYLXR

    Report date: 10/24/2023 12:43:49

    Purge Log

    10/24/23 13:45:57.745 [62.4.147.241:44264 1 91Y] 💓 ------GET HTTP/1.1 (HTTPS)  /wp-admin/admin.php
10/24/23 13:45:57.745 [62.4.147.241:44264 1 91Y] Query String: page=litespeed-cache&LSCWP_CTRL=purge&LSCWP_NONCE=e6a918cd3e&litespeed_type=purge_all
10/24/23 13:45:57.745 [62.4.147.241:44264 1 91Y] HTTP_REFERER: https://***private***/wp-admin/admin.php?page=litespeed-cache
10/24/23 13:45:57.745 [62.4.147.241:44264 1 91Y] User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36
10/24/23 13:45:57.745 [62.4.147.241:44264 1 91Y] Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
10/24/23 13:45:57.745 [62.4.147.241:44264 1 91Y] Accept Encoding: br
10/24/23 13:45:57.745 [62.4.147.241:44264 1 91Y] Cookie _lscache_vary: cb1d846862a0ef6ba4becbde958309bb
10/24/23 13:45:57.745 [62.4.147.241:44264 1 91Y] X-LSCACHE: true
10/24/23 13:45:57.745 [62.4.147.241:44264 1 91Y] LSCACHE_VARY_VALUE: +webp
10/24/23 13:45:57.745 [62.4.147.241:44264 1 91Y] X-LiteSpeed-Purge: public,963_ => LiteSpeed\LSC->send_headers()@616 => WP_Hook->apply_filters(,ARRAY)@310 => WP_Hook->do_action(ARRAY)@334 => 
/***private***/public_html/wp-includes/load.php@517

    debug log

    10/24/23 14:55:16.762 [62.4.147.241:18174 1 0bY] 💓 ------POST HTTP/1.1 (HTTPS)  /wp-admin/admin-ajax.php
10/24/23 14:55:16.762 [62.4.147.241:18174 1 0bY] Query String: 
10/24/23 14:55:16.762 [62.4.147.241:18174 1 0bY] HTTP_REFERER: https://***private***/wp-admin/post.php?post=9688&action=elementor
10/24/23 14:55:16.762 [62.4.147.241:18174 1 0bY] User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36
10/24/23 14:55:16.762 [62.4.147.241:18174 1 0bY] Accept: application/json, text/javascript, */*; q=0.01
10/24/23 14:55:16.762 [62.4.147.241:18174 1 0bY] Accept Encoding: br
10/24/23 14:55:16.762 [62.4.147.241:18174 1 0bY] Cookie _lscache_vary: admin_bar:1;guest_mode:1;logged-in:1;role:99;role_exclude_cache:1
10/24/23 14:55:16.762 [62.4.147.241:18174 1 0bY] X-LSCACHE: true
10/24/23 14:55:16.914 [62.4.147.241:18174 1 0bY] [API] Disabled_all due to elementor edit mode in HTTP_REFERER
10/24/23 14:55:16.914 [62.4.147.241:18174 1 0bY] 👮  No cache due to Admin page
10/24/23 14:55:16.914 [62.4.147.241:18174 1 0bY] [Core] Bypassed due to debug disable all setting
10/24/23 14:55:17.120 [62.4.147.241:18174 1 0bY] [Router] LSCWP_CTRL bypassed empty
10/24/23 14:55:17.120 [62.4.147.241:18174 1 0bY] [Media] init




10/24/23 14:55:28.408 [62.4.147.241:29692 1 KB1] 💓 ------GET HTTP/1.1 (HTTPS)  /wp-admin/plugins.php
10/24/23 14:55:28.408 [62.4.147.241:29692 1 KB1] Query String: 
10/24/23 14:55:28.408 [62.4.147.241:29692 1 KB1] HTTP_REFERER: https://***private***/wp-admin/admin.php?page=litespeed-crawler
10/24/23 14:55:28.408 [62.4.147.241:29692 1 KB1] User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36
10/24/23 14:55:28.408 [62.4.147.241:29692 1 KB1] Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
10/24/23 14:55:28.408 [62.4.147.241:29692 1 KB1] Accept Encoding: br
10/24/23 14:55:28.408 [62.4.147.241:29692 1 KB1] Cookie _lscache_vary: admin_bar:1;guest_mode:1;logged-in:1;role:99;role_exclude_cache:1
10/24/23 14:55:28.408 [62.4.147.241:29692 1 KB1] X-LSCACHE: true
10/24/23 14:55:28.408 [62.4.147.241:29692 1 KB1] LSCACHE_VARY_VALUE: +webp
10/24/23 14:55:28.630 [62.4.147.241:29692 1 KB1] 👮  No cache due to Admin page
10/24/23 14:55:28.689 [62.4.147.241:29692 1 KB1] [Ctrl] X Cache_control -> private ( logged in user )
10/24/23 14:55:28.689 [62.4.147.241:29692 1 KB1] [Media] init
10/24/23 14:55:28.689 [62.4.147.241:29692 1 KB1] [LQIP] init
10/24/23 14:55:28.689 [62.4.147.241:29692 1 KB1] [CDN] init
10/24/23 14:55:28.689 [62.4.147.241:29692 1 KB1] [Router] CDN bypassed: is not ajax call
10/24/23 14:55:28.689 [62.4.147.241:29692 1 KB1] [Router] CDN bypassed: ajax call from admin
10/24/23 14:55:28.689 [62.4.147.241:29692 1 KB1] ⏰  Init
10/24/23 14:55:28.690 [62.4.147.241:29692 1 KB1] [Router] LSCWP_CTRL bypassed empty
10/24/23 14:55:34.037 [62.4.147.241:29692 1 KB1] [Core] CHK html bypass: miss footer const
10/24/23 14:55:34.038 [62.4.147.241:29692 1 KB1] [Ctrl] not cacheable before ctrl finalize
10/24/23 14:55:34.038 [62.4.147.241:29692 1 KB1] [Router] get_role: administrator
10/24/23 14:55:34.038 [62.4.147.241:29692 1 KB1] [Vary] role in vary_group [group] 99
10/24/23 14:55:34.038 [62.4.147.241:29692 1 KB1] [Vary] show_admin_bar_front: true
10/24/23 14:55:34.038 [62.4.147.241:29692 1 KB1] [Vary] admin bar : true
10/24/23 14:55:34.038 [62.4.147.241:29692 1 KB1] [Vary] no custimzed vary
10/24/23 14:55:34.038 [62.4.147.241:29692 1 KB1] [Ctrl] ❌ forced no cache [reason] DONOTCACHEPAGE const
10/24/23 14:55:34.038 [62.4.147.241:29692 1 KB1] 💰 X-LiteSpeed-Cache-Control: no-cache
10/24/23 14:55:34.038 [62.4.147.241:29692 1 KB1] [Vary] uid: 5
10/24/23 14:55:34.038 [62.4.147.241:29692 1 KB1] [Vary] role in vary_group [group] 99
10/24/23 14:55:34.038 [62.4.147.241:29692 1 KB1] [Vary] show_admin_bar_front: true
10/24/23 14:55:34.039 [62.4.147.241:29692 1 KB1] [Vary] admin bar : true
10/24/23 14:55:34.039 [62.4.147.241:29692 1 KB1] [Core] CHK html bypass: miss footer const
10/24/23 14:55:34.039 [62.4.147.241:29692 1 KB1] [Core] run hook litespeed_buffer_finalize
10/24/23 14:55:34.039 [62.4.147.241:29692 1 KB1] Response headers --- array (
  0 => 'X-Powered-By: PHP/8.1.22',
  1 => 'Expires: Wed, 11 Jan 1984 05:00:00 GMT',
  2 => 'Cache-Control: no-cache, must-revalidate, max-age=0, no-store, private',
  3 => 'Referrer-Policy: strict-origin-when-cross-origin',
  4 => 'X-Frame-Options: SAMEORIGIN',
  5 => 'Content-Type: text/html; charset=UTF-8',
  6 => 'X-LiteSpeed-Cache-Control: no-cache',
)
10/24/23 14:55:34.040 [62.4.147.241:29692 1 KB1] End response
--------------------------------------------------Duration: 5,633.00 ms------------------------------
    • This topic was modified 1 year, 1 month ago by nemovich.
Viewing 7 replies - 1 through 7 (of 7 total)
  • Plugin Support qtwrk

    (@qtwrk)

    1 year, 1 month ago

    your setting set server IP to 45.xx.xxx.109 yet your DNS goes to 149.xxx.xxx.157 which one is the actual server IP ? because .109 seems not working

    Thread Starter nemovich

    (@nemovich)

    1 year, 1 month ago

    I just talk to hosting support and the IP address 45.xx.xxx.109 is correct.

    The issue is I’m using the preview domain they gave me to setup my website and these domains have CDN enabled by default and cannot be disabled or pointed elsewhere. That’s why you see a different IP every time you do a DNS lookup.

    They told me that this issue will be fixed when I put my actual domain.

    So my question is if I remove the IP in plugin settings, will it be able to crawl normally ?

    Thanks

    Plugin Support qtwrk

    (@qtwrk)

    1 year, 1 month ago

    please try in crawler sitemap setting -> drop domain , set it to off , regenerate sitemap list and then cralw it again.

    Thread Starter nemovich

    (@nemovich)

    1 year, 1 month ago

    That did it ! now starting to see green hits and the cache is working properly, although the crawling is slow (understandable) but working nonetheless.

    Thank you.

    Thread Starter nemovich

    (@nemovich)

    1 year, 1 month ago

    One more issue, when I activate the lscache plugin my website becomes slower when logged in as admin (backend and front). If I deactivate the plugin the website becomes much faster. all of this while logged in as admin.

    What could be the issue ?
    Thank you for your help

    Plugin Support qtwrk

    (@qtwrk)

    1 year, 1 month ago

    yes , it is kind of normal when cache is miss , the page could take longer to load , as some optimization option will take time to do

    like no plugin TTFB 2 second , cache miss 2.5 seconds , cache hit 0.2 seconds

    but you could also try disable object cache , I have seen case where it was not set properly that slow things down.

    Thread Starter nemovich

    (@nemovich)

    1 year, 1 month ago

    So after much testing and plugin conflict detection, I narrowed it down to lscache (5.7.0.1) and Woocommerce (8.2.1) both on latest version. If I disable one of them, the website becomes snappy.
    I disabled object cache (memcached) but it had very little effect compared to disabling one of the two plugins.

Viewing 7 replies - 1 through 7 (of 7 total)
  • The topic ‘Crawler never gives a green hit, only blue and not warming the cache’ is closed to new replies.