Sitemap Generation for Crawler

  • Resolved Anonymous User 16759664

    (@anonymized-16759664)


    5 years, 8 months ago

    Is the Litespeed cache sitemap generation suppose to detect Woocommerce product and category URLs or is that only possible with a custom sitemap. I ran it but it does not pickup Woocommere products or categories.

Viewing 15 replies - 1 through 15 (of 39 total)
1 2 3
  • Plugin Support qtwrk

    (@qtwrk)

    5 years, 8 months ago

    Hi,

    No, it doesn’t.

    Please use custom sitemap for the time being.

    We are working on improve the sitemap generation

    Best regards,

    Thread Starter Anonymous User 16759664

    (@anonymized-16759664)

    5 years, 8 months ago

    I entered my custom sitemap from SEOPress and all I get is:
    Sitemap created successfully: 0 items

    Debug log shows failed to parse:

    11/21/20 18:01:40.513 [98.222.181.165:56610 1 Ugy] 🐞🗺️ ❌ failed to prase custom sitemap: Failed to parse xml
    11/21/20 18:01:40.513 [98.222.181.165:56610 1 Ugy] 🐞🗺️ Truncate sitemap
    11/21/20 18:01:40.535 [98.222.181.165:56610 1 Ugy] 🐞🗺️ Generate sitemap
    11/21/20 18:01:40.544 [98.222.181.165:56610 1 Ugy] 🐞 Crawler #0 touched end
    11/21/20 18:01:40.545 [98.222.181.165:56610 1 Ugy] 🐞 _terminate_running Touched end, whole crawled. Reload crawler!
    11/21/20 18:01:40.548 [98.222.181.165:56610 1 Ugy] [Ctrl] 301 from https://photoboothframesusa.com/wp-admin/admin.php
    11/21/20 18:01:40.548 [98.222.181.165:56610 1 Ugy] [Ctrl] 301 to https://photoboothframesusa.com/wp-admin/admin.php?page=litespeed-crawler
    11/21/20 18:01:40.548 [98.222.181.165:56610 1 Ugy] [Ctrl] X Cache_control -> no Cache ( 301 to same url )
    11/21/20 18:01:40.548 [98.222.181.165:56610 1 Ugy] [Ctrl] not cacheable before ctrl finalize
    11/21/20 18:01:40.548 [98.222.181.165:56610 1 Ugy] [Router] get_role: administrator
    11/21/20 18:01:40.549 [98.222.181.165:56610 1 Ugy] 💰 X-LiteSpeed-Cache-Control: no-cache
    11/21/20 18:01:40.549 [98.222.181.165:56610 1 Ugy] [Optm] bypass: Not frontend HTML type
    11/21/20 18:01:40.549 [98.222.181.165:56610 1 Ugy] End response

    Plugin Support qtwrk

    (@qtwrk)

    5 years, 8 months ago

    Hi,

    11/21/20 18:01:40.513 [98.222.181.165:56610 1 Ugy] 🐞🗺️ ❌ failed to prase custom sitemap: Failed to parse xml

    Could you please post few lines before this line ?

    Best regards,

    Thread Starter Anonymous User 16759664

    (@anonymized-16759664)

    5 years, 8 months ago

    text block before that line

    11/21/20 18:01:39.242 [98.222.181.165:56610 1 EUz] 💓 ——GET HTTP/1.1 (HTTPS) /wp-admin/admin.php
    11/21/20 18:01:39.242 [98.222.181.165:56610 1 EUz] Query String: page=litespeed-crawler&_=1605999696668
    11/21/20 18:01:39.242 [98.222.181.165:56610 1 EUz] HTTP_REFERER: https://photoboothframesusa.com/wp-admin/admin.php?page=litespeed-crawler
    11/21/20 18:01:39.242 [98.222.181.165:56610 1 EUz] Cookie _lscache_vary: admin_bar:1;logged-in:1;role:99
    11/21/20 18:01:39.256 [98.222.181.165:56610 1 Ugy] 💓 ——GET HTTP/1.1 (HTTPS) /wp-admin/admin.php
    11/21/20 18:01:39.256 [98.222.181.165:56610 1 Ugy] Query String: page=litespeed-crawler&LSCWP_CTRL=crawler&LSCWP_NONCE=229076d6ef&litespeed_type=start
    11/21/20 18:01:39.256 [98.222.181.165:56610 1 Ugy] HTTP_REFERER: https://photoboothframesusa.com/wp-admin/admin.php?page=litespeed-crawler
    11/21/20 18:01:39.256 [98.222.181.165:56610 1 Ugy] Cookie _lscache_vary: admin_bar:1;logged-in:1;role:99
    11/21/20 18:01:39.484 [98.222.181.165:56610 1 EUz] [Ctrl] X Cache_control -> no Cache ( logged in user )
    11/21/20 18:01:39.509 [98.222.181.165:56610 1 Ugy] [Ctrl] X Cache_control -> no Cache ( logged in user )
    11/21/20 18:01:39.575 [98.222.181.165:56610 1 Ugy] [Router] LSCWP_CTRL: crawler
    11/21/20 18:01:39.575 [98.222.181.165:56610 1 Ugy] [Router] LSCWP_CTRL verified: ‘crawler’
    11/21/20 18:01:39.576 [98.222.181.165:56610 1 Ugy] 🐞 Init
    11/21/20 18:01:39.576 [98.222.181.165:56610 1 Ugy] [Router] parsed type: start
    11/21/20 18:01:39.576 [98.222.181.165:56610 1 Ugy] 🐞 ……crawler manually ran……
    11/21/20 18:01:39.576 [98.222.181.165:56610 1 Ugy] 🐞 ……crawler started……
    11/21/20 18:01:39.577 [98.222.181.165:56610 1 Ugy] 🐞 TouchedEnd. regenerate sitemap….
    11/21/20 18:01:39.593 [98.222.181.165:56610 1 EUz] [Ctrl] X Cache_control -> no Cache ( Admin page )
    11/21/20 18:01:39.631 [98.222.181.165:56610 1 EUz] 🐞 Init
    11/21/20 18:01:39.643 [98.222.181.165:56610 1 EUz] [Ctrl] not cacheable before ctrl finalize
    11/21/20 18:01:39.644 [98.222.181.165:56610 1 EUz] [Router] get_role: administrator
    11/21/20 18:01:39.644 [98.222.181.165:56610 1 EUz] 💰 X-LiteSpeed-Cache-Control: no-cache
    11/21/20 18:01:39.644 [98.222.181.165:56610 1 EUz] [Optm] bypass: Not frontend HTML type
    11/21/20 18:01:39.644 [98.222.181.165:56610 1 EUz] End response

    Plugin Support qtwrk

    (@qtwrk)

    5 years, 8 months ago

    still doesn’t look like a completed log to

    could you please filter Ugy] and post all ?

    Thread Starter Anonymous User 16759664

    (@anonymized-16759664)

    5 years, 8 months ago

    I’m not sure what you mean by filter Ugy]. I will have to learn that. I enabled debug level advanced and went through the process again.

    [Moderated: Giant paste removed]

    • This reply was modified 5 years, 8 months ago by t-p.
    • This reply was modified 5 years, 8 months ago by t-p. Reason: giant paste removed
    Moderator t-p

    (@t-p)

    5 years, 8 months ago

    Side note to @nanoprobes ,

    Large code excerpt removed per forum guidelines. Please use Pastebin and Gist and copy your code there. Then return here and paste the link to that here.

    Thread Starter Anonymous User 16759664

    (@anonymized-16759664)

    5 years, 8 months ago

    Okay I am genuinely sorry and apologize for the inconvenience that may have caused.

    Thread Starter Anonymous User 16759664

    (@anonymized-16759664)

    5 years, 8 months ago

    https://pastebin.com/cwJ0EVyJ

    Plugin Support qtwrk

    (@qtwrk)

    5 years, 8 months ago

    Hi,

    is that the log you click “manual run” ?

    please try with “refresh sitemap” button , I would expect log like this

    https://www.litespeedshare.net/s/ihf-rvoq

    the query should be like this 

    
11/23/20 02:16:21.942 [xxx 1 0Wx] 💓 ------GET HTTP/1.1 (HTTPS) /wp-admin/admin.php
11/23/20 02:16:21.942 [xxx 1 0Wx] Query String: page=litespeed-crawler&LSCWP_CTRL=crawler&LSCWP_NONCE=4db7a7a09c&litespeed_type=refresh_map

    Best regards,

    Thread Starter Anonymous User 16759664

    (@anonymized-16759664)

    5 years, 8 months ago

    That’s the log when hitting manual run

    Here’s the log when hitting refresh crawler map
    https://pastebin.com/dbJ2QmCL

    Thread Starter Anonymous User 16759664

    (@anonymized-16759664)

    5 years, 8 months ago

    With “advanced log” option enabled
    https://pastebin.com/FaMKAEsw

    11/22/20 19:38:33.054 [185.105.38.25:64277 1 JSk] [Ctrl] 301 from https://photoboothframesusa.com/sitemap.xml
    11/22/20 19:38:33.054 [185.105.38.25:64277 1 JSk] [Ctrl] 301 to https://photoboothframesusa.com/sitemaps.xml

    It’s picking up my old sitemap that is no longer there from plugin SEO Framework which I have deleted and now using another plugin. It should be “sitemaps” not “sitemap” in the URL

    Plugin Support qtwrk

    (@qtwrk)

    5 years, 8 months ago

    you need to directly set your final URL in custom site map setting

    Thread Starter Anonymous User 16759664

    (@anonymized-16759664)

    5 years, 8 months ago

    Yes and it is set in the custom site map setting under sitemap settings tab

    Plugin Support qtwrk

    (@qtwrk)

    5 years, 8 months ago

    Hi,

    Could you please create a php file with code 

    <?php
$a = file_get_contents( 'https://photoboothframesusa.com/sitemaps.xml' );
$b = simplexml_load_string($a);
var_dump($b, !$b);
?>

    then access it by browser , see what it shows ?

    Best regards,

Viewing 15 replies - 1 through 15 (of 39 total)
1 2 3

The topic ‘Sitemap Generation for Crawler’ is closed to new replies.