• Resolved joy0114

    (@joy0114)


    Hi,

    I wonder about the actual functioning of my crawler…
    The sitemap contains 460 pages.

    What I do:
    I purge the cache
    I launch the crawler
    The crawler starts, stops without having gone to the end (25 pages sometimes, 140 pages sometimes, for example).
    Sometimes the crawler reaches 460 pages.

    In this case (full crawl):

    Test of the random pages (from another browser, not logged):
    At the bottom of the page I find date and hour which are the ones of THIS visit:

    <!– Page optimized by LiteSpeed Cache @2022-09-30 12:09:56 –>
    <!– Page generated by LiteSpeed Cache 5.2.1 on 2022-09-30 12:09:54 –>
    <!– X-LiteSpeed-Cache-Control: public,max-age=604800 –>
    <!– X-LiteSpeed-Tag: 219_HTTP.200,219_product,219_URL./produit/bracelet-br0011-vogue-les-flots/,219_Po.9270,219_ –>
    <!– Object Cache [total] 26498 [hit_incall] 25611 [hit] 105 [miss_incall] 246 [miss] 536 [set] 571 –>

    I visit a few pages, then come back to the first page tested:
    Then at that time, the date and time of the cache will correspond to the first visit. OK.
    But I expected, from the first visit, to the date and time of the crawl !

    Besides, I don’t understand how the “refresh map” button works in the “map” tab:
    If I click this button, it empties the cache and prepares the sitemap for a next crawl, it doesn’t give me the list of currently cached pages.

    Am I wrong somewhere ?

    Thanks in advance for any advice.

    My report number: ZYBEGLRZ

    The page I need help with: [log in to see the link]

Viewing 1 replies (of 1 total)
  • Thread Starter joy0114

    (@joy0114)

    Hi,

    It seems to be ok now.
    purge and re-crawl…

    As for the button “refresh map”, I haven’t really understood yet.

    Best regards

Viewing 1 replies (of 1 total)
  • The topic ‘Does my crawler really work’ is closed to new replies.