crawler idle issue

  • Resolved unitiket

    (@unitiket)


    6 months, 3 weeks ago

    Hello,

    Please see the attached screenshot. According to the interval I set, the crawler should restart 7 minutes after a pause. However, it has already been idle for 20 minutes and does not start. Moreover, it appears to be stuck because its status shows “Status: crawling, updated position”, which indicates that it should still be scanning the sitemap — but it is clearly not doing so.

    Between the configured 7-minute intervals, the status should change to “Status: stopped_maxtime”, but instead it remains on “Status: crawling, updated position”.

    In practice, the crawler only starts approximately every 60–80 minutes, runs for 5–7 minutes, and then goes back to sleep again with the status “Status: crawling, updated position”. It never transitions to “Status: stopped_maxtime” at all.

    Could you please advise on how we can fix this behavior?

    Thank you in advance for your assistance.

Viewing 3 replies - 1 through 3 (of 3 total)
  • Plugin Support litetim

    (@litetim)

    6 months, 3 weeks ago

    @unitiket I am curios about how cron is running on your server.
    Is it set to WP default? or you have a server cron enabled?
    What is the time between cron runs?

    Thread Starter unitiket

    (@unitiket)

    6 months, 3 weeks ago

    Hello,

    We have try with define(‘DISABLE_WP_CRON’, true); option in wp-config.php (and set every 5 minites cron wtih curl URL to domain/wp-config.php )

    and with default WordPress cron setup, but with exclude from cache wp-cron.php URI

    Same result

    Plugin Support qtwrk

    (@qtwrk)

    6 months, 3 weeks ago

    please go to toolbox -> debug setting, debug log to ON, debug level to advance, debug include URI , add async_litespeed , then try to run crawler by manual or cron , then check the log view, see if there is anythign show up in log

Viewing 3 replies - 1 through 3 (of 3 total)

The topic ‘crawler idle issue’ is closed to new replies.