Server cron job to run the crawler

  • Resolved Ivan Nugraha

    (@mangwp)


    1 year, 2 months ago

    Hi, what is the best method to use server cron job and cli command to purge the cache then run the crawl until all the url is cached?

    I have 4 crawler (guest mobile, guest, guest mobile webp and guess webp) and i disable to crawler cron from the settings. Currently the wp litespeed-crawler run command is only able to crawl one crawler for each command. So I have to run command 4 times and i have to run the command again to be able to cached the url (so in total i have to run 8 times). But the strange things, is after the last crawler (on 4th ru or on 8th run) when the crawler should go back to the first crawler, the command doesnt work, I have to click button “Manually Run” from the setting page to make next command works.

    The point is everyday at specific time, i wanna purge and cache the pages. Do you have any best method to achieve that? Thanks

Viewing 4 replies - 1 through 4 (of 4 total)
  • Plugin Support qtwrk

    (@qtwrk)

    1 year, 2 months ago

    instead of “manual run” , try use wp litespeed-crawler reset

    see if it makes any difference

    Thread Starter Ivan Nugraha

    (@mangwp)

    1 year, 2 months ago

    I tried that but doenst work. After crawler goes back to the first, the command become doesnt work work. It stuck at

    Start watching…
    22 Aug 2024 16:35:48     Size: 37     Crawler: #1     Position: 1     Threads: 1     Status: end.

    I have to press buttopn “Manually Run” then the command is working again. Is there any cli command to trigger the “Manually Run” button?


    Plugin Support qtwrk

    (@qtwrk)

    1 year, 2 months ago

    they are calling same functions in code , should act same

    when you see it suck , please run the cli twice or thrice again like few minutes apart, what does it show ?

    cfipilot

    (@cfipilot)

    9 months ago

    How do you set more then 1 crawler? I can run wp litespeed-crawler run as many times as i want but i still only get a single crawler

Viewing 4 replies - 1 through 4 (of 4 total)

The topic ‘Server cron job to run the crawler’ is closed to new replies.