Title: Crawler stopped_highload Last modified: August 12, 2022 --- # Crawler stopped_highload * Resolved [gecko002002](https://wordpress.org/support/users/gecko002002/) * (@gecko002002) * [3 years, 9 months ago](https://wordpress.org/support/topic/crawler-stopped_highload/) * Hello guys, * So my crawler stopped working for a few days, actually it is stopping after crawling a few pages, like 30 out of more than 1000. Ended reason: stopped_highload. * I tried to restart the crawler manually or reset position but nothing happens. * Also I have a lot of cron jobs defined, 8 of them. Are all of them necessarily? Or can I disable some of them and run the crawler more often? * Cron jobs: [https://ibb.co/wRG30wj](https://ibb.co/wRG30wj) Litespeed settings: [https://ibb.co/6JYNF0y](https://ibb.co/6JYNF0y) * For hosting I have something like this: WordPress PRO 3 SSD – 200 GB storage SSD, 16 GB RAM, 6 x 3,2 Ghz CPU * Thanks guys for any help! * The page I need help with: _[[log in](https://login.wordpress.org/?redirect_to=https%3A%2F%2Fwordpress.org%2Fsupport%2Ftopic%2Fcrawler-stopped_highload%2F%3Foutput_format%3Dmd&locale=en_US) to see the link]_ Viewing 15 replies - 1 through 15 (of 23 total) 1 [2](https://wordpress.org/support/topic/crawler-stopped_highload/page/2/?output_format=md) [→](https://wordpress.org/support/topic/crawler-stopped_highload/page/2/?output_format=md) * Plugin Support [Hai Zheng⚡](https://wordpress.org/support/users/hailite/) * (@hailite) * [3 years, 9 months ago](https://wordpress.org/support/topic/crawler-stopped_highload/#post-15912136) * You can try a higher number for `Server Load Limit` setting. * [serpentdriver](https://wordpress.org/support/users/serpentdriver/) * (@serpentdriver) * [3 years, 9 months ago](https://wordpress.org/support/topic/crawler-stopped_highload/#post-15912283) * [@gecko002002](https://wordpress.org/support/users/gecko002002/) * If you have SSH access, run “uptime” command to see the average load of your server. If done please post these 3 values. * Are you on shared hosting or is your server dedicated? * Thread Starter [gecko002002](https://wordpress.org/support/users/gecko002002/) * (@gecko002002) * [3 years, 9 months ago](https://wordpress.org/support/topic/crawler-stopped_highload/#post-15912470) * [@hailite](https://wordpress.org/support/users/hailite/) this did the trick, the stopped_highload error dissapeared. It ran for a while but now I get the error stopped_maxtime. * [@serpentdriver](https://wordpress.org/support/users/serpentdriver/) unfortunately its a shared server and I dont have access to SSH. * Thread Starter [gecko002002](https://wordpress.org/support/users/gecko002002/) * (@gecko002002) * [3 years, 9 months ago](https://wordpress.org/support/topic/crawler-stopped_highload/#post-15912474) * [@hailite](https://wordpress.org/support/users/hailite/) actually I was wrong, I got the stopped_highload again. * Plugin Support [Hai Zheng⚡](https://wordpress.org/support/users/hailite/) * (@hailite) * [3 years, 9 months ago](https://wordpress.org/support/topic/crawler-stopped_highload/#post-15912597) * Try v5.2 (latest dev) * [serpentdriver](https://wordpress.org/support/users/serpentdriver/) * (@serpentdriver) * [3 years, 9 months ago](https://wordpress.org/support/topic/crawler-stopped_highload/#post-15912989) * [@gecko002002](https://wordpress.org/support/users/gecko002002/) * > [@serpentdriver](https://wordpress.org/support/users/serpentdriver/) unfortunately > its a shared server and I dont have access to SSH. * No problem. There are other methods to find the best settings so that the crawler does not terminate prematurely due to excessive load or timeout. However, the possibilities are limited on shared hosting, so you first have to find out what the ideal value for Server Load Limit is. The Server Load Limit setting is not very helpful for this. Especially since the value for the load limit is different on each server. This is especially true for shared hosting. But I can help you to find the right settings. But I need your support for this. * Shall we do this together? * Thread Starter [gecko002002](https://wordpress.org/support/users/gecko002002/) * (@gecko002002) * [3 years, 9 months ago](https://wordpress.org/support/topic/crawler-stopped_highload/#post-15913876) * Yes of course! What do you need from me? * But now it seems to work, maybe because its weekend and the server is less stressed I guess. * [serpentdriver](https://wordpress.org/support/users/serpentdriver/) * (@serpentdriver) * [3 years, 9 months ago](https://wordpress.org/support/topic/crawler-stopped_highload/#post-15914506) * There is not much you have to do. To get the information we need you must install PHPSysInfo, a script that displays information about your system nicely on your server. Install is easy. Just download it, create a directory (wherever you want) with name of your choice on your server and upload the script. In the root directory of this script there is a file named phpsysinfo.ini.new. Rename it to phpsysinfo. ini, that’s it. * There is a demo available that you can find at: [http://phpsysinfo.sourceforge.net/multi/index.php?disp=bootstrap](http://phpsysinfo.sourceforge.net/multi/index.php?disp=bootstrap) * Download from Sourceforge: [https://phpsysinfo.github.io/phpsysinfo/](https://phpsysinfo.github.io/phpsysinfo/) * Request the script in your browser. - This reply was modified 3 years, 9 months ago by [serpentdriver](https://wordpress.org/support/users/serpentdriver/). * Thread Starter [gecko002002](https://wordpress.org/support/users/gecko002002/) * (@gecko002002) * [3 years, 9 months ago](https://wordpress.org/support/topic/crawler-stopped_highload/#post-15927653) * Hello, * Sorry for the late response, had a few busy days. Here is the script that you told me about. * [https://craftup.ro/system/index.php](https://craftup.ro/system/index.php) * We did what you said but it doesnt seem to work. * Thanks! * [serpentdriver](https://wordpress.org/support/users/serpentdriver/) * (@serpentdriver) * [3 years, 9 months ago](https://wordpress.org/support/topic/crawler-stopped_highload/#post-15927747) * [@gecko002002](https://wordpress.org/support/users/gecko002002/) * Did you rename phpsysinfo.ini.new into phpsysinfo.ini? * Thread Starter [gecko002002](https://wordpress.org/support/users/gecko002002/) * (@gecko002002) * [3 years, 9 months ago](https://wordpress.org/support/topic/crawler-stopped_highload/#post-15927844) * Yes, we did. * [https://ibb.co/ZR93mDQ](https://ibb.co/ZR93mDQ) * [serpentdriver](https://wordpress.org/support/users/serpentdriver/) * (@serpentdriver) * [3 years, 9 months ago](https://wordpress.org/support/topic/crawler-stopped_highload/page/2/#post-15927892) * Okay, then I have bad news for you. Your provider removed all files and functions on your server to get information about the load. This also means that LiteSpeed cache plugin also can’t use load information because it also needs these system files. * If you click on the warning shield on top of the page you will see what is missing to run PhpSysInfo. * [serpentdriver](https://wordpress.org/support/users/serpentdriver/) * (@serpentdriver) * [3 years, 9 months ago](https://wordpress.org/support/topic/crawler-stopped_highload/page/2/#post-15927973) * Did you already give up? 😉 We have one last chance. Are you still ready? 🙂 * Thread Starter [gecko002002](https://wordpress.org/support/users/gecko002002/) * (@gecko002002) * [3 years, 9 months ago](https://wordpress.org/support/topic/crawler-stopped_highload/page/2/#post-15928081) * Haha, of course not. But this could also be a problem with our website. Some plugin or something that uses a lot of CPU. * Also was thinking about contacting the hosting provider to ask them about phpsysinfo * But if you have any ideeas, I cant wait to hear them 🙂 * Thanks again! * [serpentdriver](https://wordpress.org/support/users/serpentdriver/) * (@serpentdriver) * [3 years, 9 months ago](https://wordpress.org/support/topic/crawler-stopped_highload/page/2/#post-15928135) * Okay, just create a blank php file with a name of your choice and place it wherever you want. Paste the code below into this file and request it in browser. I have no hope that it works, but hope dies last…. 🙂 * ```