So I'm investigating slowness on my server causing timeouts and I've been using New Relic to try pin down what is holding everything up.
It seems that the W3 Prime Cache hitting the sitemap pages were taking 5 seconds per sitemap page to load. Causing priming to be slow, posting to be slow, and wp-cron to be slow.
I tried in a browser to load directly an XML sitemap page and it indeed takes 4-5 seconds to load for me.
I thought the Sitemap was written to file, but it seems to be dynamically generated when called. So when sitemap_index.xml is called is the Yoast SEO plugin tapping the entire database, pulling relevant information and dividing it out into urls per page every single time the sitemap is accessed?
I'd love to know what is happening on the sitemap side of things, should it be writing to a file somewhere and the sitemap and plugin then doesn't have lots of work to do, it only needs to add new posts on publish or what is the right process. Dynamically loading each time doesn't seem right to me and is causing a very slow process
With it taking so long to generate I've been trying to play around with the settings to see is there an optimal window
I've set Sitemap size to 1000, sitting around the 5 seconds load. doubling to 2000 per sitemap throws the load time into the 20 second range.
Dropping the sitemap size down to 100 seems to give me faster load times in the browser. So I'm waiting now to see if I get an improved load time in the wp-cron or if the 5 seconds will be distributed as 0.5 seconds per sitemap of 100 that was previously 1000.
Is anyone else suffering this problem?