Sitemap cannot be read by Google

  • Resolved ronalddts

    (@ronalddts)


    6 years ago

    After a month of not having 10% of my post indexed by Google, I just recognized that Yoast sitemap take huge time to load, and cause 524 error (my site is behind Cloudflare). That makes Google/Bing cannot access my website through sitemap.

    I am currently having over 500k posts. And trying to make the cache sitemap, but that doesn’t make any sense. When noone, except Google bot, access the sitemap, if the sitemap has been refresh, bot will hardly reach any of the posts again.

    This is just outrageous. I am suppose to leave really bad review on this matter. However, I am trying to post in the support to see is there is any solution from the team.

Viewing 8 replies - 1 through 8 (of 8 total)
  • Plugin Support amboutwe

    (@amboutwe)

    6 years ago

    In some cases, caching the sitemap with the code below can help. Typically code snippets are added to your theme’s functions.php file.

    If you’re not familiar with using code snippets, these articles have wonderful advice:
    http://www.wpbeginner.com/plugins/how-to-easily-add-custom-code-in-wordpress-without-breaking-your-site/
    http://www.wpbeginner.com/beginners-guide/beginners-guide-to-pasting-snippets-from-the-web-into-wordpress/

    /* Enable Yoast SEO sitemap caching
 * Last Tested: Mar 27 2019 using Yoast SEO 10.0.1 on WordPress 5.1.1
 * Credit: Yoast development team
 *********
 * Please note that changes will be applied upon next sitemap update.
 * To manually create the cached sitemap, please disable and enable the sitemaps then load each sitemap.
 * Once cached, Yoast SEO will update sitemap cache as needed.
 */
add_filter('wpseo_enable_xml_sitemap_transient_caching', '__return_true');
    Thread Starter ronalddts

    (@ronalddts)

    6 years ago

    Hi,

    Thanks for your reply. I knew about this solution, also tried already. However, there are some problems with this solution:

    1- My sitemap has over 500ish sup-sitemaps.
    2- Each sub-sitemaps has 1000 urls.
    3- It takes 5-7 minutes to finish loading 1 sub-sitemap. Practically, if I preload all those sub-sitemaps, it will take at least 2 days.
    4- Loading each sub-sitemap raises extensively cpu usage from mysql (even though my vps is quite strong).

    Therefore, loading 500ish sub-sitemaps everyday or twice a day for caching purpose is not viable.

    If you have any other advices, please let me know.

    Plugin Support amboutwe

    (@amboutwe)

    6 years ago

    You can create less sitemaps by increasing the number of items per sitemap. Example code snippet is below. This limits it to 10 items per sitemap so you’ll want to change that to a higher number.

    Per Google guidelines:

    All formats limit a single sitemap to 50MB (uncompressed) and 50,000 URLs.

    /* Change the number of entries per sitemap
 * Credit: Yoast Team
 * Last Tested: Oct 17 2018 using Yoast SEO 8.4 on WordPress 4.9.8
 *********
 */
 
add_filter( 'wpseo_sitemap_entries_per_page', 'max_entries_per_sitemap' );

 function max_entries_per_sitemap() {
    return 10;
}
    Thread Starter ronalddts

    (@ronalddts)

    6 years ago

    Hi,

    How would I specify the cache time of the sitemap?

    Thanks.

    Thread Starter ronalddts

    (@ronalddts)

    6 years ago

    Also, increasing the number of items per sitemap doesn’t help. Because with 1000 url, it takes 5-7 mins to generate the sub-sitemap, so that means there will be 50x times longer to generate 1 sub-sitemap with 50000 urls.

    I am testing. It’s already 1 hour from the start, but mysql is still running for 1 sub-sitemap. If the process takes more than 1 hour, I don’t think this is a good way to generate the sitemap.

    Regards.

    Plugin Support amboutwe

    (@amboutwe)

    6 years ago

    Rebuilding the sitemap feature to be faster is something on our radar (noted here) after the pending re-build of our front-end code.

    Learn more about the upcoming version that includes the front-end rebuild and how you can test it here: https://developer.yoast.com/upcoming-release-yoast-seo-14-0-indexables/

    Jerlyn

    (@jerparx)

    6 years ago

    Closed. No further questions.

    Thread Starter ronalddts

    (@ronalddts)

    5 years, 11 months ago

    After reading all the problems with Yoast plugin, I am genuinely hesitated to upgrade to the latest version. My site massively has 600k posts, which will take forever to recover if something goes wrong.

    Were the plugin tested with large site?

Viewing 8 replies - 1 through 8 (of 8 total)

The topic ‘Sitemap cannot be read by Google’ is closed to new replies.