Support » Plugin: XML Sitemaps » Sitemap access blocked by robots.txt

  • Resolved antoine69

    (@antoine69)


    Hi
    I have an error in GWT when Google tries to access my sitemap. The error message is “restreint access by robots.txt file”.

    There were no robots.txt file before installing the plugin so I let the plugin manage all of his files.

    So what can I do please (I use version 4.0.4 of the plugin).

    Thanks

    https://wordpress.org/plugins/google-sitemap-generator/

Viewing 3 replies - 1 through 3 (of 3 total)
  • Arne

    (@arnee)

    Hi,

    the robots.txt is managed by WordPress. Can you check the “Search Engine Visibility” under the WordPress “Reading Settings”? The plugin just adds the URL to the sitemap to the generated file.

    Thread Starter antoine69

    (@antoine69)

    Yes, the box is unticked.
    The box was checked some days ago so the site was blocked for search engines. Then I untiked the box and installed Google XML Sitemaps.
    Now in GWT, I have still the website blocked but the date indicated is 23/04/2014 (04/23/2014 if you read the date as english person). And now we’re 24/04/2014, so it’s not up-to-date.
    In GWT, in “Blocked URLs” section, in the upper box, I have the robots.txt code:
    User-agent: *
    Disallow: /

    But when I click above, the link to the robots.txt I see:
    User-agent: *
    Disallow: /wp-admin/
    Disallow: /wp-includes/
    Sitemap: http://www.mywebsite.com/sitemap.xml

    Do I have to wait until tomorrow to see what will happen ?
    If I uninstall the plugin and then reinstall it, will it be gone ?

    Thanks

    Arne

    (@arnee)

    The robots.txt looks good. You can resubmit your sitemap manually in GWT, but it will take a little bit of time until Google tries it again after a “blocked by robots.txt” error.

    You can also use the “Test Sitemap” function in GWT to make sure it is accessible and correctly working. If everything is OK there, you will just have to wait a little bit. The plugin can’t do anything against that.

Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘Sitemap access blocked by robots.txt’ is closed to new replies.