Support » Plugin: Yoast SEO » [Plugin: WordPress SEO by Yoast] Robots.txt file not updating

  • Resolved Tristan



    I have problem with the robots.txt file when using this plugin.

    As far as I understand, if you make changes to the indexation settings page it should automatically update the robots.txt file right?

    Well, my robots.txt file doesn’t change. At first I didn’t have one (just the virtual one that WP creates), so I created my own, but none of the disallow rules get added to it when I change it in the plugin.

    Any way to fix this?

    Thanks! 🙂

Viewing 5 replies - 1 through 5 (of 5 total)
  • I’m also having issues submitting my sitemap to google webmaster… I get this error:

    URL restricted by robots.txt
    We encountered an error while trying to access your Sitemap. Please ensure your Sitemap follows our guidelines and can be accessed at the location you provided and then resubmit.

    Plugin Author Joost de Valk


    This plugin does nothing with your robots.txt file other than allowing you to edit it through the SEO -> Files section. The robots.txt is only one of the ways one can control what a search engine does, and frankly not a very good one.

    Hey Joost,

    thanks for the reply 🙂 So when choosing to not have the login pages indexed it just adds a nofollow, noindex to the login page rather than changing the robots file?

    Any idea why the sitemap isn’t being accepted? Do you have a link to a good robots.txt file example?



    P.S. – Thanks for making this awesome plugin! It could be easily be sold so really appreciate that you give it to us for free! 🙂

    Works now, logged in today and sitemap was accepted.

    Thanks anyway 🙂

    Plugin Author Joost de Valk


    Glad to see you resolved it 🙂

Viewing 5 replies - 1 through 5 (of 5 total)
  • The topic ‘[Plugin: WordPress SEO by Yoast] Robots.txt file not updating’ is closed to new replies.