Support » Plugin: Yoast SEO » [Plugin: WordPress SEO by Yoast] incompitable vith functions do_robotstxt & do_robots

  • Just wanted to say since plugin use do_robotstxt & do_robots functions it’s not possible to make dynamic robots.txt files.
    While itš possible to produce virtual robots.txt files, because of same functions, all content from robots.txt also attach conent to sitemaps.xml 🙁

    based on old (2008) hack at

    <!– XML Sitemap Generated by Yoast WordPress SEO, containing 10 URLs –>
    </urlset>Disallow: /wp-admin
    Disallow: /wp-includes
    Disallow: /wp-login.php`

Viewing 7 replies - 1 through 7 (of 7 total)
  • forgot to say: I’m using WordPress MS 3.1 with subdomains AND domain mapping

    I’m also having issues with this, using the my_global_robots function. Using WP 3.1 MU, with subdirectories and domain mapping.

    Sitemaps work fine until I add my global robots file to mu-plugins, then
    accessing sitemaps.xml or gz results in displaying contents of the robots.txt file.

    I’ve had to disable my global robots for now.


    Using a custom function, ie add_action(‘do_robots’, ‘my_global_robots’) appears to works for the sitemap.xml.gz file(gwebmaster still loading) but breaks for the xml page as shown above. In fact it even adds the w3tc cache timestamps at the footer like it’s a regular wp page. Not sure if the undefined Disallow: before the rest will have any negative effect but I’d think not.

    Using a custom function plus remove_action(‘do_robots’,’do_robots’) in order to completely overwrite breaks both.

    Apologies but disregard above about xml.gz still working.

    I assumed since the downloaded file looked the same it should be fine, but after checking webmaster tools today it ins’t validating for any of my sites. I’m guessing the headers are wrong then, just like how they are wrong for the sitemap.xml, rendering the html as text.

    Plugin Author Joost de Valk


    I might have found the issue here, update coming in next release which I hope fixes this. In a future version I’ll move away from using the robots hook for my sitemaps anyway, so it’s a short term pain.

    look like we are on same path, that sitemap will be produced from feeds (according to Your tweet)

    Plugin Author Joost de Valk


    yeah my google news module, which i’m beta testing with a small group now, already does that and it works nicely.

Viewing 7 replies - 1 through 7 (of 7 total)
  • The topic ‘[Plugin: WordPress SEO by Yoast] incompitable vith functions do_robotstxt & do_robots’ is closed to new replies.