Support » Plugin: Yoast SEO » Robots.txt

  • Resolved bobjones


    I’ve set my robots.txt settings in WordPress SEO. But I also noticed the robots.txt page in the root folder of my site is blank.

    Should I leave the blank robots.txt? Is WordPress SEO handling those settings in another way?

    Meanwile, my site has been active for a month and has not yet been indexed. So, I want to make sure everything is OK with robots.txt.


Viewing 3 replies - 1 through 3 (of 3 total)
  • Plugin Author Joost de Valk


    You don’t need a robots.txt, my plugin doesn’t do to much with it either.

    You don’t need a robots.txt?

    There are endless discussions on the web of what’s appropriate for robots.txt. Now you say we don’t need it at all?

    When did this new orthodoxy take hold?

    By the way, if your plugin isn’t editing robots.txt, how does it determine whether items such as tags and categories get indexed or not? I understood that to be the job of robots.txt.

    Plugin Author Joost de Valk


    No it’s not the job of robots.txt, it’s the job of the robots meta tag on each page. You should read this article and watch the video:

    Preventing your site from being indexed, the right way

Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘Robots.txt’ is closed to new replies.