Yoast SEO
[Plugin: WordPress SEO by Yoast] robots.txt doesn't seem to be changing (5 posts)

  1. suprsam
    Posted 4 years ago #

    I have checked not to index the following:
    Subpages of archives and taxonomies
    Author archives
    Tag archives

    But my robots.txt still reads:
    User-agent: *
    Disallow: /wp-admin/
    Disallow: /wp-includes/

    Am I doing something wrong?


  2. bamme
    Posted 4 years ago #

    i am getting the same issue.. for a temporary fix could someone paste the correct robots.txt content, so we can create a similar manual robots.txt file?

  3. Primitive
    Posted 4 years ago #

    I thought this plugin excluded robots with the meta robots tag rather than a robots.txt file?

  4. bamme
    Posted 4 years ago #

    how would i go about checking that has worked? i looked in source at meta tags, i dont see much hinting at anything working.. maybe if someone could post accurate code i can search??

  5. Primitive
    Posted 4 years ago #

    If you ticked the box to stop indexing something, the html should show
    <meta name="robots" content="noindex,follow"/>

    Here's an example of the tag archives being noindex on one of my sites:

Topic Closed

This topic has been closed to new replies.

About this Plugin

  • Yoast SEO
  • Frequently Asked Questions
  • Support Threads
  • Reviews

About this Topic