WordPress.org

1. suprsam
   Member
   Posted 1 year ago #
   

   I have checked not to index the following:
   Subpages of archives and taxonomies
   Author archives
   Tag archives

   But my robots.txt still reads:
   User-agent: *
   Disallow: /wp-admin/
   Disallow: /wp-includes/

   Am I doing something wrong?

   http://wordpress.org/extend/plugins/wordpress-seo/

2. bamme
   Member
   Posted 1 year ago #
   

   i am getting the same issue.. for a temporary fix could someone paste the correct robots.txt content, so we can create a similar manual robots.txt file?

3. Primitive
   Member
   Posted 1 year ago #
   

   I thought this plugin excluded robots with the meta robots tag rather than a robots.txt file?

4. bamme
   Member
   Posted 1 year ago #
   

   how would i go about checking that has worked? i looked in source at meta tags, i dont see much hinting at anything working.. maybe if someone could post accurate code i can search??

5. Primitive
   Member
   Posted 1 year ago #
   

   If you ticked the box to stop indexing something, the html should show
   <meta name="robots" content="noindex,follow"/>

   Here's an example of the tag archives being noindex on one of my sites:
   http://j.mp/yWe15D

Topic Closed

This topic has been closed to new replies.