WordPress.org

Ready to get started?Download WordPress

Forums

[Plugin: WordPress SEO] Tags are still being indexed (4 posts)

  1. xaine05
    Member
    Posted 3 years ago #

    I recently consolidated 3 plugins, All In One SEO, Robots Meta and RSS Footer, by using Joost de Valks new WordPress SEO plugin. It has been a great asset but I have noticed a consistent problem across all sites that I have updated, the tag archives are being indexed.

    I have selected for them not to be indexed but I am wondering if because I am selecting them to be turned off within WordPress SEO after importing the options from Robots Meta it is making this selection null and void.

    I never had this problem before when using Robots Meta, in fact it worked great, but now that I am using WordPress SEO on all my sites I am now finding I am having to fix sites that are getting a dupe content penalty for these tags.

    So my question may be more directed to the plugin author to establish why it is that when even having selected for a tag archive not to be indexed, they are being so regardless.

    Just to reiterate when I install the plugin on a site I import the options and data from the three plugins. Then when I go into WordPress SEO none of those options appear to be active or selected. So I am just wondering that by me re-selecting them I am in fact overriding the previous plugins options of having them switched on.

  2. xaine05
    Member
    Posted 3 years ago #

    Sorry for multiple posts but I have been doing a little research and trying to identify the problem.

    For one of my sites http://www.golfclothingwear.co.uk I can see in Webmaster tools that the robots.txt file does exist when I amend the URL with the robots.txt extension but all I see within it is the:

    User-agent: *
    Disallow:

    So basically wouldn't this suggest that the entire site is allowed to be indexed? How does this plugin tell the Googlebots not to index or follow the tag archives or any other option I have selected. Shouldn't the robots.txt file be amended with the added strings of data telling it not to index certain parts of the site?

    I'm not entirely sure how the process works but my understanding was that this file should have the selected criteria appended to the robots.txt file.

  3. Spiritualight
    Member
    Posted 3 years ago #

    I have been having the same problem and emailed Joost for some clarification but never got a reply. I have had to manually update the robots.txt file in 'Edit Files' in the SEO admin panel, and have the offending tag urls removed by using the remove url in webmaster tools. I also have

    User-agent: *
    Disallow: /cgi-bin/
    Disallow: /wp-
    Disallow: /wp-admin/
    Disallow: /wp-includes/
    Disallow: /wp-content/
    Disallow: /comments/feed/
    Disallow: /category/
    Disallow: /tag/

    this in my robots.txt and they are still getting indexed. I'm just having to delete them as they get indexed, thought about going back to All In One SEO and Robots Meta plugins, but I really like the snippet tool on the SEO side... but some more information on this from the developer would have been nice!

  4. xaine05
    Member
    Posted 3 years ago #

    The indexing problem is still ongoing! A site that I've had running for years with All in One SEO is now since the install of WordPress SEO having the tag archives indexed. I have no idea how Google is even finding these pages as they can't be found anywhere from the site.

    I have noticed on the header.php file that this is present

    <meta name="robots" content="follow, all" />

    Should this be present or show different information if I have opted for certain things to not be indexed. On of my tag archives I can see that in the header section when I check the source code it has this:

    <meta name="robots" content="follow, all" />

    Not entirely sure why it is that since the install of this plugin I am seeing a great deal of unusual things happening with the indexing of the site. None of the tag URL's can be found within my sitemap either.

Topic Closed

This topic has been closed to new replies.

About this Topic