Popular WordPress SEO Plugins including the Yoast WordPress SEO Plugin (uses nofollow and noindex) and the All in One SEO Pack Plugin (uses nofollow and noindex) when used incorrectly could seriously damage a WordPress sites search engine rankings.
If you’ve researched noindex and nofollow and read noindexing and/or nofollowing entire WordPress sections (categories, tags and other archives) to prevent duplicate content issues is best SEO practice, I’m afraid your SEO sources are wrong: they are 5+ years out of date. Noindex and nofollow can cause serious SEO damage, best SEO practice 2016 and beyond is do NOT use noindex or nofollow.
Yoast SEO Plugin and All In One SEO Plugin Cause SEO Damage
Using Yoast’s or All In One SEO’s section wide noindex and/or nofollow features in particular is SEO suicide. If you use Yoast SEO or All In One SEO do NOT use the section wide noindex/nofollow settings: these add noindex and nofollow robots meta tags to specific sections of a site like on Category archives!!!
Webmasters spend a lot of their SEO time gaining (even paying for) valuable backlinks to their WordPress blogs only to waste a significant proportion of that hard earned link benefit (PR : PageRank) by installing WordPress SEO plugins which delete/waste that link benefit through nofollow/noindex features.
See the Stallion WordPress SEO Plugin Documentation and the Other Notes tab for more information and discussion.
Stallion WordPress SEO Plugin Not Index Alternative to noindex
The Stallion WordPress SEO Plugin has an alternative solution (based on canonical URLs) to using noindex called Stallion Not Index. With the Stallion Not Index feature most of the link benefit (the ~15%) that’s wasted by the noindex robots meta tag is recycled back to other pages on the site.
With the Stallion WordPress SEO Plugin we can link to internal categories, tags, monthly archives, custom taxonomies, custom post type archives… and recycle most of the link benefit via canonical URLs through one easy to use options page.
Note: We still loose ~15% of the ~15% of the link benefit we would loose via noindex: we recover ~12.75% of the ~15% we’d loose via noindex (we loose around ~2.25% vs loosing ~15% using noindex). Hoping that was as confusing to read as it was to write 🙂 This is because Google considers a canonical URL as a soft 301 redirect in terms of link benefit (PR) transfer. A 301 redirect costs ~15% of the link benefit accumulated by the redirected webpage: Google is one smart cookie, they know SEO’s are trying to sculpt PageRank so add dampening factors to limit abuse, without dampening factors a site could reuse PageRank over and over again.
Stallion WordPress SEO Plugin Features
The Stallion WordPress SEO Plugin uses canonical URLs to mimic the affects of noindex, but without wasting all the valuable link benefit. When a page has a canonical URL it tells major search engines like Google where the preferred URL for that page is.
This is what the Stallion WordPress SEO Plugin can do:
Stallion Not Index login pages redirecting link benefit to home.
Stallion Not Index date archive pages redirecting link benefit to home.
Stallion Not Index author archive pages redirecting link benefit to home.
Stallion Not Index paged author archive pages (pages 2,3,4 etc…) redirecting link benefit to the first page in the author archive.
Stallion Not Index search results pages redirecting link benefit to home.
Stallion Not Index paged search results pages (pages 2,3,4 etc…) redirecting link benefit to the first page in the search result.
Stallion Not Index category archive pages redirecting link benefit to home (not a suggested option).
Stallion Not Index paged category archive pages (pages 2,3,4 etc…) redirecting link benefit to the first page in the category.
Stallion Not Index tag archive pages redirecting link benefit to home (not a suggested option).
Stallion Not Index paged tag archive pages (pages 2,3,4 etc…) redirecting link benefit to the first page in the tag.
Stallion Not Index custom taxonomy archive pages redirecting link benefit to home (not a suggested option).
Stallion Not Index paged custom taxonomy archive pages (pages 2,3,4 etc…) redirecting link benefit to the first page in the custom taxonomy archive.
Stallion Not Index custom post type archive pages redirecting link benefit to home (not a suggested option).
Stallion Not Index paged custom post type archive pages (pages 2,3,4 etc…) redirecting link benefit to the first page in the custom post type archive.
Stallion Not Index paged home archive pages (pages 2,3,4 etc…) redirecting link benefit to the home page.
Stallion Not Index paged comments pages (pages 2,3,4 etc…) redirecting link benefit to the main blog post or static page.
How Canonical URLs Work
Most WordPress blogs use search engine friendly permalinks, but that does not mean the default dynamic WordPress URLs no longer exist. There are always at least two ways (two different URLs) to access a WordPress Post, having canonical URLs (which is core WordPress) prevents Google etc… from indexing the same post under multiple URLs (reduces the chances of duplicate content issues).
Canonical basically redirects most of the link benefit (~15% is lost: same cost as a 301 redirect) and SEO rankings to the preferred canonical URL whilst still allowing visitors to those webpages to view the pages in a browser: to Google a canonical URL is like a 301 permanent redirect without the browser redirecting, a soft 301 redirect.
By adding targeted canonical URLs to sections of a WordPress site we do not want indexing, Google etc… will not index those sections, whilst redirecting most of the link benefit to the canonical URL (less wasted link benefit). The Stallion WordPress SEO Plugin sets canonical URLs (via the Stallion Not Index options) to the home page or the first page of categories, tags, author archives, custom taxonomy archives, custom post type archives and search results archives recycling the link benefit that would be wasted from using noindex robots meta tags.
WordPress SEO Plugin, WordPress SEO, SEO Plugin, SEO, WordPress, Plugin, Stallion SEO, Stallion WordPress SEO, Stallion WordPress SEO Plugin, Stallion SEO Plugin, Yoast WordPress SEO, Yoast SEO, Yoast, All In One WordPress SEO, All In One SEO, All In One, Search Engine Optimization, Search Engine Optimisation, Canonical, Canonical URL, Noindex, Nofollow, Google, Search Engine, PageRank, PR, Link Benefit, Robots Meta Tags, Meta Tags
NEVER use nofollow
SEO Experts have known for years (since at least 2009) nofollow deletes valuable Google link benefit (PR), a Google employee (Matt Cutt’s) has even confirmed this SEO fact on his blog http://www.mattcutts.com/blog/pagerank-sculpting/
So what happens when you have a page with “ten PageRank points” and ten outgoing links, and five of those links are nofollowed? Let’s leave aside the decay factor to focus on the core part of the question. Originally, the five links without nofollow would have flowed two points of PageRank each (in essence, the nofollowed links didn’t count toward the denominator when dividing PageRank by the outdegree of the page). More than a year ago, Google changed how the PageRank flows so that the five links without nofollow would flow one point of PageRank each.
Still not convinced watch this YouTube video by Matt Cutts.
Note Matt Cutts says the PageRank that goes through nofollow links “evaporates or disappears”. Nofollow deletes link benefit, never use it!!!
Come on SEO plugin developers, we’ve known this since 2009, update your SEO plugins. At least remove section wide nofollow features so clueless WordPress users don’t inadvertently block most of their site from being indexed fully and don’t waste half their link benefit.
Using nofollow is by far the dumbest SEO thing a webmaster can do, NEVER, EVER USE NOFOLLOW!!!
All WordPress SEO Plugins that add nofollow robots meta tags and/or add rel=”nofollow” attributes to internal links within your site should be used with great care, a nofollowed link is deleting valuable link benefit!
The WordPress core commenting system uses nofollow on comment author links, WordPress by default is already deleting too much of your valuable link benefit (it’s possible to remove this at theme level), don’t make it worse by adding more via WordPress SEO plugins!
Joost De Valk Finally Listens to SEO Sense
Good news: after years of the Yoast WordPress SEO Plugin deleting your hard earned PageRank via it’s dumb section wide nofollow features, Yoast has finally removed the section wide nofollow features. Yeah, only took about 5 or 6 years of yours truly moaning about it for them to finally admit they were wrong and to fix it.
Bad news for All In One SEO Plugin users, the section wide nofollow features still exist, guess I need to have online arguments with Michael Torbert (All In One SEO’s developer) like I had with Joost De Valk (Yoast SEO’s developer) before they’ll fix the SEO damaging features. Come on Michael, get up to date with 2016 SEO (2009 SEO :-)) and remove the section wide robots nofollow meta tags.
Why you should AVOID noindex
Although noindex is not as SEO damaging as nofollow (doesn’t delete link benefit per se), noindex does waste link benefit. When a webpage is noindexed the link benefit flowing through the webpage does NO SEO work (gains no Google SERPs : Search Engine Results Pages) on THAT noindexed webpage!
All good SEO experts know when link benefit flows through a webpage (via links) a proportion of that link benefit is used (spent) on that webpages SERPs (the webpages Google rankings).
It is estimated ~15% of the link benefit (PageRank : PR) flowing through a webpage is used to rank SERPs, it’s the dampening factor (originally 15%) within the original PageRank formula. If a webpage is noindexed the ~15% is still used (spent), BUT the webpage gains no SERPs (no Google traffic, it’s wasted link benefit).
Having the odd page that’s noindex is probably not a big deal, but having section wide robots noindex meta tags on archives like the categories and tags is SEO stupidity! Some WordPress sites could have dozens of wepages not indexed, wasting link benefit which could be used to generate Google traffic, it’s nuts!
The perfect SEO setup is ALL webpages on a website are indexed and target one or more keywords/keyword phrases (SERPs) so ALL the link benefit flowing through the site at least attempts to generates traffic from search engines like Google through relevant SERPs.
Few sites are perfect search engine optimization wise, we have Contact, About, Privacy, Disclaimer, Shopping Basket and other pages that though important to our visitors have little to no SEO value. The robots noindex meta tag can block these pages from being indexed, but noindex does NOT recycle the link benefit (the 15% that’s lost). There is little value in noindexing a page when the link benefit (the ~15%) is wasted, OK the page isn’t indexed, but the link benefit is still used (wasted).
Should I noindex Webpages?
The answer is nearly always no, don’t use noindex.
The first question to ask before noindexing a webpage is what’s the problem with Google indexing it, why does it NEED blocking?
An About page has little SEO value (probably won’t gain Google SERPs), but there’s no harm in it being indexed (there’s no SEO harm). The same is true for Contact, Privacy and Disclaimer type webpages, they probably won’t rank for anything, but there’s no harm in having them indexed.
There’s no SEO damage if Google indexed these webpages, so there’s not a GOOD reason to noindex them.
There are webpages you ideally do not want indexing, Shopping Baskets and affiliate redirect links for example. I noindex an entire folder
/go/ via the robots.txt file because it contains a redirect script which links to affiliate URLs.
Part of my robots.txt file which blocks all search engines from indexing everything under
User-agent: * Disallow: /go/
I don’t want Google to associate the redirect URLs (to Google they are webpages on my site which 302 redirect to an affiliate site). That’s a specific case where a noindex meta tag would make sense: I used the robots.txt file as it was easier to setup.
If a Shopping Basket system isn’t setup correctly it might be indexable under multiple dynamic URLs, if you can’t find a solution to the issue (ideally you’d have a canonical URL to the main Shopping Basket page so only one of the webpages is indexed), a simple noindex robots meta tag makes sense.
When adding a robots noindex meta tag to a page, always ask does indexing this page potentially cause SEO damage? If the answer is yes AND you can’t find an alternative solution, (some solutions recover some of the link benefit which is lost using noindex) noindex it.
Why Do SEO Plugin Developers Include Noindex and Nofollow Features?
There’s a lot of SEO misinformation on the Internet, just look at the noindex and nofollow features within the Yoast WordPress SEO Plugin and the All In One SEO Plugin. Either the SEO plugin developers don’t understand SEO to a level required to build useful SEO tools, or they have realized to market a WordPress SEO plugin to the clueless they have to add these damaging SEO features!
I suspect the latter, they can’t be that SEO clueless to not have read the freely available SEO information about nofollow and noindex: though saying that see the screenshot section, screenshot 5.
If plugin developers remove these damaging SEO features, what’s left?
Title Tag : most themes built today put them in the right sort of order – SEO plugin feature not needed with most themes. Had a use back in pre 2010 when most themes put the blog name in the title tag (usually at the front!) sitewide.
Meta Keywords Tag : Google has never taken the keywords meta tag into account in the ranking algo – SEO plugin feature not needed, in fact a complete waste of time. No WordPress SEO plugin should have added this as a feature. I admit to adding meta keywords features to my themes because that’s what theme users want: though did add information that they are wasting their time over a decade ago.
Meta Description Tag : Google has never taken the description into account in the ranking algo, Google treats the description like an ad – SEO plugin feature not needed, though if you are good at writing ad copy could be useful: I’m rubbish at writing ad copy, so never use the meta description tag. I don’t have an issue with this feature, doesn’t help with Google rankings, but nice to have.
Even newer features related to social networks (Twitter, Facebook etc…) have no direct SEO value. Google ignores social media activity, so being able to easily share content on Facebook etc… aren’t exactly killer SEO features. Again, they are nice to have, but are they really SEO features? I and many SEO experts argue they aren’t SEO features.
XML sitemaps are nice, but Google won’t rank a webpage based on it being in an XML sitemap. To rank in Google a webpage still needs backlinks: I’ve tested this by adding content only linked via XML sitemaps, as you would expect it doesn’t rank well at all.
With noindex and nofollow features it feels like the webmaster is achieving something SEO wise, content is being blocked, only the “good” content is being indexed and we are avoiding duplicate content penalties. Problem is it doesn’t work this way, Google is really good at determining which content to index and rank first. Google can figure out the content on your categories is related to your Posts and Google doesn’t have a problem with it: it’s not duplicate content, copying another websites content is duplicate content and even then Google doesn’t automatically ban the content. For example it’s acceptable to use public domain content from other websites, no one owns public domain content, there’s no “original source”.
Since noindex and nofollow wastes your valuable SEO work don’t use it to try to prevent non-existent duplicate content issues. It does give the clueless webmaster a feature to spend time setting up, which is why I think WordPress SEO plugin developers add it to their plugins 🙁
WordPress SEO Resources
For WordPress SEO tips and tutorials, check out the following articles:
- WordPress SEO Tutorial
- WordPress SEO Tutorial Friendly Permalinks
- WordPress SEO Tutorial Title Tag Optimization
- WordPress SEO Tutorial 301 Redirects
- WordPress SEO Tutorial Anchor Text
- WordPress SEO Tutorial Alt Text
- WordPress SEO Tutorial Related Posts Plugins
- WordPress SEO Tutorial Duplicate Content
- WordPress SEO Tutorial Google XML Sitemaps
- WordPress SEO Comments Plugin
WordPress SEO Plugins
- The Stallion WordPress SEO Plugin Lacks Title and Meta Tags Features
The Stallion WordPress SEO Plugin only deals with protecting link benefit flowing through a site that other WordPress SEO plugins like Yoast and All In One SEO damages. The Yoast WordPress SEO Plugin and the All in One SEO Pack WordPress Plugin title element and meta tag features are if used correctly not SEO damaging.
You can safely use this plugin alongside other WordPress SEO plugins, for it to work correctly you should disable all nofollow and noindex features of other WordPress SEO plugins and turn off their canonical URL features: for Yoast and All In One SEO I’ve added some code to stop them adding their canonical URL code on pages where this plugin adds it’s canonical URLs. Some of the features of the Yoast WordPress SEO Plugin and the All in One SEO Pack WordPress Plugin are useful, it’s the nofollow and noindex options that should be disabled.
- How To Use This WordPress SEO Plugin
Since canonicals do not recover 100% of the link benefit it’s best to avoid using them.
If you think a section of your WordPress site (like monthly archives: they are AWFUL SEO wise) shouldn’t be indexed by Google, ask why does it exist?
If you can not justify a section, remove it. For example I see no good reason for keeping monthly archives, so never add the dated archive widgets including the Calendar Widget: if you don’t add those widgets, there’s no internal links wasting link benefit: no reason to noindex via Yoast SEO or All In One SEO or Stallion Not Index them.
It’s recommended you use the Stallion WordPress SEO Plugin Not Index Options on indexed sections of a website with repeating Title Tags. Title Tags of pages 2,3,4, etc… of a Paged archive set have almost identical Title Tags tend to be:
“Name of Archive – Page 2”
“Name of Archive – Page 3”
These have little SEO value.
WordPress Archives Like:
Home Page Archives
Search Results Archives
Custom Taxonomy Archives
Custom Post Type Archives
Page one of a paged archive will be the only webpage in the set that’s likely to gain SERPs: the Home page for example, first page of a Category for example… Page 2, Page 3 etc… of a Category set are not going to gain traffic, no point indexing them in Google, though you still want them spidered by Google (they link to your deeper posts) and you want the links from them to follow: NEVER nofollow the links from WordPress archives, this is SEO suicide.
It therefore makes sense to add a canonical URL on Pages 2,3,4,5… of an archive set back to the first page for Home page Archives, Dated Archives, Author Archives, Categories, Tags, Search Results, Custom Taxonomy Archives and Custom Post Type Archives simply because pages 2,3,4,5… of an archive set are highly unlikely to generate any Google SERPs.
By adding a canonical URL back to Page 1 you recover most of the link benefit that would be spent on Page 2, Page 3 etc… trying to gain Google SERPs. This recovered link benefit will aid Page 1 in an archive set gaining SERPs: it not only recovers PR, it helps gaining SERPs.
When a Stallion WordPress SEO Plugin Not Index option is set to:
Index ###### Only : Block Paged 2,3,4…
Only page one of the ##### Archive is indexed.
- How Do I Know The SEO Plugin Is Working?
In the old WordPress star rating system (it allowed anonymous star ratings) this plugin had a LOT of 1 star ratings, but it wasn’t broken.
The plugin does all it’s search engine optimization work behind the scenes (you don’t see anything on your website as you view it), so maybe when users activate the plugin and see no obvious changes they think it’s broken or maybe they assume the Stallion SEO plugin will be like the Yoast WordPress SEO Plugin and the All In One SEO Plugin and have options for changing title tags and meta tags etc… and when they don’t find those options assume it’s broken.
To check the Stallion SEO plugin is working on the Stallion Main Options page set “Cleaner Head” to “NOT Cleaner Head”: this adds additional code to the Stallion SEO plugins output so you can see it when viewing the HTML code source.
Go to a relevant part of the site and view source in a browser like Firefox (“Right Click” the page, click “View Source”), for example if you have “Index First Search Result Only : Block Paged 2,3,4…**” set, do a Search on your site and go to Page 2 or higher of the search result, view source and you’ll find code like this:
<!-- Stallion WordPress SEO Plugin 3.0.0 by David Cameron Law http://stallion-theme.co.uk/stallion-wordpress-seo-plugin/ --> <link rel="canonical" href="http://example.com/search/keyword/" /> <!-- Index First Search Result Only : Block Paged 2,3,4...** - Stallion SEO -->
This is a canonical URL to the main search URL with some commented out code above and below so you know it’s working. If Google managed to spider a search on your site (it happens a lot) you’ll find only one page is indexed. Had you set “Block All Search Results” the canonical URL code would look like this:
<!-- Stallion WordPress SEO Plugin 3.0.0 by David Cameron Law http://stallion-theme.co.uk/stallion-wordpress-seo-plugin/ --> <link rel="canonical" href="http://example.com/" /> <!-- Block All Search Results - Stallion SEO -->
And Google etc… wouldn’t index any search results, any SEO benefit will be directed back to home.
If you want to keep your head code clean, on the main options page set “Stallion SEO Plugin Clean Head” to “Cleaner Head”. When “Cleaner Head” is set the output looks like this (harder to see, but cleaner):
<link rel="canonical" href="http://example.com/" />
- Does The Stallion WordPress SEO Plugin compete with the Yoast WordPress SEO Plugin and the All In One SEO Plugin?
Short answer is NO.
I put most of my time into the Stallion Responsive Theme which adds 10 times more search engine optimization to a WordPress site than any combination of current WordPress plugins and themes can achieve. The Stallion Responsive Theme is years ahead of anything else available because you CAN NOT add full search engine optimization to a WordPress site ONLY using a plugin, you have to SEO the theme code as well. There’s too much output that’s part of the themes coding (H1, H2, H3… headings, anchor text of links etc…) so the only way to fully SEO a WordPress site is via it’s theme so my time goes into developing the Stallion Responsive Theme not stand alone WordPress SEO plugins that tinker with the SEO output.
For example the Stallion WordPress SEO Plugin features is a tiny part of the Stallion Responsive Theme options. Stallion Responsive has a dozen options pages with hundreds of options.
Anything added via a plugin can also be added via a theme, the reverse is not true.
For example theme developers have tended to add a sitewide H1 header including the name of the site, this is bad SEO practice, the H1 header should include the main SERP for that page which on blog posts and static pages tends to be the title of the post or page. This is very easy to achieve by editing a themes template files, almost impossible to achieve using a plugin because the H1 code is different for each theme and even if a plugin was developed to overwrite the H1 headers the CSS wouldn’t match the themes CSS and the output wouldn’t look right. If the theme you use adds a sitewide H1 header with the name of the site, change themes.
This is just one example, there’s dozens of theme elements which should be search engine optimized that plugins realistically can’t touch. In comparison changing the title tag and meta tags is low hanging SEO fruit, which is why most WordPress SEO Plugins deal with the title tags and meta tags and not H1 headers, or the anchor text for Read More links or the damaging rel = nofollow attributes added by WordPress core and themes.
It makes so much sense, thanks for all this work Dave.
- Major update of code base, rebuilt the plugin using the WordPress Plugin Boilerplate as it’s base. Much better code base, basically a new plugin using the old option names (will use the old 2.1.1 options). Will make adding new future SEO features easier.
- Added Stallion WordPress SEO Plugin Warnings : indicates any Not Index features that cause SEO damage.
- Added Yoast WordPress SEO Plugin Warnings : indicates any noindex features that cause SEO damage.
- Added All In One WordPress SEO Plugin Warnings : indicates any noindex/nofollow features that cause SEO damage.
- Added Not Index Option for Author Archives
- Added Not Index Option for Custom Taxonomy Archives
- Added Not Index Option for Custom Post Type Archives
- Added Cleaner Head Code option, SEO plugins tend to add a bunch of self promoting crap in the head code, other than for testing the Stallion WordPress SEO Plugin is working you can easily remove it with one option.
- Added language localization code.
- Changed to serialized options (two entries in the database rather than multiple database entries).
- Removed option to not index specific Posts and Pages (it wasn’t user friendly), plan to add a new (better) custom canonical URL feature metabox to Posts and Pages.
- Everything tested under PHP 5.3, PHP 5.4, PHP 5.5, PHP 5.6, PHP 7.0.*, so should work on most servers without issues.
- sanitized all options, hardening security further.
- security fix to prevent possible XSS attack.
- Thanks goes to Joost de Valk (Yoast WordPress SEO Plugin author) for letting me know there was a possible security issue.
- fixed some depreciated code.
- finished the block individual posts/pages feature.
- added canonical support for paged comments.
- Small code change to remove PHP warnings (Use of Undefined Constant), had missed out ” around a few constants.
- First release.