handfordrCan someone clarify an issue I have configuring robots.txt so search engines do not find duplicate content
In best practise documentation they all state you do something like:
Disallow: /blog/wp-admin
Disallow: /blog/wp-includes
Disallow: /blog/wp-content/plugins
Disallow: /blog/wp-content/cache
Disallow: /blog/wp-content/themes
Disallow: /blog/category/*
Disallow: /blog/trackback
Disallow: /blog/feed
Disallow: /blog/comments
Allow: /blog/wp-content/uploads
Understand the logic of these maps but assuming the only logical page I want the search engines to find is the actual post page itself shouldn't I also have something to stop crawling the tags urls, something like:
Disallow: /blog/tag/*
