I was told search engines don't like duplicate content, well I have quadruple content! Posts from my blog show up multiple times in Google search results. For example a single post will be indexed as:
It seems like the only one I would want indexed is #4 which is how the url is for each single post page. Is that correct?
Or would it be #2, which is the whole blog? The issue with #2 is that at /blog, older posts are no longer on the page.
With advice from another thread, I put a simple robots.txt file on my site to do this:
I found the following recommended robots.txt file at askapache.com. Does anyone see any problem with adding its content to my robots.txt file?
Will it work or is it overkill? It gets confusing for me, but one potential problem I see is the following robots.txt file disallows "/category/" but because all my single page posts have a url like #4, above, I probably should not disallow "/category/"? And trackbacks for posts are the same as #4 with "/trackback/" appended.
Apache recommended robots.txt for WordPress 2.+