Problem with Robot txt.

Hi all,

I’m quite new to this so I hope someone can help. I have a WP site for a client. I recovered a lot of the original content from the way back machine. It’s a reasonable size about 150 pages. A lot of the content is very similar but useful for users when looking for the area they live in. As I understand, having all the pages indexed would cause 2 problems. Duplicate content penalty and over optimzation keywords penalty. I therefore decided to only let the engines crawl about 30 pages of new unique content I have written that is not over optimized. I use the WP Yoast plug in and on the advanced setting used the no follow. The robot txt. has been inserted under the head of all the pages. I asked google for a fetch on webmaster tools. It is still indexing all the pages that I’m asking it not to, why? Am I doing this right? If anyone could help I’d be really grateful.

Thanks

Jim

The link is http://alldublinplumbers.com/ I have put the no follow in to all the individually named towns/areas pages.