Disallowing Permalink'd page in robots.txt

Viewing 3 replies - 1 through 3 (of 3 total)
  • tomontoast

    (@tomontoast)

    13 years, 7 months ago

    That is correct but this will also disallow robots from accessing pages in the same structure as ‘/thisisapage1/’ eg ‘/thisisapage1/foo/’ will also be disallowed

    Thread Starter robertallen

    (@robertallen)

    13 years, 7 months ago

    Thank you very much!

    Should I also disallow /tag/ and /category/ ? I’m guessing that helps to avoid duplicate content as the blog posts already have their own URL’s, right?

    tomontoast

    (@tomontoast)

    13 years, 7 months ago

    I’m no SEO expert, in my opinion search engines recognise tag and category pages and won’t penalise you for using them so I wouldn’t worry.

Viewing 3 replies - 1 through 3 (of 3 total)

The topic ‘Disallowing Permalink'd page in robots.txt’ is closed to new replies.