Support » Fixing WordPress » Robots.txt Disallow WP Blog Pages

  • Hi All

    On my wp site I have a blog on a different page rather than homepage.

    Let’s say my posts appear on site.com/blog

    So today I have dozens of posts and my site appears on google with site.com/blog/page/2 site.com/blog/page/3 etc.

    My questions is how can I disallow /page/2 /page/3 /page/4 etc. But to allow /blog

    Is it correct to write code like this

    disallow: /blog/page/

    Will it remain /blog open for google bot?

    Thank you

Viewing 8 replies - 1 through 8 (of 8 total)
  • Moderator justingreerbbi

    (@justingreerbbi)

    Forum Moderator

    I am not an SEO expert but wouldn’t you want to get as mush traction as you possibly could?

    The links being indexes archive listings. If you do not want to include your archive listing for indexing then you could place a NOFOLLOW in the header.php

    Something like

    if(is_archive()){
     echo '<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">';
    }

    Hi justingreerbbi

    Looks like these pages are not configured as archive pages. I use All in One SEO and I checked NOINDEX for archive pages, but these pages are still being crawled by Google.

    Moderator justingreerbbi

    (@justingreerbbi)

    Forum Moderator

    Yes you are correct. Sorry, since you are using a the page “blog” to list your archive it is considered a “page”

    if(is_page() && is_paged()){
     echo '<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">';
    }

    I really recommend not adding a “nofollow, noindex” to this but that is just me.

    Does it mean all my pages within the site will be noindexed, doesn’t?

    Moderator justingreerbbi

    (@justingreerbbi)

    Forum Moderator

    No

    &&

    is a conditional statement which must match both conditions.

    You could be more specific and do

    if(is_page('blog') && is_paged()){
     echo '<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">';
    }

    Great, I’ll try it.

    Why you think it is not recommended to nonindex these pages? They are pure duplicate content.

    Thanks a lot.

    Moderator justingreerbbi

    (@justingreerbbi)

    Forum Moderator

    Just my preference. Are you displaying all content in the listings or just excerpt?

    Only excerpt

Viewing 8 replies - 1 through 8 (of 8 total)
  • The topic ‘Robots.txt Disallow WP Blog Pages’ is closed to new replies.