WordPress.org

Ready to get started?Download WordPress

Forums

Robots.txt Disallow WP Blog Pages (9 posts)

  1. apollo24
    Member
    Posted 4 months ago #

    Hi All

    On my wp site I have a blog on a different page rather than homepage.

    Let's say my posts appear on site.com/blog

    So today I have dozens of posts and my site appears on google with site.com/blog/page/2 site.com/blog/page/3 etc.

    My questions is how can I disallow /page/2 /page/3 /page/4 etc. But to allow /blog

    Is it correct to write code like this

    disallow: /blog/page/

    Will it remain /blog open for google bot?

    Thank you

  2. justingreerbbi
    Member
    Posted 4 months ago #

    I am not an SEO expert but wouldn't you want to get as mush traction as you possibly could?

    The links being indexes archive listings. If you do not want to include your archive listing for indexing then you could place a NOFOLLOW in the header.php

    Something like

    if(is_archive()){
     echo '<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">';
    }
  3. apollo24
    Member
    Posted 4 months ago #

    Hi justingreerbbi

    Looks like these pages are not configured as archive pages. I use All in One SEO and I checked NOINDEX for archive pages, but these pages are still being crawled by Google.

  4. justingreerbbi
    Member
    Posted 4 months ago #

    Yes you are correct. Sorry, since you are using a the page "blog" to list your archive it is considered a "page"

    if(is_page() && is_paged()){
     echo '<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">';
    }

    I really recommend not adding a "nofollow, noindex" to this but that is just me.

  5. apollo24
    Member
    Posted 4 months ago #

    Does it mean all my pages within the site will be noindexed, doesn't?

  6. justingreerbbi
    Member
    Posted 4 months ago #

    No

    &&

    is a conditional statement which must match both conditions.

    You could be more specific and do

    if(is_page('blog') && is_paged()){
     echo '<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">';
    }
  7. apollo24
    Member
    Posted 4 months ago #

    Great, I'll try it.

    Why you think it is not recommended to nonindex these pages? They are pure duplicate content.

    Thanks a lot.

  8. justingreerbbi
    Member
    Posted 4 months ago #

    Just my preference. Are you displaying all content in the listings or just excerpt?

  9. apollo24
    Member
    Posted 4 months ago #

    Only excerpt

Reply

You must log in to post.

About this Topic

Tags

No tags yet.