robots.txt and SEO

  • jtt89

    (@jtt89)


    12 years ago

    I was told to use this code to avoid issues of duplicate content with Google, but at the same time I am using Categories as my main navigation and I think that it would be better to get it indexed, since I am using a lot of keywords in category names (actually 100% of them are keywords). I was also going to add Categort descriptions and display them on pages to further improve SEO.
    So basically I am wondering if there is a way to get around that and spider categories as navigation and allow spidering category pages (with h1 tags that I want to create) and at the same time avoid issues related to duplicate content etc…

    User-agent: *
    Disallow: /cgi-bin
    Disallow: /wp-admin
    Disallow: /wp-includes
    Disallow: /wp-content/plugins
    Disallow: /wp-content/cache
    Disallow: /wp-content/themes
    Disallow: /trackback
    Disallow: /comments
    Disallow: /category/*/*
    Disallow: /tag/
    Disallow: */trackback
    Disallow: */comments

    Thank you.

Viewing 4 replies - 1 through 4 (of 4 total)
  • soumen123

    (@soumen123)

    12 years ago

    Don’t tag a single post for multiple categories…one post one category. You can use multiple tags for a single post, but use only one category.

    [ sig moderated ]

    Thread Starter jtt89

    (@jtt89)

    12 years ago

    Thanks. Another question would be if there is a way to temporary exclude front page posts (lets say 6) from showing on category pages (and keep it changing as new posts are written).

    I know that there is an “excerpt” plugin, but showing full posts would be a lot better.

    Thread Starter jtt89

    (@jtt89)

    12 years ago

    …this would be probably rel=”canonical” pointing to categories on the homepage…

    Moderator Ipstenu (Mika Epstein)

    (@ipstenu)

    🏳️‍🌈 Advisor and Activist

    12 years ago

    I was told to use this code to avoid issues of duplicate content with Google

    Don’t need to worry about it. Google uses rel="canonical" and WP points that to what you define in your permalinks.

Viewing 4 replies - 1 through 4 (of 4 total)
  • The topic ‘robots.txt and SEO’ is closed to new replies.