WordPress.org

Ready to get started?Download WordPress

Forums

WordPress SEO by Yoast
How to edit robots.txt file? (7 posts)

  1. seointern
    Member
    Posted 2 years ago #

    How do I edit my robots.txt file within this plugin? I see on this page: http://yoast.com/wordpress/seo/ at the bottom that he mentions it's available in the Files Menu. At the moment, I'm only seeing the .htaccess file available? Where do I go to edit the robots.txt?

    http://wordpress.org/extend/plugins/wordpress-seo/

  2. Edward Seid
    Member
    Posted 2 years ago #

    It should be just below .htaccess
    If you don't see it, maybe you don't have robots.txt in the root.

  3. seointern
    Member
    Posted 2 years ago #

    Ya, I don't see it. Here is my robots.txt file: http://www.discountqueens.com/robots.txt

    What is an easy way to find out where the file is, if it's not in the root?

  4. Edward Seid
    Member
    Posted 2 years ago #

    I just checked mine again. After selecting SEO > Edit files, robots.txt appears first, .htaccess appears second.

    I see that you installed WordPress in the root so everything should work fine. Odd. Are you using the latest version of WordPress SEO?

  5. seointern
    Member
    Posted 2 years ago #

    Yes, here is what I see: http://screencast.com/t/B74WX2fhxP9m

    I am using the version 1.1.5 of the plugin, is there a newer one? As for our wordpress install we're still using version 3.2.1.

    Is there another place I should look?

  6. Edward Seid
    Member
    Posted 2 years ago #

    Very strange. I'm sorry, but I don't have a clue :/

  7. cmccrone
    Member
    Posted 1 year ago #

    I can help you out as this issue DROVE ME NUTS.

    1. I didnt have a Robots.txt
    2. I had Yoast installed
    3. Im pretty sure it created a Robots.txt even though it doesnt exist in my root (.com/here)
    4. My Google webmaster tools shows this

    User-agent: *
    Disallow: /wp-admin/
    Disallow: /wp-includes/
    Disallow: /cgi-bin
    Disallow: /wp-admin
    Disallow: /wp-includes
    Disallow: /wp-content/plugins
    Disallow: /plugins
    Disallow: /wp-content/cache
    Disallow: /wp-content/themes
    Disallow: /trackback
    Disallow: /feed
    Disallow: /comments
    Disallow: /category/*/*
    Disallow: */trackback
    Disallow: */feed
    Disallow: */comments
    Disallow: /*?*
    Disallow: /*?
    Allow: /wp-content/uploads
    Allow: /assets

    THIS IS HOW YOU FIX THIS ISSUE. Create a Robots.txt and only add this (or whatever you want. I keep it simple and rank well/have no issues at all. You CANT trick google)

    User-Agent: *
    Disallow: /wp-content/plugins/

    1. login to wordpress
    2. Click SEO in your side toolbar (Yoast WordPress Plugin settings).
    3. Go to edit files under SEO (in the side toolbar)

    And now you have the option to edit your Robots.txt file.

    :)

Topic Closed

This topic has been closed to new replies.

About this Plugin

About this Topic