WordPress.org

Ready to get started?Download WordPress

Forums

WordPress SEO by Yoast
[resolved] [Plugin: WordPress SEO] Can't find or modify the Robots.txt file (4 posts)

  1. maxogy
    Member
    Posted 3 years ago #

    Hello there,

    After reading Yoast's post on Affiliate Links and SEO I was interested in blocking the search engines from crawling certain links like that contain /go/ and /xyz/

    So first question is: How do I type it? Disallow: /go/, /xyz/, /etc/ or some other way?

    Now, unfortunately I have not made it that far because I cannot find where or how to modify the Robots.txt file.

    Yoast writes in his WordPress SEO Plugin page:

    Edit your robots.txt and .htaccess
    Editing your robots.txt and .htaccess files has never been this easy: you can edit them straight from the Files menu of the WordPress SEO plugin, so you can easily check and or update those files and keep it all nice and clean.

    I do find the .htaccess under the menu specified in his explanation, but that is all I see. Here's a screenshot

    Please note that I had All in one SEO Pack and Google XML Sitemaps plugins previously active although I have yet to delete them.

    Thanks for your help

  2. Joost de Valk
    Member
    Plugin Author

    Posted 3 years ago #

    Hey,

    if robots.txt doesn't show there it doesn't exist and WordPress can't create it. So, create it, make sure WordPress can write to it, and then it should appear in the plugin :)

    Cheers

  3. maxogy
    Member
    Posted 3 years ago #

    Hey Joost, thanks for your reply

    The robots.txt does seem to exist... if I go to http://mysite.com/robots.txt I do see:

    User-agent: *
    Disallow:

    I did not create this manually since I really don't know how to do it... I was under the impression that your plugin (or AIOSEO or Google XML Sitemaps) created it.

  4. Joost de Valk
    Member
    Plugin Author

    Posted 3 years ago #

    No WordPress "creates" it if you don't have it: it shows this robots.txt to prevent 404 errors. If you create it yourself you can edit it.

Topic Closed

This topic has been closed to new replies.

About this Plugin

About this Topic