WordPress.org

Ready to get started?Download WordPress

Forums

[resolved] Discourage search engines from indexing this site option and robots.txt (6 posts)

  1. kenneth.von.rauch
    Member
    Posted 10 months ago #

    Hey guys,

    I am just wondering how the "Discourage search engines from indexing this site' feature works. Here's what I mean. I was under the impression that if you tick the "Discourage search engines from indexing this site" option (in Settings -> Reading>, it'll add the following scripting in robots.txt:

    Disallow: /*

    so that all my site is blocked for search engines. Problem is, once I enable that feature, it does not change my robots.txt file. Now here's my questions. Which file or way that feature uses to discourage search engines from indexing my site?

  2. leejosepho
    Member
    Posted 10 months ago #

    once I enable that feature, it does not change my robots.txt file.

    WordPress only has a virtual robots.txt, and it drops out when you have an actual one. So, I think you will have to remove your own file to get that feature to work.

  3. kenneth.von.rauch
    Member
    Posted 10 months ago #

    Thanks for your reply. Well, I get that part., but the question remains," If I tick the "Discourage search engines from indexing this site" option, how does search engine knowI did it? I mean which file is used to communicate to the search engine?

  4. Lorenzo Orlando Caum
    Member
    Posted 10 months ago #

    @Kenneth, I took a look at this option in WP 3.6.1.

    It adds the following to the webpages:

    <meta name='robots' content='noindex,nofollow' />

  5. kenneth.von.rauch
    Member
    Posted 10 months ago #

    You made my day, Lorenzo. :) That's really how it works. Thanks a lot!!!

  6. ardinent
    Member
    Posted 2 months ago #

    When I use the same feature, it adds

    <meta name='robots' content='noindex,follow' />.

    Any idea how I can change follow to nofollow?

Reply

You must log in to post.

About this Topic