WordPress.org

Forums

[resolved] Discourage search engines from indexing this site option and robots.txt (7 posts)

  1. kenneth.von.rauch
    Member
    Posted 1 year ago #

    Hey guys,

    I am just wondering how the "Discourage search engines from indexing this site' feature works. Here's what I mean. I was under the impression that if you tick the "Discourage search engines from indexing this site" option (in Settings -> Reading>, it'll add the following scripting in robots.txt:

    Disallow: /*

    so that all my site is blocked for search engines. Problem is, once I enable that feature, it does not change my robots.txt file. Now here's my questions. Which file or way that feature uses to discourage search engines from indexing my site?

  2. leejosepho
    Member
    Posted 1 year ago #

    once I enable that feature, it does not change my robots.txt file.

    WordPress only has a virtual robots.txt, and it drops out when you have an actual one. So, I think you will have to remove your own file to get that feature to work.

  3. kenneth.von.rauch
    Member
    Posted 1 year ago #

    Thanks for your reply. Well, I get that part., but the question remains," If I tick the "Discourage search engines from indexing this site" option, how does search engine knowI did it? I mean which file is used to communicate to the search engine?

  4. Lorenzo Orlando Caum
    Member
    Posted 1 year ago #

    @Kenneth, I took a look at this option in WP 3.6.1.

    It adds the following to the webpages:

    <meta name='robots' content='noindex,nofollow' />

  5. kenneth.von.rauch
    Member
    Posted 1 year ago #

    You made my day, Lorenzo. :) That's really how it works. Thanks a lot!!!

  6. ardinent
    Member
    Posted 10 months ago #

    When I use the same feature, it adds

    <meta name='robots' content='noindex,follow' />.

    Any idea how I can change follow to nofollow?

  7. raven
    Member
    Posted 7 months ago #

    @ardinent

    You can edit the meta tag by hand in the Dashboard:

    Appearance > Editor
    On the right, select the Header file (header.php)

    Find the tag in the code, make your changes, and click the blue Update File button.

    Note that if you are using a theme you downloaded (rather than a custom or child theme you built yourself) any changes you make to the theme template files will be overridden when you update the theme.

    To avoid this problem, you can make a child theme and make your changes to header.php there.

Topic Closed

This topic has been closed to new replies.

About this Topic