robots.txt how to exclude a Page being indexed and other robots.txt questions (3 posts)

  1. jrm213
    Posted 8 years ago #


    I looked through the forums regarding robots.txt and I see a bunch of them posted and have some questions regarding why, how, and what.

    1. If I have a page with a slug of "info" and I didn't want it to be indexed how would I do that, like this?:
    Disallow: /info

    2. If I Disallow: /wp-content/ wont all my "pages" be not indexed? Or does that just tell it not to index any of the actual files like index.php in those directories?

    3. If I have a custom homepage called home.php won't Disallow: /wp-content/ make my homepage not be indexed? If so how do I make it so everything except the homepage is not indexed?

    Thanks for your help.

  2. 1. Yes.

    2. No. Remember that robots only see your URLs, not the actual location of things. Disallowing /wp-content/ will prevent them from indexing anything with an actual wp-content URL. This includes bits of your theme (CSS file) and theme images. It will also include any images you post using the normal image uploader (since those go to wp-content/uploads/ by default).

    3. No. The URL is important, not the location of the document. Disallow: /wp-content/ will make anything with a URL of http://example.com/wp-content/whatever not get indexed. The homepage does not have a /wp-content/ URL.

  3. jrm213
    Posted 8 years ago #

    Thanks Otto42, that information is very helpful!

Topic Closed

This topic has been closed to new replies.

About this Topic