robots.txt question (3 posts)

  1. DebNCgal
    Posted 6 years ago #

    This may be a dumb question, but... :-)

    I've set up my robots.txt file based off the sample in the WP Codex at: http://codex.wordpress.org/Search_Engine_Optimization_for_WordPress

    However, I've been curious as to why one would allow the indexing of:


    The reason I ask is because I have a photo blog, and I try to be protective of the images on my website. For instance, I definitely concur with disallowing Google-bot Image.

    I'm contemplating NOT allowing the indexing of the /wp-content/uploads. Would there be an unintended negative consequence to doing that?

    Any clarification on why /wp-content/uploads is allowed in the sample robots.txt would be appreciated.

    Thanks so much for any assistance.

    Deb Phillips

  2. Samuel B

    Posted 6 years ago #

    I don't allow any of my /wp-content folder to be indexed
    everything works fine

  3. DebNCgal
    Posted 6 years ago #

    Thank you, samboll. I think I'll follow suit and not allow indexing of the /wp-content/uploads folder, too. I'm still perplexed as to why that folder is specifically allowed in the WP sample robots.txt file.

Topic Closed

This topic has been closed to new replies.

About this Topic