WordPress.org

Ready to get started?Download WordPress

Forums

How do I disallow bots from my WordPress year Archive Folders (4 posts)

  1. drumat5280
    Member
    Posted 7 years ago #

    How do I disallow robots from spidering my archive folders in WordPress?

    Here is how my URL's are structured:

    http://www.example.com/blog/index.php/2007/04/

    I tried testing this in Webmaster Central, but it returned back with an allow status:

    User-agent: *
    Disallow: /*/2007/$

  2. kalico
    Member
    Posted 7 years ago #

    I have never seen the $ used in a robots.txt file -- but I am no expert either.

    Try this website on creating good robots.txt files:

    http://www.robotstxt.org/

    If you find the right combo, post it here!

    HTH :)

  3. whooami
    Member
    Posted 7 years ago #

    User-agent: *
    Disallow: /blog/2007

    will work just fine.

  4. drumat5280
    Member
    Posted 7 years ago #

    How do I test it to make sure I have done it right?

Topic Closed

This topic has been closed to new replies.

About this Topic