WordPress.org

Do I need a robots.txt file?

All well-behaved robots (which includes all search engine spiders) will request http://example.com/robots.txt before doing anything else. If you wish to allow all robots access to your entire site, then you do not need this plugin.

You need robots.txt (and therefore this plugin) to achieve any of the following goals:

• Ban a specific robot (e.g. don't allow MSN Search to crawl your site)
• Ban all but one robot (allow only Google to index your site)
• Ban robots from indexing a particular part of your site (e.g. preventing indexing of feeds and trackback URLs)
• Ban robots from indexing any of your site

You can find much more information in the robotstxt.org FAQ. You might also want to peruse some tips for optimizing your blog's rank in Google (though I do not necessarily endorse these tips).

I have a question that isn't addressed here.

You may ask questions by posting a comment to the KB Robots.txt plugin page. (If you post your support questions as comments below, I probably won't see them.)