WordPress, by default, includes a simple robots.txt file that’s dynamically generated from within the WP application. This is great! but maybe you want to change the content.
Enter WP Robots Txt, a plugin that adds an additional field to the “Reading” admin page where you can do just that.
- I totally screwed up my `robots.txt` file. How can I restore the default version?
Delete all the content from the Robots.txt Content field and save the privacy options.
- Could I accidently block all search bots with this?
Yes. Be careful! That said,
robots.txtfiles are suggestions. They don’t really block bots as much as they suggest that bots don’t crawl portions of a site. That’s why the options on the Privacy Settings page say “Ask search engines not to index this site.”
- Where can I learn more about `robots.txt` files?
Simple plugin that helped me to fix a js. problem in Google search
I am using this plugin to overcome my robots.txt problem. This is very useful plugin. http://www.gemstonesprices.in
- Initial version
- Moved the settings field “officially” to the reading page
- General code clean up