Anyone know why there is no robots.txt file included in WP? Arguably there are a few directories any admin wouldn’t want showing up for security and SEO reasons. A few examples are:
I have found that various engines will index places such as wp-* that one might not want indexed. =(
Any idea on why there is no default robots.txt?
- The topic ‘Robots.txt? Why is it not included by default?’ is closed to new replies.