How can I exclude directories if I cant have a robots.txt file?
I need to exclude sensitive folders but your instructions say that I cannot have a robots.txt file in my root folder.
I had to add one to exclude the onslaught of bots I’m receiving that are trying to access admin etc – does this mean that the plugin isn’t going to generate the sitemap correctly now?
Please reply asap – my site is getting way more traffic than I expected and I need to address this as quickly as I can.
- The topic ‘How can I exclude directories if I cant have a robots.txt file?’ is closed to new replies.