For whatever reason, when I visit http://www.example.com/robots.txt it displays the following:
User-agent: * Disallow:
Yet, the file robots.txt does not exist, and in privacy settings I have…
“I would like my blog to be visible to everyone, including search engines (like Google, Sphere, Technorati) and archivers” selected, and “I would like to block search engines, but allow normal visitors” deselected.
The latter was previously selected, but I changed this, and upgraded from 2.3.3 to 2.6.
I can’t work out how to get rid of this robots.txt file…
Please help, this site is no longer listed in google because of this issue.
- The topic ‘unwanted robots.txt’ is closed to new replies.