WordPress.org

Ready to get started?Download WordPress

Forums

"Site blocked by robots.txt" -- but there's no robots.txt! (5 posts)

  1. mattotoole
    Member
    Posted 6 years ago #

    I've been trying to hook up my sitemap and make it visible to Google. Bu on the "add sitemap" page of Webmaster Tools I'm getting an error saying "URL restricted by robots.txt" -- but I have no robots.txt! I made sure WordPress' privacy setting is checked to allow search engines.

    Apparently there's a phantom robots.txt file somewhere, but I can't figure out where. I've looked through all the directories on the server, checked all my pages for meta robots tags, etc. ???

  2. wesg
    Member
    Posted 6 years ago #

    I also have that problem. If I do not specify robots.txt in my domain root, then subsequent visits redirect to a phantom robot.txt folder.

  3. wesg
    Member
    Posted 6 years ago #

    I solved the problem by adding a robots.txt file, then adding new folders for Google to stop crawling.

  4. Chris_K
    Member
    Posted 6 years ago #

    When you visit Options -> Privacy, what do you have selected?

  5. t3ch33
    Member
    Posted 6 years ago #

Topic Closed

This topic has been closed to new replies.

About this Topic