Support » Everything else WordPress » Google couldn't crawl your site error

  • djramix123


    Ok So I just finished building a new website

    Using WordPress of course.

    I submitted the site to Google webmasters tool and I am getting this error:

    Google couldn’t crawl your site because we were unable to access the robots.txt file.robots.txt.

    I don’t know what to do, also I was trying to lookup up the robots.txt file on my server through FTP-Zilla but I don’t even see this file on the root…

    Thanks for you help guys!!

Viewing 4 replies - 1 through 4 (of 4 total)
  • unarece


    yo tengo ese mismo problema, ayuda por favor….



    Forum Moderator

    @unarece: It is impolite to interrupt another poster’s ongoing thread with a question of your own. It causes significant problems for the forum’s volunteers and prevents us from being able to track issues by topic. Please post your own topic.

    Also, these are English language forums, so please use English. Alternatively, use a WP support forum in your language.



    Ok let me give some updated info.
    I just realized that robots.txt does work if I go to

    I did a test under Webmasters Tool –> Health –> Blocked URL’s and it came up like:

    Googlebot – Allowed – Detected as a directory; specific files may have different restrictions

    but still it is not working…

    buch bewertung


    The file looks ok to me.

    If you went into Google webmaster tools and did fetch as a Google bot it has to be ok.

    Perhaps you have to wait some time.

Viewing 4 replies - 1 through 4 (of 4 total)
  • The topic ‘Google couldn't crawl your site error’ is closed to new replies.