Google couldn't crawl your site error (5 posts)

  1. djramix123
    Posted 3 years ago #

    Ok So I just finished building a new website

    Using WordPress of course.

    I submitted the site to Google webmasters tool and I am getting this error:

    Google couldn't crawl your site because we were unable to access the robots.txt file.robots.txt.

    I don't know what to do, also I was trying to lookup up the robots.txt file on my server through FTP-Zilla but I don't even see this file on the root...

    Thanks for you help guys!!

  2. unarece
    Posted 3 years ago #

    yo tengo ese mismo problema, ayuda por favor....

  3. esmi
    Forum Moderator
    Posted 3 years ago #

    @unarece: It is impolite to interrupt another poster's ongoing thread with a question of your own. It causes significant problems for the forum's volunteers and prevents us from being able to track issues by topic. Please post your own topic.

    Also, these are English language forums, so please use English. Alternatively, use a WP support forum in your language.

  4. djramix123
    Posted 3 years ago #

    Ok let me give some updated info.
    I just realized that robots.txt does work if I go to

    I did a test under Webmasters Tool --> Health --> Blocked URL's and it came up like:

    Googlebot - Allowed - Detected as a directory; specific files may have different restrictions

    but still it is not working...

  5. buch bewertung
    Posted 3 years ago #

    The file looks ok to me.

    If you went into Google webmaster tools and did fetch as a Google bot it has to be ok.

    Perhaps you have to wait some time.

Topic Closed

This topic has been closed to new replies.

About this Topic