Ready to get started?Download WordPress


Google couldn't crawl your site error (5 posts)

  1. djramix123
    Posted 1 year ago #

    Ok So I just finished building a new website

    Using WordPress of course.

    I submitted the site to Google webmasters tool and I am getting this error:

    Google couldn't crawl your site because we were unable to access the robots.txt file.robots.txt.

    I don't know what to do, also I was trying to lookup up the robots.txt file on my server through FTP-Zilla but I don't even see this file on the root...

    Thanks for you help guys!!

  2. unarece
    Posted 1 year ago #

    yo tengo ese mismo problema, ayuda por favor....

  3. esmi
    Forum Moderator
    Posted 1 year ago #

    @unarece: It is impolite to interrupt another poster's ongoing thread with a question of your own. It causes significant problems for the forum's volunteers and prevents us from being able to track issues by topic. Please post your own topic.

    Also, these are English language forums, so please use English. Alternatively, use a WP support forum in your language.

  4. djramix123
    Posted 1 year ago #

    Ok let me give some updated info.
    I just realized that robots.txt does work if I go to

    I did a test under Webmasters Tool --> Health --> Blocked URL's and it came up like:

    Googlebot - Allowed - Detected as a directory; specific files may have different restrictions

    but still it is not working...

  5. buch bewertung
    Posted 1 year ago #

    The file looks ok to me.

    If you went into Google webmaster tools and did fetch as a Google bot it has to be ok.

    Perhaps you have to wait some time.

Topic Closed

This topic has been closed to new replies.

About this Topic