Support » Fixing WordPress » Robots.txt fetch on google webmaster

  • On google webmaster under “current status” of website fwweattorneys.net

    it says there is an error with “Robots.txt fetch”; “Google couldn’t crawl your site because we were unable to access the robots.txt file.robots.txt.”

    Can anyone help me figure out how to solve this problem? Thank you, pfe.

Viewing 8 replies - 1 through 8 (of 8 total)
  • Krishna

    (@1nexus)

    Member

    Just delete the robots.txt and try fetch as Google. The main purpose of Robots.txt file is to block bots. If there is no such file, it means that bots, including Google bots are allowed. If this also do not work, try a robots.txt file as recommended by Google. Are you using any plugins that create robots.txt?

    It seems you are using https://www.attracta.com services? If it’s not necessary, then please delete the robots.txt and try a recommended by google as Krishna said.

    How exactly do I delete robots.txt? I think google recommends to not even have it.

    Krishna

    (@1nexus)

    Member

    You can login to your hosting account or use FTP to delete the robots.txt file that you can find at the root directory.

    You can login to your hosting account or use FTP to delete the robots.txt file that you can find at the root directory.

    … if it is a physical robots.txt at all. It may be generated at run-time the way WordPress itself does it.

    It looks like Attracta may even be integrated with cPanel, which could complicate this.

    flanagan I checked your robots.txt file through http://fwweattorneys.net/robots.txt
    and it is it is completely accessible. Also there aren’t any syntax error in that file. It only have a referenc to sitemap file
    http://cdn.attracta.com/sitemap/2475346.xml.gz
    Even that sitemap is valid. However, there aren’t any URLs other than your site home address in that sitemap.

    I checked your robots.txt file

    I checked it too. I see no reason why that file is causing trouble.

    in your google webmaster there is an option to change and check robots.txt . Check it there for which the bot is allowed . Then upload that file to your backend . Or delete the robots.txt

Viewing 8 replies - 1 through 8 (of 8 total)
  • The topic ‘Robots.txt fetch on google webmaster’ is closed to new replies.