Robots.txt fetch on google webmaster

  • flanagan

    (@flanagan)


    13 years, 7 months ago

    On google webmaster under “current status” of website fwweattorneys.net

    it says there is an error with “Robots.txt fetch”; “Google couldn’t crawl your site because we were unable to access the robots.txt file.robots.txt.”

    Can anyone help me figure out how to solve this problem? Thank you, pfe.

Viewing 8 replies - 1 through 8 (of 8 total)
  • Krishna

    (@1nexus)

    13 years, 7 months ago

    Just delete the robots.txt and try fetch as Google. The main purpose of Robots.txt file is to block bots. If there is no such file, it means that bots, including Google bots are allowed. If this also do not work, try a robots.txt file as recommended by Google. Are you using any plugins that create robots.txt?

    codingpet

    (@codingpet)

    13 years, 7 months ago

    It seems you are using https://www.attracta.com services? If it’s not necessary, then please delete the robots.txt and try a recommended by google as Krishna said.

    Thread Starter flanagan

    (@flanagan)

    13 years, 7 months ago

    How exactly do I delete robots.txt? I think google recommends to not even have it.

    Krishna

    (@1nexus)

    13 years, 7 months ago

    You can login to your hosting account or use FTP to delete the robots.txt file that you can find at the root directory.

    s_ha_dum

    (@apljdi)

    13 years, 7 months ago

    You can login to your hosting account or use FTP to delete the robots.txt file that you can find at the root directory.

    … if it is a physical robots.txt at all. It may be generated at run-time the way WordPress itself does it.

    It looks like Attracta may even be integrated with cPanel, which could complicate this.

    udaraaka

    (@udaraaka)

    13 years, 7 months ago

    flanagan I checked your robots.txt file through http://fwweattorneys.net/robots.txt
    and it is it is completely accessible. Also there aren’t any syntax error in that file. It only have a referenc to sitemap file
    http://cdn.attracta.com/sitemap/2475346.xml.gz
    Even that sitemap is valid. However, there aren’t any URLs other than your site home address in that sitemap.

    s_ha_dum

    (@apljdi)

    13 years, 7 months ago

    I checked your robots.txt file

    I checked it too. I see no reason why that file is causing trouble.

    Gaurav Raghav

    (@gkraghav)

    13 years, 7 months ago

    in your google webmaster there is an option to change and check robots.txt . Check it there for which the bot is allowed . Then upload that file to your backend . Or delete the robots.txt

Viewing 8 replies - 1 through 8 (of 8 total)

The topic ‘Robots.txt fetch on google webmaster’ is closed to new replies.