WordPress.org

Ready to get started?Download WordPress

Forums

Robots.txt fetch on google webmaster (9 posts)

  1. flanagan
    Member
    Posted 1 year ago #

    On google webmaster under "current status" of website fwweattorneys.net

    it says there is an error with "Robots.txt fetch"; "Google couldn't crawl your site because we were unable to access the robots.txt file.robots.txt."

    Can anyone help me figure out how to solve this problem? Thank you, pfe.

  2. Krishna
    Volunteer Moderator
    Posted 1 year ago #

    Just delete the robots.txt and try fetch as Google. The main purpose of Robots.txt file is to block bots. If there is no such file, it means that bots, including Google bots are allowed. If this also do not work, try a robots.txt file as recommended by Google. Are you using any plugins that create robots.txt?

  3. codingpet
    Member
    Posted 1 year ago #

    It seems you are using https://www.attracta.com services? If it's not necessary, then please delete the robots.txt and try a recommended by google as Krishna said.

  4. flanagan
    Member
    Posted 1 year ago #

    How exactly do I delete robots.txt? I think google recommends to not even have it.

  5. Krishna
    Volunteer Moderator
    Posted 1 year ago #

    You can login to your hosting account or use FTP to delete the robots.txt file that you can find at the root directory.

  6. s_ha_dum
    Member
    Posted 1 year ago #

    You can login to your hosting account or use FTP to delete the robots.txt file that you can find at the root directory.

    ... if it is a physical robots.txt at all. It may be generated at run-time the way WordPress itself does it.

    It looks like Attracta may even be integrated with cPanel, which could complicate this.

  7. udaraaka
    Member
    Posted 1 year ago #

    flanagan I checked your robots.txt file through http://fwweattorneys.net/robots.txt
    and it is it is completely accessible. Also there aren't any syntax error in that file. It only have a referenc to sitemap file
    http://cdn.attracta.com/sitemap/2475346.xml.gz
    Even that sitemap is valid. However, there aren't any URLs other than your site home address in that sitemap.

  8. s_ha_dum
    Member
    Posted 1 year ago #

    I checked your robots.txt file

    I checked it too. I see no reason why that file is causing trouble.

  9. Gaurav Raghav
    Member
    Posted 1 year ago #

    in your google webmaster there is an option to change and check robots.txt . Check it there for which the bot is allowed . Then upload that file to your backend . Or delete the robots.txt

Topic Closed

This topic has been closed to new replies.

About this Topic