WordPress.org

Support

Support » Plugins and Hacks » Google is blocked by robots.txt after initially dismissing warning from Plugin

Google is blocked by robots.txt after initially dismissing warning from Plugin

  • lance_kidd
    Member

    @lance_kidd

    When I first installed Yoast SEO, I had WP privacy settings set to “ask search engines to not allow crawling site.” The plug-in gave me a warning and I clicked the button to “Don’t bother me about it.”

    So later when I was ready to set up the yoast seo plugin, I went to WP privacy settings and changed the setting to allow all search engines to crawl the site. However, nothing has changed. In Google’s webmaster tools, I use the “Fetch as Google option” and it say’s Access blocked by robots.txt file. Also, the sitemap was successful, but Google says “URL blocked by robots.txt file.

    When I clicked the “Dismiss and leave me alone” warning upon initial install, I think it messed things up. How do I undo this now. Nothing is checked in the yoast SEO plug-in that should be causing anything to be blocked.

    Thanks!

Viewing 1 replies (of 1 total)
  • Within the Yoast SEO plugin settings, there’s a section called “Edit Files”. That page allows you to edit your robots.txt file and .htaccess file.

    Make sure that your robots.txt file has the following line…

    User-agent: *

Viewing 1 replies (of 1 total)
  • The topic ‘Google is blocked by robots.txt after initially dismissing warning from Plugin’ is closed to new replies.