Google is blocked by robots.txt after initially dismissing warning from Plugin (2 posts)

  1. lance_kidd
    Posted 4 years ago #

    When I first installed Yoast SEO, I had WP privacy settings set to "ask search engines to not allow crawling site." The plug-in gave me a warning and I clicked the button to "Don't bother me about it."

    So later when I was ready to set up the yoast seo plugin, I went to WP privacy settings and changed the setting to allow all search engines to crawl the site. However, nothing has changed. In Google's webmaster tools, I use the "Fetch as Google option" and it say's Access blocked by robots.txt file. Also, the sitemap was successful, but Google says "URL blocked by robots.txt file.

    When I clicked the "Dismiss and leave me alone" warning upon initial install, I think it messed things up. How do I undo this now. Nothing is checked in the yoast SEO plug-in that should be causing anything to be blocked.


  2. Anthony Hortin
    Posted 4 years ago #

    Within the Yoast SEO plugin settings, there's a section called "Edit Files". That page allows you to edit your robots.txt file and .htaccess file.

    Make sure that your robots.txt file has the following line...

    User-agent: *

Topic Closed

This topic has been closed to new replies.

About this Topic