WordPress.org

Forums

Yoast SEO
Blocked by robots.txt (3 posts)

  1. linxhurricane
    Member
    Posted 2 years ago #

    WordPress SEO version: 1.3.4.1

    I installed the plugin - set the privacy settings in wordpress to allow the search engines to index the site.

    I submitted the sitemaps to webmaster tools

    The home page is showing as being 'blocked by robots.txt' as well as 55 other URLs.

    The page sitemap is showing that 14 URLs have been submitted but only one is indexed.

    The post sitemap is showing that 2 URLs were submitted and one is blocked.

    the robots.txt file shows
    User-agent: *
    Disallow: /wp-admin/
    Disallow: /wp-includes/

    site is http://www.swanriverfinance.com

    http://wordpress.org/extend/plugins/wordpress-seo/

  2. creativepassion
    Member
    Posted 2 years ago #

    I have the exact same problem.

    Google has indexed the site, but it says:
    A description for this result is not available because of this site's robots.txt – learn more.

    (Yoast) WordPress SEO Version 1.3.4.3

    http://www.samfordtowing.com.au

  3. linxhurricane
    Member
    Posted 2 years ago #

    I managed to fix it myself - as there was no response here. and now i can't remember what i did !!

    I think I disabled the xml sitemaps - and installed the google xml sitemap creator.

    I then did something else with the robots.txt file....

    can't remember what though...

Topic Closed

This topic has been closed to new replies.

About this Plugin

  • Yoast SEO
  • Frequently Asked Questions
  • Support Threads
  • Reviews

About this Topic