WordPress.org

Support

Support » How-To and Troubleshooting » Robots.txt problem Need help

Robots.txt problem Need help

  • Hi,
    I am new at WordPress. My site is working fine from last few days and Google crawls all my posts but now i got red mark on robots.txt in webmaster and my last few posts doesn’t crawl. I also check privacy and select “Allow search engines to index this site.” I have also installed Sitemap.xml plugin and select make virtual robots.txt. Now i have installed another Special plugin for Robots.txt named as “WP Robots Txt”. When i open mysite.com/robots.txt it says

    # Added by SEO Ultimate’s Link Mask Generator module
    User-agent: *
    Disallow: /go/
    # End Link Mask Generator output

    User-agent: *
    Disallow: /wp-admin/
    Disallow: /wp-includes/
    Sitemap: http://www.mysite.com/sitemap.xml.gz

    I want Google to crawl my site what should i do now.

Viewing 8 replies - 1 through 8 (of 8 total)
Viewing 8 replies - 1 through 8 (of 8 total)
  • The topic ‘Robots.txt problem Need help’ is closed to new replies.