WordPress.org

Ready to get started?Download WordPress

Forums

Pulling hair out over robots.txt (2 posts)

  1. ElSnarkistani
    Member
    Posted 1 year ago #

    Website I'm running is sunnyinkabul.com. I keep getting this "A description for this result is not available because of this site's robots.txt – learn more." message when I try to search the site via Google. How do I fix this?

    I've verified the site through Google, using Google Analytics. That site tells me that most of the pages on my site are blocked from being crawled, and no such page is found when I try to click over from the findingmytribe.me in wordpress.com on the site re-direct. In other words, if I search for a page from that site, it redirects to the new site and domain, but the page can't be found until I search for it in the new site.

  2. Abhishek Ghosh
    Member
    Posted 1 year ago #

    You are using WordPress SEO by Yoast.

    Cleanup your installation table by table to remove plugin specific data, use any other light plugins which people uses for past few years. WordPress SEO by Yoast is a complex plugin, some function often does not work rightly on certain setup.

    Delete domain ownership from Google. Freshly verify using cname or html at root plus meta tag.
    Closer to your issue :

    http://wordpress.org/support/topic/plugin-wordpress-seo-by-yoast-google-webmaster-tools-cannot-parse-sitemapxml?replies=20

    Always create a physical robots.txt at root and properly chmod it to make it writable by WP.

Topic Closed

This topic has been closed to new replies.

About this Topic

Tags