WordPress.org

Support

Support » How-To and Troubleshooting » [Resolved] Robots.txt, does this look OK to you

[Resolved] Robots.txt, does this look OK to you

  • hi,
    little bit frustrtated with this, do a new site only to have no descriptive text appear in the serp’s.

    I have site set to be indexed by search bots in WP,
    Google is saying “A description for this result is not available because of this site’s robots.txt – learn more”

    My robots.txt file and I can see this in browser when i look at http://www.mysite.com/robots.txt

    User-agent: *
    Allow: /
    Disallow: /cg-bin
    Disallow: /webalizer
    Disallow: /wp-admin
    Disallow: /oldside
    Disallow: /wp-includes
    Disallow: /wp-content/plugins/
    Disallow: /wp-content/cache/
    Disallow: /wp-content/themes/

    Sitemap: http://www.mysite.com/sitemap.xml

    Q1) Is my robots.txt making sense
    Q2) Is this just a waiting issue until google crawls over site again, Patience is a virtue and all that 😉

    thanks
    T

Viewing 2 replies - 1 through 2 (of 2 total)
Viewing 2 replies - 1 through 2 (of 2 total)
  • The topic ‘[Resolved] Robots.txt, does this look OK to you’ is closed to new replies.