WordPress.org

Ready to get started?Download WordPress

Forums

[resolved] Robots.txt, does this look OK to you (3 posts)

  1. tom182
    Member
    Posted 1 year ago #

    hi,
    little bit frustrtated with this, do a new site only to have no descriptive text appear in the serp's.

    I have site set to be indexed by search bots in WP,
    Google is saying "A description for this result is not available because of this site's robots.txt – learn more"

    My robots.txt file and I can see this in browser when i look at http://www.mysite.com/robots.txt

    User-agent: *
    Allow: /
    Disallow: /cg-bin
    Disallow: /webalizer
    Disallow: /wp-admin
    Disallow: /oldside
    Disallow: /wp-includes
    Disallow: /wp-content/plugins/
    Disallow: /wp-content/cache/
    Disallow: /wp-content/themes/

    Sitemap: http://www.mysite.com/sitemap.xml

    Q1) Is my robots.txt making sense
    Q2) Is this just a waiting issue until google crawls over site again, Patience is a virtue and all that ;-)

    thanks
    T

  2. Alain Lankers
    Member
    Posted 1 year ago #

    Yes this looks like a good robots.txt. Just wait until Google spiders your site again.

  3. tom182
    Member
    Posted 1 year ago #

    :)

    thanks

Topic Closed

This topic has been closed to new replies.

About this Topic

Tags

No tags yet.