Robots.txt blocking important page?

  • dsconstructs

    (@dsconstructs)


    14 years, 8 months ago

    Getting a message from WMT that my robots.txt is blocking some important page. This is my robots.txt file

    User-agent: *
    Disallow: /wp-content/themes
    Disallow: /wp-content/plugins
    Disallow: /plugins/*
    Disallow: /page/*/
    Disallow: /category/*/*
    Disallow: /wp-admin/
    Disallow: /blog-archives
    Disallow: /author/
    Disallow: /?*
    Disallow: /*?*
    Disallow: /trackback
    Disallow: /xmlrpc.php?rsd
    Disallow: /cgi-bin/
    Disallow: /wp-comments*

    Sitemap: http://donsheppardhomerepairs.com/sitemap.xml.gz

    I can’t figure out how it’s blocking my page(s)
    It’s a small site, and it did drop from 17 pages indexed to 15.
    Any advice?

Viewing 2 replies - 1 through 2 (of 2 total)
  • richarduk

    (@richarduk)

    14 years, 8 months ago

    Is this a wind-up?

    As far as I know, these two at least are pointless since they’re internal paths.

    Disallow: /wp-content/themes
    Disallow: /wp-content/plugins

    All I would suggest is get rid of that long list and just start again. Get the urls of the pages or categories that you don’t want, and do the disallow stuff with them.

    Thread Starter dsconstructs

    (@dsconstructs)

    14 years, 8 months ago

    It’s a legit newbie question. I was given the robots.txt to use and it worked fine for several months til just the last couple days.

    I’ll try clearing it out and starting more basic.
    Thank you

Viewing 2 replies - 1 through 2 (of 2 total)

The topic ‘Robots.txt blocking important page?’ is closed to new replies.