WordPress.org

1. dsconstructs
   Member
   Posted 8 months ago #

   Getting a message from WMT that my robots.txt is blocking some important page. This is my robots.txt file

   User-agent: *
   Disallow: /wp-content/themes
   Disallow: /wp-content/plugins
   Disallow: /plugins/*
   Disallow: /page/*/
   Disallow: /category/*/*
   Disallow: /wp-admin/
   Disallow: /blog-archives
   Disallow: /author/
   Disallow: /?*
   Disallow: /*?*
   Disallow: /trackback
   Disallow: /xmlrpc.php?rsd
   Disallow: /cgi-bin/
   Disallow: /wp-comments*

   Sitemap: http://donsheppardhomerepairs.com/sitemap.xml.gz

   I can't figure out how it's blocking my page(s)
   It's a small site, and it did drop from 17 pages indexed to 15.
   Any advice?

2. richarduk
   Member
   Posted 8 months ago #

   Is this a wind-up?

   As far as I know, these two at least are pointless since they're internal paths.

   Disallow: /wp-content/themes
   Disallow: /wp-content/plugins

   All I would suggest is get rid of that long list and just start again. Get the urls of the pages or categories that you don't want, and do the disallow stuff with them.

3. dsconstructs
   Member
   Posted 8 months ago #

   It's a legit newbie question. I was given the robots.txt to use and it worked fine for several months til just the last couple days.

   I'll try clearing it out and starting more basic.
   Thank you

Reply

You must log in to post.