robots.txt issues

  • danohawkaz

    (@danohawkaz)


    11 years, 2 months ago

    My site is excluded from Google’s search by keywords due to a robots.txt issue. I have re-done my robots file- checked it and still I get this notice under the site link: Dark Horse Media
    http://www.darkhorsemedia.com/
    A description for this result is not available because of this site’s robots.txt

    I’ve removed the file- re-done it and still I have this message. What am I doing wrong? I’m using a custom theme- but it was working before we did some site improvements.

    Currently I have this robots.txt file in the root of my server- but it wasn’t working before I uploaded this anyway. Any help appreciated!
    Line 1 User-agent: *
    Line 2 Disallow: /cgi-bin
    Line 3 Disallow: /wp-admin
    Line 4 Disallow: /wp-includes
    Line 5 Disallow: /wp-content
    Line 6 #Disallow: /show-error-
    Line 7 #Disallow: /*/feed
    Line 8 #Disallow: /*/feed/
    Line 9 #Disallow: /private
    Line 10 #Disallow:User-agent: Mediapartners-Google
    Line 11 #Disallow:User-agent: Adsbot-Google
    Line 12 #Disallow:User-agent: Googlebot-Image
    Line 13 #Disallow:User-agent: Googlebot-Mobile

    http://wordpress.org/extend/plugins/post/

Viewing 1 replies (of 1 total)
Viewing 1 replies (of 1 total)
  • The topic ‘robots.txt issues’ is closed to new replies.