BIG robots.txt problem :( !

  • justbobbsap

    (@justbobbsap)


    10 years ago

    Hi ya guys!

    I will work on making this descriptive so my chances of getting it resolved are higher.

    I’ve been trying to resolve this for hours now –

    * I attempted to submit my sitemap to Google – however they came back and said it is being blocked by my robots.txt. Sadly, I even didn’t know what was that robots.txt until last night. So… I searched for my website in google.com to see my web-site there: and what?! It’s said: For this result, no description available because of robots.txt on the site – find out more.

    * I called up my host (SUPERHOSTING) many times and told them this. They had guys looking at it for about 20-30 minutes. The only thing they could derive from this issue is that I have some plug-in. I deleted it. Again – the same. (one thing more: I didn’t have any plug-in before that…and it was the same: NO SEO …)

    * I created a robots.txt in my domain.com\cpanel saying the following:

    # robots.txt generated at http://www.mcanerin.com
    User-agent: *
    Disallow:
    Disallow: /cgi-bin/
    Disallow: /wp-admin/
    Disallow: /wp-includes/
    Sitemap: http://www.pathofwarrior.com/sitemap_index.xml

    And in Google Webmasters tool: the same problem:

    Google tried to crawl these URLs in the last 90 days, but was blocked by robots.txt.
    Googlebot is blocked by http://……example….

    and so on :(((

    *BUT when I go to “Sitemap” in google webmasters, it says:
    An error occurred while trying to access your Sitemap. Please make sure that it meets our guidelines and can be accessed from the location you provided and then resubmit.

    BUT WHEN I TEST IT ON: BLOCKED URL ADDRESS (in google webmaster tool) it says the following:

    Analysis of robots.txt
    value
    Row 7: Sitemap: http://www…../sitemap_index.xml
    result
    Found/Discovered a valid reference to the Sitemap

    ************************

    So…….. what to do,really :/ ?

    Remember, I do not have a visible robots.txt file anywhere on my server. I’ve went through every folder… and I created one in cpanel…but it doesn’t really work…

    Can someone please help?

    Thanks in advance,

    Bobb.

  • The topic ‘BIG robots.txt problem :( !’ is closed to new replies.