Forums

KB Robots.txt
Robots.txt? (2 posts)

  1. PMS_Warrior
    Member
    Posted 9 months ago #

    My privacy setting on my blog are set to pubic to allow everyone / everything to search all.

    Nonetheless, when I tried to claim my blog with Technorati, it was rejected because technorati says it was blocked by my robots file.

    I don't really understand how to add a robots.txt file to the web hierarch as instructed, so I just left it...

    BUT

    then I logged in to Webmaster Tools (Google) and under Crawler Access / Test robots.txt, all seems well (even without me doing anything):

    Text of http://www.pmswarrior.com/robots.txt

    User-agent: *
    Disallow:

    Sitemap: http://www.pmswarrior.com/sitemap.xml.gz

    (PS. I also downloaded kb-robots.txt plug in in the hope that that might help, but then in the Readme file I read that it's not really meant for sites where you're happy to give web spiders unrestricted access. If only technorati could SEE my unrestricted access!)

    Can anyone tell me what's going on?

    Thanks in advance

    http://wordpress.org/extend/plugins/kb-robotstxt/

  2. PMS_Warrior
    Member
    Posted 9 months ago #

    eek! 'public' obviously - not 'pubic'!

Reply

You must log in to post.

About this Plugin

About this Topic

Tags