WordPress.org

Ready to get started?Download WordPress

Forums

Technorati and robots.tx (14 posts)

  1. inoah
    Member
    Posted 4 years ago #

    I received this message when trying to register my site with Technorati.

    "Our access is explicitly blocked by your robots.txt file."

    My privacy settings are set to "I would like my blog to be visible to everyone, including search engines (like Google, Bing, Technorati) and archivers"

    I am not sure what to do next.

  2. esmi
    Forum Moderator
    Posted 4 years ago #

    What's in your robots.txt file? It should be in your domain's root folder.

  3. inoah
    Member
    Posted 4 years ago #

    I am not sure how get to my root folder. Can you give me an idea of where it might be/what folder in the control panel for my host (Blue Host)?

  4. bh_WP_fan
    Member
    Posted 4 years ago #

    Your root folder should be the public_html folder, unless your site is installed in a subfolder, then your root folder will be something like public_html/yoursite

    You will still want to check the robots file in your public_html even if your site is in a subfolder like public_html/yoursite. There may also be a deny in the .htaccess file, so if checking the robots.txt file doesn't help, make sure hidden files are enabled and check your .htaccess.

  5. inoah
    Member
    Posted 4 years ago #

    I am really challenged. :) I can't find the file in Public_html folder. This is relatively new to me. Can yo give me a sample pathway?

    Thanks.

  6. esmi
    Forum Moderator
    Posted 4 years ago #

    Let's try another approach - site url?

  7. inoah
    Member
    Posted 4 years ago #

    naturalproductsbiz.com

  8. Robert Chapin
    Member
    Posted 4 years ago #

    That looks like the do_robots file from WordPress. You should create your own file that doesn't include the ambiguous syntax used by default.

  9. bh_WP_fan
    Member
    Posted 4 years ago #

    You can access your folders using the File Manager tool within the Files section of the cPanel. Your public_html should be in there and you can edit an existing robots.txt file from within there or create your own robots.txt file from within there.

  10. inoah
    Member
    Posted 4 years ago #

    I uploaded a file named robots.txt to the root file with the below included. Technorati is still telling me the robots.txt file is prohibiting them from crawling the site???

    User-agent: *
    Disallow: /cgi-bin/
    Disallow: /tmp/
    Disallow: /junk/

    Did I provide the wrong information?

  11. esmi
    Forum Moderator
    Posted 4 years ago #

    That looks fine to me.

  12. Robert Chapin
    Member
    Posted 4 years ago #

    I looked a little closer, and it appears your blog feed is invalid. Check it out:

    <link rel="alternate" type="application/rss+xml" title="RSS 2.0" href="http://feeds.feedburner.com/NaturalProductsBiz" />

    Technorati is going to take one look at that and then ignore your blog.

  13. inoah
    Member
    Posted 4 years ago #

    How do I fix the feed?

  14. esmi
    Forum Moderator
    Posted 4 years ago #

    It's not the feed. It's the feed url. You;re pointing to Feedburner instead of your own feeds. Try replacing:

    <link rel="alternate" type="application/rss+xml" title="RSS 2.0" href="http://feeds.feedburner.com/NaturalProductsBiz" />

    with:

    <link rel="alternate" type="application/rss+xml" title="RSS 2.0" href="<?php bloginfo('rss2_url'); ?>" />
    <link rel="alternate" type="text/xml" title="RSS .92" href="<?php bloginfo('rss_url'); ?>" />
    <link rel="alternate" type="application/atom+xml" title="Atom 0.3" href="<?php bloginfo('atom_url'); ?>" />

    in header.php

Topic Closed

This topic has been closed to new replies.

About this Topic