Forums

Google XML Sitemaps and Robot.txt File (2 posts)

  1. Buckycat7
    Member
    Posted 1 year ago #

    Okay I have no idea what is going on here. Google is saying I have a robots.txt file that is blocking all indexing and following on a couple of my sites. Here is what I have found:

    If you go to http://www.erikakers.com/robots.txt/ it shows their is a robots.txt file. But that file does not exist in the root directory on the server, plus if you go to the /index.php page of the site and view the source of the site there is a META tag that is a robots noindex nofollow there.

    I didn't place either of those things on the site. The robots file really doesn't exist on the server. So I downloaded the whole contents of my server to my hard drive and did a search of robots on all the files. It got hits on the sitemap.php files of the Google XML Sitemaps plugin, so I deactived that plug and to see if that get rid of the robots.txt file and it didn't. So whats going on here? Anyone have any ideas? I checked in my Google webmasters account and the site hasn't been indexed since March 28 because of a robots file that isn't there and it's not accepting my XML sitemaps file because of the unknown robots file.

  2. moongoose
    Member
    Posted 9 months ago #

    The robots.txt is auto generated by WordPress. That's why you don't see it in your root directory. If you have your Privacy Settings set to not allow search engines to access your site, then the robots.txt file that is auto-generated will block your site from Google.

    If this is not the case, then you'll want to check with your web host to make sure they are not blocking some of Google's IP addresses.

Topic Closed

This topic has been closed to new replies.

About this Topic