Support » Plugin: Virtual Robots.txt » pc robots.txt – site delisted

  • Resolved ds1970

    (@ds1970)



    Hi, something strange happened and I don’t know if it’s directly related to me installing the pc robots.txt plugin a couple of days ago.

    My site was on page 3 of google last week for its topic, but today, it’s no longer in the search results on any page.

    Could the pc robots.txt file affect if a site gets delisted or deindexed? installing this plugin is the only thing I’ve done with the site in the past few days, so I’m confused as to what happened. I didn’t change any of its setting, so I’m at a loss.

    thanks

Viewing 5 replies - 1 through 5 (of 5 total)
  • Moderator Kathryn Presner

    (@zoonini)

    Automattic Happiness Engineer

    Could the pc robots.txt file affect if a site gets delisted or deindexed?

    Absolutely. Not sure if you understand what robots.txt are for, but if improperly configured, it could be telling Google or other search engines not to spider your site.

    Can you provide a link to your site so I can take a look at your robots.txt?

    Hi, thanks for your response.
    Here’s the link: http://www.breast-pump-reviews.net/robots.txt

    what’s inside is what the plugin created.

    thanks

    Also, I unchecked the box for “Add sitemap URL to the virtual robots.txt file” in my xml sitemaps today. i read that it needs to be unchecked after you install pc robots.txt. Is that correct? thanks

    Moderator Kathryn Presner

    (@zoonini)

    Automattic Happiness Engineer

    I personally would remove everything except these lines:

    User-agent: *
    Disallow: /cgi-bin/
    Disallow: /wp-admin/
    Disallow: /wp-includes/
    Disallow: /wp-content/plugins/
    Disallow: /wp-content/cache/
    Disallow: /wp-content/themes/
    Disallow: /wp-login.php
    Disallow: /wp-register.php

    Also, I unchecked the box for “Add sitemap URL to the virtual robots.txt file” in my xml sitemaps today. i read that it needs to be unchecked after you install pc robots.txt. Is that correct?

    Sorry, I don’t know.

    You should not have to specify a path to your sitemap in robots.txt. As long as it’s properly named (sitemap.xml) and at the root of your site, spiders should be able to find it.

    Cool, thanks for your help.

Viewing 5 replies - 1 through 5 (of 5 total)
  • The topic ‘pc robots.txt – site delisted’ is closed to new replies.