WordPress.org

Ready to get started?Download WordPress

Forums

[resolved] Bots have been restricted from my site, but don't know how it happened? (3 posts)

  1. DanoPano
    Member
    Posted 1 year ago #

    Hi,

    The bots have somehow become restricted from crawling my site. I made sure that I did not check mark the "Block search engines, but allow normal visitors" setting inside ClickBump's SEO module.

    Also made sure that I did not go private with my WordPress Reading settings by not placing a check mark inside the box next to "Discourage search engines from indexing this site".

    Additionally, since I am using Yoast WordPress SEO, I made sure that none of the indexing rules were implemented. I left this section in the default mode with none of the options checked.

    However, now, when I do a site check for indexing on my site using the following search command in Google: site:lovephrasesx.com I get this message back in return,

    Love Phrases
    lovephrasesx.com/Share A description for this result is not available because of this site's robots.txt

    This basically informs me that there is a restriction on indexing/crawling my site which has been implemented by a robots.txt file.??? Where did this file come from, and how was it created?

    This is very disconcerting and I am not able to make sense of it given the settings I have made to my site.

    Can anyone shed some light on this for me? How do I remedy this and how can I avoid this in the future?

    Thanks,
    -Dan Sacapano

  2. Odai Athamneh
    Member
    Posted 1 year ago #

    You can manually submit your website to Google to get around this problem. Google only indexes a site every so often, and it's likely that your robots.txt was still bad the last time Google checked your site.

    I manually accessed your robots.tx, and it looks like you're okay for now. I'd guess that a poorly written plugin messed with it? Tough to say, I've never heard of this.

    Check your robots.txt every so often for a few weeks, and if it doesn't happen again, I wouldn't worry about it.

  3. DanoPano
    Member
    Posted 1 year ago #

    Hi, thank you for responding to my issue. What I finally decided to do was to uninstall WordPress SEO by Yoast...it became too much of a problem for me. After uninstalling it, my website is just fine and I am using, Google Sitemaps now.

    Thanks again,
    -Dan Sacapano

Topic Closed

This topic has been closed to new replies.

About this Topic