Support » Plugin: Wordfence Security » Rate Limiting Blocking Google Bots

  • In the two days I’ve had Wordfence activated; it has systematically destroyed my SEO with Google and likely Bing as well. Webmaster tools is generating hundreds of denied crawl errors with them attempting to access my site.

    I could tell just by searching for my site; it’s been pushed way down.

    I’ve read about this issue and I see is ‘don’t look at us.’ I can also see that it’s set to block fake Googlebots but apparently it can’t tell the difference between the real ones and the fakes ones.

    You folks are making it really difficult to justify going ‘premium’. In the interim, I’ve had no choice but to disable ‘Rate Limiting’ all together.

    To be honest, I don’t really want to have to spend days much less hours configuring this plugin. I have way more than I can handle running my site. This is the reason I don’t use W3TC. Yoast is the only one I’ll spend a significant amount of time with.

    Help me, help you and make this worth my while.

Viewing 3 replies - 1 through 3 (of 3 total)
  • bluebearmedia

    (@bluebearmedia)

    Long time Wordfence user here…

    I’ve been using WF for quite some time, and have not experienced any issues with legitimate google bots being blocked, so it’s likely there’s something else about your site configuration that is the cause of your issues. Could be many things – your server config, htaccess settings, plug-in conflicts, or even your Wordfence settings.

    But even so – if you were to turn crawler limiting off completely, WF still provides many other benefits to protecting your site. Throttling crawler-traffic is the least of them, IMO. Case in point, country-blocking is one of the best premium features.

    I was trying to use an out of the box solution and you’re right the Country Block is a big reason I’ll ultimately upgrade. Russia accounts for a 1/3 of my traffic and are sucking up a lot of data (which costs me) and CPU resources.

    I’m about to implement a forced register/login to slow down the bots and possible initiate ‘hotlink protection’ though that’s been an issue in the past.

    Right now, I cannot have Google blocked and I checked my robots.txt file and nothing going on there. All of the Google crawl errors specifically state ‘blocked by Wordfence.

    I’ll go through everything in more detail after the first of the year. Thanks.

    Plugin Support wfalaa

    (@wfalaa)

    Hi Keith,
    May I know what you get when using “Fetch as Google for your website” from Google Webmasters tools?

    You can share a screenshot showing the result here in the forum, or email it to “alaa [at] wordfence [dot] com”, so I can take a closer look at this issue.

    Thanks.

Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘Rate Limiting Blocking Google Bots’ is closed to new replies.