You wouldn’t want to block all bots. As well, the “bot detection” is not exact, so there are many human site interactions flagged as bot that you wouldn’t/shouldn’t be blocking.
Sorry to hear you have to use Adwords, but one does what one has to do… Can you find any commonality at all with those hits? MTN
Yes, I’m sorry I have to use adwords (Google shopping) too… but I do so in a very limited fashion…unfortunately cannot even do that now, due to relentless vicious competitors hitting us with click fraud Bots… Google recommended that we complain with the Better Business Bureau (BBB)…nice.
I cannot find any commonality between the fraudulent hits…once in a while the IPs do repeat themselves… and they repeatedly hit the same 10-20 products (out of 600+). I have blocked access to these products for now… the URLs all include a Google GLCID. There is no common URL, IP, Country, or Referrer, or User Agent, etc. that I can identify. I have analyzed the data in WordFence traffic and Google Analytics, and CPANEL web logs.
I realise that the the “bot detection” is not exact in WF, I don’t mind accidentally blocking human traffic for a while.
So the question remains – Is there a way to create a WordFence rule to block ALL hits that WordFence identifies as BOT (not HUMAN), with exception to my whitelisted IP’s?
thanks. gsh2000
gsh, one key concept in blocking bad bots is whether they obey directives in robots.txt, using that concept you can do some clever things. In my case, at one time I added directives to my robots.txt that mentioned fake directory names such as /private-passwords, I then added that URL to the Wordfence “Immediately Block URL” list. This formed a very effective honey pot, as it seems that a lot of bots look at robots.txt for ideas on what to attack.
The other thing I did was place a hidden link on my homepage for a while, leading to a file named “private-passwords.html” with a list of fake user names and passwords. I disallowed this in robots.txt. After giving the bad bots and their masters time to learn that tempting file was there, I then deleted it and added the name to the Wordfence “Immediately Block URLs” list. A year or more later, bots are still trying to hit that file name and getting blocked. Fun.
Bad bots ignore robots.txt and just hit everything, including hitting URLs that get them blocked! Not sure if this would help you, just an idea of a direction.
https://blog.sqreen.io/detect-block-bad-bots/
If it’s any indication of how tough bot detection is, this is witnessed by even Wordfence having recent problems showing any sort of meaningful bot detection. My eternal hope is that the impressive and well funded brain trust at Wordfence will add effective bot detection to Wordfence, and give us various options that utilize bot detection.
Thanks for the good input Mountain Guy. I like your honeypot ideas, although in my specific case I am trying to block this very specific competitor (?) who is running a smart click fraud bot, targeting my google shopping feed. Oh well… will save myself some AdWord dollars, and will concentrate on other means of getting traffic.
By the way, another honey pot suggestion, is to use the nice WP WPS Hide Login plugin. I then set a WordFence rule, to automatically block accesses the wp-login.php URL. Works great.
Indeed, I’m a big advocate of WPS Hide Login, always strange Wordfence doesn’t offer this functionality, and use it as a way to ban bots as you suggest.
There has to be a way of fooling that click fraud bot into a trap…
MTN
Hi @gsh2000
Currently, there is no option to block all hits that Wordfence identifies as bot.
In your case, I recommend filtering the traffic by “Type = Bot” in Live Traffic page, then going through these bots you want to block and block them one by one, of course it would be easier if you can recognize anything similar with these bots, for example something common in the hostname or the user-agent or perhaps quite similar IPs that can be grouped into a range to be blocked in the Custom Pattern Blocking page.
Thanks.
