Well, much worse than that:
I UNticked block bad bots, saved, and promply am thrown out of our site, admin and even frontend: You have no right to access this page.
That frankly is a mess!
Please email us the logs to support [at] semperfiwebdesign.com and we can take a look. We’ve not seen this behavior on any other site before.
Hey Steve, I documented all steps.
– So, when you rolled out the bad bot feature you set default to enable.
– When I noticed your feature today I at first ticked via htaccess.
– You rightfully warn there: “Warning: this will change your web server configuration, make sure you are able to edit this file manually as well.”
– Rightfully, because you are not handling the htaccess safely
– You *delete* custom entries in the htaccess from other plugins
– In our case you deleted everything written by fastestcache and by s2member, plus our own redirect entries.
– In our unfortunate case, this resulted in “No access to this site”
Strongly recommended: Do not make changes to existing entries in the htaccess – if you wish to handle/offer htaccess at all.
From other plugins I noticed earlier, very very few plugins are able to write to the htaccess correctly. So better don’t. Just my conclusion from today’s disaster. Had I known it was a corrupted htaccess, we wouldn’t have had to roll out a backup.
Hi CoolDavidoff,
I am sorry for the problems you experienced, however, I would like to note two things:
1). We absolutely do not activate the Bad Bot Blocker by default. You have to go to Feature Manager and activate that feature. Once you’ve activated that feature you then have to check a box in the settings to Add rules to .htaccess. We don’t touch your .htaccess file unless you check that box and it is not checked by default.
2). We absolutely do not delete anything from the .htaccess file, all we do is append our rules and only if you’ve checked the setting mentioned above. We would never touch any other rules in an .htaccess file.
Finally, we always maintain that before making a change to your site that you back up your site just in case something goes wrong. Whenever you install anything, activate anything, deactivate anything or change any setting, there is always a risk that something can go wrong so always take a backup.
Again, I am sorry for your experience today, it’s entirely possible you had a conflict with another plugin on your site.
I have just tested our Bad Bot Blocker with WordPress v4.4.2, the current version of s2member and the current version of WP Fastest Cache and did not experience any issues. Obviously I cannot replicate the exact environment that you have but I wanted to make sure we don’t have a conflict with those plugins you mentioned.
I appreciate you have to say “We absolutely do not delete anything from the .htaccess file”, and I continue to wholeheartedly support your plugin.
Nonetheless your statement is factually wrong: The very second I clicked Save in your plugin, it broke the entire client site as described. Likewise, as I said, I downloaded a copy of the htaccess after your plugin’s htaccess changes, compared the before and after versions, and saw exactly what your plugin has done.
And no, fastestcache wasn’t even activated at the time, and s2member doesn’t change the htaccess unless the customer saves new settings (which wasn’t done either), and no other plugin CAN change the htaccess here.
Anyway, I won’t let this waste more of my time, I only posted this to allow you to review your htaccess changes and bad bot feature. That you aren’t aware of all the changes being made is obvious: else you would have prevented such case I am sure. Thus again, if you want to review the plugin in this regard, I would appreciate it – and others too.
Speaking of bad bot feature, you didn’t mention one word on this: I wrote:
“all log entries are blocking the client’s own site (this is why traffic gone down so much). Example for (hundreds/thousands of these):
2016-02-12 16:47:29 Blocked bot with IP [client IP] — matched user agent WordPress/4.4.2 [that’s installed wp yes]; https://client site] found in blocklist. Thought you will want to know.”
I consider this to be caused by another conflict with the existing rewrite rules. htaccess is a delicate thing, Steve. If you write a line it often affects other existing lines further below, and vv.
Hi CoolDavidoff,
Thank you for the report. I’m sorry you’re having these issues. I’m not 100% certain on the cause, and would like to investigate and find out, but would first like to clear up a few things:
1) This isn’t a new feature, and has been around since the first part of last year.
2) This feature isn’t enabled by default. You have to manually enable it.
3) Steve is correct that nothing is deleted, and the statement “you are not handling the htaccess safely” is incorrect. The ONLY code used to modify the file is the built-in WordPress function “insert_with_markers,” which appends content into the .htaccess. https://wpseek.com/function/insert_with_markers/ This built-in WordPress function handles 100% of the inserting into .htaccess (in other words, WordPress handles the actual modifying of the .htaccess, the plugin only supplies the content and the markers). According to WordPress documentation, this function: Inserts an array of strings into a file (.htaccess ), placing it between BEGIN and END markers. Replaces existing marked info. Retains surrounding data. Creates file if none exists. There is no code in All in One SEO Pack that can delete anything in .htaccess.
It is odd that it would block someone with a WP user agent. There’s nothing in All in One SEO Pack that automatically adds to the bad bot list. It only checks against the default list (which obviously doesn’t include WP) and anything the user manually adds (obviously you didn’t add WP). There is code that specifically checks to make sure the user isn’t logged in, which I assume would be the case here.
Having said that, clearly something unexpected is going on. Unfortunately, I’m not able to replicate this issue and need more information.
If you are interested in us investigating further and/or would be willing to send us the logs and before/after .htaccess files, please contact us at semperplugins.com/contact/
