Caleb
Forum Replies Created
-
@skaye, I am a little confused.. Why would you want all IP’s that happen to try an invalid (or blacklisted) login name to have their IP blocked PERMANENTLY, in addition to already being locked out by their use of a bad username? Personally, I would NOT want that to happen, and would protest loudly if every IP trying a bad username was autoblocked forver.
One thing to realize is that many or most of these attempts are running scripts that arrive on batches of Proxy or VPN IPs. Their are not actually the originating IP-address. The attacker does not use his own IP address.. Just like burglars at your house do not leave their own business cards behind.
In many other cases, the requests could be arriving on dynamically allocated IPs. Think phone-company and in some countries local ISP IPs.. Random and changing DHCP allocation of IPs. Used by Mr. bad guy today, used by Ms. Buy-Pretty-Clothes tomorrow.That’s why you frequently see the same screwy login name being attempted from IP’s belonging to a multitude of countries at almost the same time. 10-15 countries do not actually decide to attack you at the same time. It is one robotic attack, that COULD be originating as close as your neighbors house, but passing its traffic through Proxy or VPN servers around the world. (Companies like HideMyAss and iPredator sell this type of access in batches of hundreds or even thousands of IP addresses at a time for spammer and hacker use).
So if invalid or brute-force attempts are not blocked by the firewall as attempts, then you have a concern. But you typically don’t really want every single VPN/proxy service blocked individually automatically, as a valid user might arrive on it the next hour.
If a whois shows that the attack originates from a bad IP range (Hertzner hosting, AWS cloud, …), then it is better to simply block the whole range, assuring that all their traffic is blocked. (Valid users do not come out of bad server hosting).
Blocking individual IP’s one at a time because of what is frequently a temporary infraction (IP level speaking) is simply not the way to go about it.BTW.. That use of Proxies and VPNs is also why country-blocking (blocking a whole country at a time) is typically a poor idea. “Countries” do not attack. People do.
And the ACTUAL, final hacker-user of that set of proxies, spread all over the world, can just as easily sit in your own backyard, or in Canada, as he can in China or Moldovia. And some normal visitors, because they do not understand the realities of proxies and VPN buy or use these services as well.
Either because some fool told them it makes their traffic more secure or secret, or because they are trying to skirt licensing issues. Such as a Canadian/US users wanting to watch British BBC licensed programming by hiding behind a British proxy IP address.Botton-line, though.. I’ll thank the Wordfence team to not automatically block every IP permanently, just because of a bad username use.
Thanks, looking forward to a fix, since I really hesitate to install WordFence updates until it’s fixed. The latest coming down would UNDO my own fixes. π
On what contributed to this case.
Looking into the WordPress code as well, I think the reason in this case is that WordPress is a little bit schizophrenic, when it comes to how prefixes are handled. In class wpdb::set_prefix() and especially in wpdb::get_blog_prefix().
In a new install, typically the first blog created becomes blog ID 1, and its private tables are in format {prefix}{blogid=1}’ ‘{tablename}.
When you go to network admin, you are then supposed to be operating without the blogid in the tables. Off domain/blog #1 as the main domain.
HOWEVER… What if things change over time?In get_blog_prefix(), they have this code:
if ( is_multisite() ) { if ( null === $blog_id ) $blog_id = $this->blogid; $blog_id = (int) $blog_id; if ( defined( 'MULTISITE' ) && ( 0 == $blog_id || 1 == $blog_id ) ) return $this->base_prefix; else return $this->base_prefix . $blog_id . '_'; } else { return $this->base_prefix; }Notice first off, that for a single install, as I suspected, all tables are off ‘base_prefix’. That last return in the code. So, prefix and base_prefix are the same. That’s why your code works for single installs.
Secondly, VERY IMPORTANT. Notice in the above code that for multisites, blogid=0(no ID) and blogid=1 (typically that very first blog) are treated special. In the multisite case, for blog 1, they ALWAYS return “base_prefix”. Which explains why most of your installs work, even when they are multi-sites. That first blog-domain is still alive and is running the Network Admin screens.
In my case, however, for historic reasons, blogid #1 is a deleted site, and BLOGID=3 is the controlling domain. As in, when you are Network Admin, invisibly to the user, the running domain is actually from blogid=3, which is what I saw in “db->prefix” when you were attempting to read the tables and all queries failed. it would always try to read off “ftrav_3_wfBlockedIPLog”, where the correct table is “ftrav_wfBlockedIPLog” (no blogid).
Independent of all that, when your tables are network global, the correct prefix IS “base_prefix”.. Only if you expect a query to be run of a blog level table should “prefix” be used. Since WordFence is network/globally installed, and wfSchema.php always uses “base_prefix” to install them.. It would then be a sound idea to try to find them again off that same prefix. π
BTW.. Looking at the WordFence code, it’s history and probably different people have made naming schemes is something that might need some cleaning up while y’all are at it. I can understand how it happened, but also why people making changes to it get confused. π
Some examples:
In some code, “$this->db” is a variable assigned directly to the global $wpdb, which is how accessing “prefix” and “base_prefix” works.
In other areas, “$this->db” is actually assigned the value of “new wfDB()”, your wrapper class. Which by fun coincidence has a routine “prefix()”, which always returns the value of “$wpdb->base_prefix”. To complete the naming confusion.You even in the wfDb class have a function networkPrefix(), which always returns the value of get_blog_prefix(), which is how it all connects back to my explanation above about “set_prefix” and “get_blog_prefix” in the main WordPress code.
Your “networkPrefix” would then on a fresh network install always return the value of “base_prefix” (from get_blog_prefix(), because Network Admin is on Blogid=1)..
But in some installs, like mine, where the controlling domain is not blogid=1, but 3, you would get some other prefix, and include that blog-# in the table names.. But that is not where you installed your tables… πBetween the various locations and uses, somewhat confusing.
The values of properties and function calls
“db->prefix” (a $wpdb property),
‘db->prefix()’ ( function in wfDb always returning base_prefix),
‘db->base_prefix” (a $wpdb property),
and networkPrefix() (sometimes returning base_prefix, sometimes NOT)are used kind of randomly in various sections/functions/classes of your code, and to make the confusion even worse, WordPress’s own multisite decisions in set_prefix() and get_blog_prefix() then further interfere with the values of those fields. π
If it was me, I would clean up that code some, and maybe refactor the naming schemes to make it more clear what is expected. Lest new programmers editing your code get confused. π
Bottom-line is still.. If you install your tables off “base_prefix”, which you always do, then that is where they should be accessed later as well. π
Also notice, that in your own wrapper class wfDB does the same.
Function prefix() returns the value of “$wpdb->base_prefix”.
The WordPress DB user does not have limited privileges. It has full privileges. I used the same ID/login to manually add the BlockType field through MySQL WorkBench. It has never changed. I have no idea why it did not upgrade correctly back when. Your upgrade procedure in the plugin should have worked.
The issue of using ‘prefix’ versus ‘base_prefix’ is separate..
Per the WordPress Codex for the wpdb structure:
$prefix
The assigned WordPress table prefix for the site.
$base_prefix
The original prefix as defined in wp-config.php. For multi-site: Use if you want to get the prefix without the blog number appendedSince my freshly installed WordFence tables sure does not have a blog-id mixed into the table names (are global to the multisite) install, the correct prefix to use is “base_prefix”.
As also indicated by the fact that only two routines in wfActivityReport (wfActivityReport::getBlockedCount and wfActivityReport::getTopIpsBlocked) uses ‘prefix’ to access your tables. Both of them ONLY show data in the dashboard after I change them to use ‘base_prefix’. In addition, wfRecentFirewallActivity::run(), tries to use ‘prefix’. Also wrong.
All other routines in wfActivityReport.php correctly uses ‘base_prefix’.
Funny enough, your routine “rotateIPLog()’, which on a weekly basis rotates that same exact table with IP stats, uses ‘base_prefix’, not prefix, like getBlockedCount().
So, there are two different ways used for accessing the same table, in the same PHP class and file. πAnd lastly, in wfSchema.php, where you actually create all the tables, you use ‘base_prefix’, not ‘prefix’.
As inglobal $wpdb; $this->db = new wfDB(); $this->prefix = $wpdb->base_prefix;Surely, the prefix used to create/install the tables should be used to access them, lest they get lost. As they did. π
It seems that at different times (and maybe by different people) sort of random decisions have been made on which prefix to use to access tables all installed in the same place (all installed global to the install, not per-blog). π
Notice, that on all single (as in not multisite) installs out there, using either one might work fine. dbprefix and base_prefix would return the same value, I think, as long as there is no blog-id numbering involved in table naming. Which might explain how many others are not complaining. Single-blog installers would not see a difference.
However, if you search this forum you will see that others have questioned why their IPs blocked list is always empty. The question was never adequately answered.
But I can promise you, that after I changed it to use ‘base_prefix’, all my Top IPs blocked showed up in the dashboard immediately. πNo, the WordPress DB user does not have limited privileges. It has full privileges. I used the same ID/login to manually add the BlockType field through MySQL WorkBench. It has never changed. I have no idea why it did not upgrade correctly back when. Your upgrade procedure in the plugin should have worked.
The issue of using ‘prefix’ versus ‘base_prefix’ is separate..
Per the WordPress Codex for the wpdb structure:
$prefix
The assigned WordPress table prefix for the site.
$base_prefix
The original prefix as defined in wp-config.php. For multi-site: Use if you want to get the prefix without the blog number appendedSince my freshly installed WordFence tables sure does not have a blog-id mixed into the table names (are global to the multisite) install, the correct prefix to use is “base_prefix”.
As also indicated by the fact that only two routines in wfActivityReport (wfActivityReport::getBlockedCount and wfActivityReport::getTopIpsBlocked) uses ‘prefix’ to access your tables. Both of them ONLY show data in the dashboard after I change them to use ‘base_prefix’. In addition, wfRecentFirewallActivity::run(), tries to use ‘prefix’. Also wrong.
All other routines in wfActivityReport.php correctly uses ‘base_prefix’.
Funny enough, your routine “rotateIPLog()’, which on a weekly basis rotates that same exact table with IP stats, uses ‘base_prefix’, not prefix, like getBlockedCount().
So, there are two different ways used for accessing the same table, in the same PHP class and file. πAnd lastly, in wfSchema.php, where you actually create all the tables, you use ‘base_prefix’, not ‘prefix’.
As inglobal $wpdb; $this->db = new wfDB(); $this->prefix = $wpdb->base_prefix;Surely, the prefix used to create/install the tables should be used to access them, lest they get lost. As they did. π
It seems that at different times (and maybe by different people) sort of random decisions have been made on which prefix to use to access tables all installed in the same place (all installed global to the install, not per-blog). π
Notice, that on all single (as in not multisite) installs out there, using either one might work fine. dbprefix and base_prefix would return the same value, I think, as long as there is no blog-id numbering involved in table naming. Which might explain how many others are not complaining. Single-blog installers would not see a difference.
However, if you search this forum you will see that others have questioned why their IPs blocked list is always empty. The question was never adequately answered.
But I can promise you, that after I changed it to use ‘base_prefix’, all my Top IPs blocked showed up in the dashboard immediately. πSo, as it turns out, even after a complete reinstall, there was still an issue/bug. It was not merely invalid table structures.
At first, I thought that all the zeroes that still appeared in the dashboard was caused by maybe caching, and was planning to wait until the next daily email report.
However, looking at it again, it seems that there is an issue with the db prefix used when extracting certain numbers.
In wfActivityReport::getBlockedCount(), you use $this->db->dbprefix to generate the queries, which in fact is the wrong prefix. All the WordFence tables are off ‘base_prefix’. Similar to how you already do it in getTopCountriesBlocked() and others. So the queries silently failed and returned nothing, which when typecast to (int) became 0 (zero).
I verified it by changing ‘dbprefix’ to ‘base_prefix” and all the correct numbers then started showing in the dashboard.
So, for now at least you can close this particular issue. From my perspective.
Finally got some time to look at why this thing was failing its reports, and as it turns out the wfBlockedIPLog table was missing the blockType field.. Causing logging of blocks to fail of course. (db insert failure)
The wfBlockedIPLog table was completely empty.The blockType field was from what I can see supposed to have added itself in version 6.3.5?
After first adding the field manually and changing primary index, the logging started.
Checking around, some other tables seemed off.. So I hit the big reset button, removed WordFence, assured all tables were dropped, and reinstalled.
It now gathering data.. I haven’t yet seen new reports, but per the dashboard at least it seems to be up and running.
Only remaining curiosity is why the table upgrade procedures did not work, since the missing field(s) were not detected. Then causing the logging function to merely silently fail.
The Dashboard Firewall status indicate “0” (zero) attacks blocked across the board. For the day, week, month.
The Login Attempts widget shows the failed logins. Like 3 (seemingly manual) attempts on June 26 to find admin accounts.. Those also show in the daily emailed report for June 26. I think the login attempts are counted correctly. Apparently few of the hacking attempts actually as far as actually trying a login/pw combo.
But despite both the Dashboard AND the email for today (June 27) reporting zero blocked accesses/attacks, on the Live Traffic page I gave up scrolling down to count all the blocked accesses (not actually getting to full login attempts). Just filtering on today’s attacks on wp-login and xmlrpc the blocked IPs from all over the world are so many. And I kind of assumed that they would have been counted.
If for nothing else for the advertising value of reporting how much is actually blocked. π
Hmm.. If we are suddenly not to trust that template code is actually executed, that is the first I have heard of it. If conditional “code” is not supposed to be trusted to be executed (especially from version to version), that would make any “conditional setup useless.
The fictional if/each case I typed above is in fact a very simple, almost straight example from the tutorials on pods.io. (See for example http://pods.io/docs/build/template-tags-in-pods-templates/)
I am certainly not suggesting that clearing it up is necessarily neither easy nor fast.
Merely that if it is a long-time affair to revert to functional code, that maybe the bad version should be removed from the repository. Would stop things like WordFence from complaining every day about sites using out-of-date plugins and encouraging people to upgrade to an in this case non-working plugin.As far as conditionals not supposed to be nested or trusted to work from version to version.. That would defeat the purpose of having them. Also invalidates the pods.io tutorials describing their introductions and use. Both in the code shown and the statements made.
Per tutorials, conditionals were introduced in templates to replace the support for straight PHP code (2.4 timeframe). But if the new template code and conditionals such as if/each are not to be trusted, then maybe it would be better to stick to pure PHP templates.
So I can’t help but ask.. When will there be a new version of the plugin or downgrading the repository to 2.6.8?
Depending on the site, this can be a quite serious bug.. Akin to WP itself removing posts from a site.
When shortcodes stop working within conditionals, this can make whole sections of web-pages or links to pages virtually disappear. With search engine bots going berserk, keyword ranking change for the site, total internal site linkage change, …, as a result.
As merely one example, think code similar to
if RelatedItems
Heading textEach RelatedItem
……
links
……end if
If one or more conditional sections of web-pages magically disappear because shortcodes are not interpreted after upgrading to 2.6.9, this can have a SIGNIFICANT effect on search engine interpretations, not to speak of the obvious effect of the content access disappearing for users.
I see in GitHub that the problem was assigned ~4 days ago, but if there is no quick new version of the plugin coming, maybe the bad version should be removed from the repository? Reverting to 2.6.8?
If you would let us know what your end result was, and maybe mark as resolved IF you find it do be so?
This has nothing to do with ZIP encoding (Which IS entirely lossless.).
Nor do I think you “lost” anything.You might simply be viewing your code file in a “dumb” viewer on Windows that cannot handle the difference in how End-of-Lines are handled on Linux versus DOS/Windows.
On Unix/Linux systems each newline in a file is representing as simply an LF (line-feed) character.. On DOS/Windows they use two characters (CR+LF, a CarriageReturn followed by a Line-feed).
SO… When you view a Linux file (with only an LF between lines) on Windows, dumb viewers (like the Windows Notepad app) do not show the LF alone. Does not see it as needing to show a new line. It visually looks like all the lines have been concatenated (combined).
Try viewing the same file in a different Windows App.. Like Wordpad, which understands the LF alone as meaning go to a fresh line on screen.
In the end, it does not matter much.. Files edited/created in Windows apps (and hence has CRs inserted as well) are understood when executed in PHP on Linux. It ignores the stupid CR characters Windows uses at the end of lines.
It might be worth noting, that this is the same issue as described in this posting.
See my replies in that thread.
Also note, that if you are really “out of Quota”, then the fact that it works with the other plugin might be somewhat meaning less. The act of removing the current plugin (and its temp files, older archives, …) might just temporarily have brought you under Quota limit. Meaning that when you get up there again, it might just fail too.
The only way an alternate Backup plugin could “fix” the issue, would be if it wrote backup files DIRECTLY to the offsite backup location, like DropBox. Without trying to create a temporary local backup archive first, which is what hits the Quota limit..
As in, the alternate backup plugin not needing that local space at all.- This reply was modified 9 years, 1 month ago by Caleb.
@seville76, I sent you an email.
It is definitely true, that even a partial backup can be useful at times.
Partial or even partly corrupted backups have saved my butt in extracting some files a number of times.
So leaving the file behind, even if only partial, is not a bad idea at all.. But maybe it should be named differently. adding a “partial”, or “failed” keyword into its name.On admin’s noticing that a backup failed, that requires a clear idea of the impending doom from the log. π
The log the OP attached above (see below) is totally devoid of any failure information.
Reading the log, one would believe that everything went smoothly, and that there is a valid backup file. No issue to go fix.In my opinion, the log should include the failure information, and even probably the Zip-status message.. Not sure whether this OP has logs emailed to them or what the email subject would be.. As the OP indicated above
Where it seems that the backup has been done and sent. So, or there is a Disk quota issue or the zipping succeeded⦠Which one should I trust?
But IF my emailed backup logs contained nothing but the below, there would be little to indicate that the uploaded backup was not OK for anything but potential catastrophic file-extraction and partial use.
[23-Mar-2017 01:00:30] Backup archive created. [23-Mar-2017 01:00:30] Archive size is 147,11 MB. [23-Mar-2017 01:00:30] 9001 Files with 223,35 MB in Archive. [23-Mar-2017 01:00:31] 1. Try to send backup file to Dropbox β¦ [23-Mar-2017 01:00:32] Authenticated with Dropbox of user: Drauth.org Web Agency [23-Mar-2017 01:00:32] Uploading to Dropbox β¦ [23-Mar-2017 01:02:42] Backup transferred to https://content.dropboxapi.com/1/files/sandbox/Le-Dimore-del-Quartetto/Dimore_2017-03-23.zip [23-Mar-2017 01:02:43] One file deleted from Dropbox [23-Mar-2017 01:02:43] One old log deleted