Forum Replies Created

Viewing 15 replies - 16 through 30 (of 218 total)
  • Thread Starter Caleb

    (@crudhunter)

    After 3 months of waiting, maybe it is time to fix this BUG!?

    Getting tired of getting flurries of DB query failures and having to manually fix up this plugin again on every update sent down.

    a) Remove the robots.txt stuff.. That is in most cases irrelevant. What will kill off the old stuff is the redirects, which will eventually make Google forget the bad URLs. After a year, that should have happened a long time ago. (Except, see b.)

    b) Despite a), if your site still on some locations/pages/…show links to the bad paths, that means that GoogleBot will still find them. Over, and over, and over. So you need to figure out where these links to the bad area might still be appearing. Potentially the links section in Google webmaster can tell you. Or a search in your database. But pages are likely still containing or generating bad links and sticking them on your pages. Which causes GoogleBot to retry them over and over, while then hitting the redirect.
    Removing the /hacked_foldername is not enough to clean. That just means that no one can actually get there anymore. It does not mean that you do not still have pages that point to that location.

    b) If the Redirects show the hacked file in the URL, you did the redirect wrong in .htaccess. You probably have a “RewriteRule” line with $1 at the end or similar.. Otherwise, all the bad paths would redirect to a clean “http://homepage/”. The “hacked_file” part would have vanished, unless you have a redirect to something like “http://homepage/$1” or a different way of transferring the “hacked_file” part from the input- to the output-part of the redirect rule..

    wp-settings.php always includes kses.php. That is not an infection. It is standard WordPress and is done on all WordPress sites.

    The failure to load is because @krambrose manually deleted an essential WordPress file (kses.php), causing WordPress to be unable to boot up.

    To fix it so the site loads again, simply grab a clean copy of ‘kses.php’ from another site or from a downloaded, fresh, WordPress install archive, and put it back in the /wp-includes/ directory where it belongs. Then WordPress will start up again.

    If only admin/editing screens are affected, then you could be seeing actual false hits in the firewall.. The “actions” taken in WordPress’ post-editing screens have a tendency to be blocked as unsafe in various blocking mechanisms.
    Even using ModSecurity, I have to target-disable 4-6 “rules” in ModSecurity for such things as post.php, post-new.php to even function. Same or similar rules are for the most part also used in WordFence’ firewall. When using Opera data saving, the actions observed on editing screens could be seen slightly different from when going direct.
    I have never tested with Opera.

    Of course you might need get_blog_prefix() in some queries, but never in any of the queries we are discussing here, which are accesses to your own tables.. Your private tables stay where they were installed. No need to repeatedly ask WordPress where they went. 🙂

    My point is, that Scanning seem to be getting all the attention. watch the change-logs to see what is being changed. Scan related new features, rather than fixing existing, now old, bugs in the core.

    Wanting to fix or optimize scanning is partly understandable, since that expensive process can get WordFence banned from hosting companies if not efficient.
    But still, daily scanning is by definition secondary functionality. If infections are found through scanning, that means that the first-line defense (core wordFence) have failed to block an incoming attack or infection. It would therefore always seem of higher importance to users that the core is both bug-free and working as expected.
    Especially when the changes needed are both known and very simple to implement, it seems strange to have clear bugs hanging around for months of debate, while putting all the efforts into the (again, to users) much less important scan code.

    Opera in data serving modes works by your local Opera browser no longer connecting directly to the sites you visit, but instead uses Opera owned servers around the world as proxies. In those proxy servers is where the various data/html/image transformations (data saving) happen.

    In essence, the Opera proxy servers browse on your behalf, and they afterwards send the user’s browser transformed (compressed sort of) versions of the pages requested.

    That of course can have various side-effects on a monitoring plugin, like WordFence.

    For one, the visiting servers will be seen as crawlers. They do not behave like a human visitor would, or like a normal proxy-user would. The transformation is not a simple proxy server use, where there are a 1-to-1 relationship in both request and speed between what files the Browser and the proxy calls for. The data transforming servers actually interpret the whole page and makes independent loading and caching decisions.

    Second, depending on rate-limiting settings, these servers can much more easily get caught up in rate limiting or even deemed to be scraper proxies, then blocked manually.
    Read-ahead page loading, multiple users using Opera savings modes, and arriving through the same Opera proxy server. WordFence (and your site) could see them IP-wise as ONE visitor, not as separate visitors. Confusing the heck out of rate-limiting. Also, since rate-limiting blocks only for a “period”, the “evidence” of the failures by rate-limiting, so to speak, can vanish into the blue when the period expire. 🙂
    What fails at some points or in some cases, depending on speed of browsing, can look to work perfectly when testing it later. Hence there is not necessarily a way to compare @wfsupport’s test with what site-local users are seeing. Maybe not in the busy timezone for this particular site, where they are more likely to see the accidental rate-limiting. Imaging a new tour group, for some reason more likely to have multiple Opera users suddenly visiting in bulk. Maybe sitting on similarly configured church/library computers looking at the “Bible tour” they are all going on. 🙂 All seen as one, if using Opera data-saving.. 🙂

    Third, you can see visits from many “mysterious” places depending on where the particular Opera server chosen by Opera is located. Originally, most visits would be seen as “Iceland”, where they were located, but these days it can be sort of anywhere. A human Opera user in the UK, visiting his own site in the UK, have seen his own visits as coming from Nairobi, Kenya or similar odd places.

    Fourth, as a side-effect of the above, the “visits” from local users might suddenly appear to be from a country that is otherwise blocked on the site, if some kind of country blocking is enabled. There is absolutely no guarantee, that the Opera servers doing the data transformation will operate out of the same country as your browser would with data-saving turned off. So Country blocking turns kinda useless when directed at Opera users. Or generally proxy users, which is really what Opera’s data-saving is.

    BTW, This is principally similar to how Chrome’s Data Saver mechanism works for Chrome by running all content through Google’s servers (much more likely to be in the US probably).

    @wfasa.. I CAN GUARANTEE YOU, that get_blog_prefix(), called through your private networkPrefix() is a BIG problem. It just happens to not be @irisslesley problem YET. But it will be.

    I can also guarantee you, that the problem is in other places than merely getting the update of a few stats tables to work, because the bad use of prefixes is also the cause for THIS issue:

    https://wordpress.org/support/topic/wfschemawffilemods-missing-table-fields/

    Which causes even fresh installs of WordFence to FAIL if the installation over time has the type of multi-site installation that both @irisslesley and I are seeing.

    If you want to see more trouble, try uninstalling WordFence (with removal of tables), then reinstall WordFence.. That will immediately introduce the table structure issue, because the RunInstall() cannot work it ways through fixing the bad tables installed by the way too old table-defs in wfSchema.php.. Suddenly you have queries failing with missing and wrong table fields. wffileMods especially is shot to heck.

    To fix the issue overall, you need to change

    4 lines in wordfenceClass.php calling on networkPrefix (which fails using get_blog_prefix()) and hence FAILS to install tables correctly because wfSchema is also many versions old without being updated),

    3 lines in wfNotifications.php that falsely use $wpdb->prefix,

    3 lines in wfActivityReport that falsely use $wpdb->prefix.

    The lines in wordFenceClass.php, can also be “fixed” by simply teaching networkPrefix() in wfDB.php to NEVER call get_blog_prefix(), but simply return base_prefix. Since $wpdb->base_prefix is ALWAYS your prefix.

    Also, preferably wfSchema.php should be updated to always carry the latest table structures, as is the norm. Rather than createAll() installing table structures that are many versions old, and then runInstall() having to “upgrade” them all and beat them into into submission on fresh installs. (Any new fresh WordFence install will now fail miserably on @irisslesley’s installation).

    I know these changes well, because I have as mentioned in several “incidents” had to fix this issue myself repeatedly to get WordFence working correctly in a multi-site environment. I have the diffs.

    I would suggest that a day or two is taken off from everyone constantly playing with the file-scan functionality, and maybe fixing some of the bugs and old code that are piling up in the core WordFence code.
    Just my opinion.

    @wfasa/@irisslesley,

    It is not enough to merely change BLOG_ID_CURRENT_SITE to 3 to test the impact.
    There are more changes to it, when moving the controlling domain on a multi-site.

    In WordPress, ‘BLOG_ID_CURRENT_SITE’ is not even used at all (in ms-load.php), unless your wp-config.php also define both of ‘DOMAIN_CURRENT_SITE’ and ‘PATH_CURRENT_SITE’.
    That is (part) of how the controlling (network admin) domain of the multi-site installation moves to a new domain.

    With the following in wp-config.php:

    define('DOMAIN_CURRENT_SITE', 'thenewdomain.com');
    define('PATH_CURRENT_SITE', '/');
    define('SITE_ID_CURRENT_SITE', 1);
    define('BLOG_ID_CURRENT_SITE', 3);

    running your test code on blog-id “3”, (not as normal “1”) gives this:

    Base prefix: string(6) "ftrav_" 
    Just prefix: string(8) "ftrav_3_" 
    Network prefix: string(8) "ftrav_0_" 
    get_blog_prefix(0): string(8) "ftrav_0_"

    Confusion abound. 🙂

    In the end maybe also a bug in WordPress’ get_blog_prefix(), since it assumes that only blog #0 and #1 can be “special” (because of the initial fresh install). It tests specifically for ‘0’ and ‘1’.

    if ( defined( 'MULTISITE' ) && ( 0 == $blog_id || 1 == $blog_id ) )
            return $this->base_prefix;

    But with a different BLOG_ID_CURRENT_SITE activated (using ‘DOMAIN_CURRENT_SITE’ and ‘PATH_CURRENT_SITE’), that test is just no longer true..

    Calling get_blog_prefix() is meant for plugins/themes that do not install GLOBAL private tables. It is how they find their tables, if installed at the “blog level private” level. 🙂

    But, as I mentioned in earlier explanations, the logic to use is actually really, really simple..

    WordFence Install code (wfSchema.php, CreateAll(), and RunInstall()) ALWAYS/unconditionally/hard-coded, install all WordFence tables to live off “$wpdb->base_prefix”. All table installs and updates use a hard-coded “base_prefix” directly.
    Independent from whether you are on a single- or a multi-site installation.
    Because WordFence is always network/multi-site global.

    Since the tables are not likely to suddenly move anywhere else afterwards, that by mere logic means that any and all later accesses to these tables should also unconditionally and always be off “base_prefix”.
    No need to ever call get_blog_prefix() to ask WordPress where your private tables might be have been (re-)located to. Such a move ain’t gonna happen. 🙂
    The tables stay where you put them. 🙂 Living off “base_prefix”. 🙂
    Likewise about ever using $wpdb->prefix (including a blog prefix).. Your tables will not be found there.

    @irisslesley,
    If you notice, the fact that WordFence stops counting statistics is what I originally reported.. It is caused by most of the stats/hits DB updates failing miserably, when WordFence tries to use table names that do not exist. With numbered prefixes that do not exist. 🙂

    Depending on which of it’s 3 methods gets called in each update case to “find the table”, you will see some DB calls fail against “prefix_3_tablename” and other cases fail against “prefix_0_tablename”.. Neither of which will ever exist, since all WordFence tables are installed directly as “prefix_tablename”.. No blog numbering prefix involved. Ever.

    With the multiple requests to customize the messaging going back at least more than a year, might I suggest an intermediate suggestion?

    Use WordPress’ standard functionality for this.
    Define the current message string as the default, but simply add a call to apply_filters() with a named wordfence filter-hook before it is passed out to the world.

    That would allow more advanced users to simply add a filter function on their own to modify the messaging on the way out, without needing more setup options in WordFence, without even needing any formal documentation. It would be a “hidden” ability in WordFence that all the people that obviously already read the wf503.php code could use to modify the messaging, without actually being able to change the blocking event itself.

    Filters exist by the hundreds or thousands all over WordPress. Some formally documented, some not.

    Such a change would be simple and fast, one added line of code only, endangering no security aspects, and costing no time to implement at all.
    Yet allowing people “in the know” the ability to extend WordFence by merely filtering a string, without actually editing any WordFence code.

    🙂 🙂

    Yes, I got the technical detail from your site, but there is no security related information in those posted details at all. Just caching headers.
    First off, it is what anyone in the whole world can see every time they visit your site (which you put in the incident), and secondly those details are the same for every site in the world with a similar setup. Nothing extraordinary or dangerous exposed at all. Just random content/caching headers and tags, which change over time and per page.

    Cpanel’s editing of .htaccess not working when they use a reverse-proxy is something the hosting company would either have to fix by replacing it’s scripts to add different lines to Apache’s .htaccess file, make changes to Nginx, or they should simply remove that option from their Cpanel for users running behind Nginx control. To lessen the confusion.
    .htaccess files have no impact on the fronting Nginx server. ‘.htaccess’ files only mean something to Apache, which I assume is still running your site (behind the Nginx front).

    At any rate, the hosting company should probably better educate their support people on the configurations they actually use locally and their potential side-impacts on things such as Cpanel and on where plugins like WordFence have to find IP addresses to not simply block the reverse-proxy server every 2 seconds.

    Of course, I am here merely making qualified guesses on their actual, technical configuration. Only the hosting company can know exactly how they have set up their Nginx front to run interference in front of you. 🙂

    This is caused by this BUG in WordFence

    https://wordpress.org/support/topic/activity-report-email-bad-guys-have-left-the-internet/

    which so far has not gotten fixed (and my forum incident even marked as “resolved” to better the statistics), because supposedly I am the only site-owner in the whole world that is seeing that issue. 🙂

    So, per the WordFence guys, you cannot be seeing what you and I are seeing. (But it is supposedly in their internal systems as case “FB5929”.)

    As you can see in my explanations down below in that other incident, it is caused by faulty (very confused) handling of table prefixes in WordFence.

    This (non-existing) issue has caused me to not download any of the last three WordFence updates, because I am tired of cleaning up the code manually to make it work and stop the bad table accesses when my fixes get replaced by bad WordFence updates with still unfixed code. 🙂

    FYI, These are your site’s response headers, which show the Nginx reverse-proxy caching server being active:

    accept-ranges:bytes
    cache-control:max-age=2592000
    content-encoding:gzip
    content-length:1276
    content-type:text/css
    date:Mon, 11 Sep 2017 12:03:45 GMT
    etag:"e5f-55883754985d8-gzip"
    expires:Wed, 11 Oct 2017 12:03:45 GMT
    host-header:192fc2e7e50945beb8231a492d6a8024
    last-modified:Wed, 06 Sep 2017 11:15:15 GMT
    server:nginx
    status:200
    vary:Accept-Encoding
    x-proxy-cache:MISS

    Cpanel’s (and WHM’s) IP Blocker functionality is very simplistic.
    It does NOT interface with any Firewall software, like IpTables, APF, or CSF, or anything like that, since such features are not available to users on Shared Hosting.

    All the Cpanel/WHM functionality does is to edit your .htaccess file for you, and insert a
    deny from that-Ip-Address-Here
    line, which would make the web-server (Apache) block the access for that IP when seen.
    Cpanel actually runs through the .htaccess files for ALL domains you might be hosting in your account and inserts the same line in all of them.
    Everything after that, Cpanel is completely oblivious to. Cpanel does NOT partake in any actual blocking of that IP.

    So check your main .htaccess file in the root of the web-site..
    If there is no longer a “deny from” line with that bad IP address, then the Cpanel line at some point has been deleted from that .htaccess file, and the IP is now unblocked as far as Apache is concerned (leaving the IP to hit in WordFence).

    Your host should have explained that.. It is true, that as long as .htaccess is blocking it, WordFence would not be seeing it.
    But that requires the .htaccess line to actually be there (still). 🙂

    Note also, that all this also depends on how your WordFence “How to get IPs” is set up.
    For example, if you are behind a caching- or reverse-proxy service, WordFence would be getting it’s IP addresses from the proxy headers to get at the true IP address, and those are what you see in it’s logs.

    The .htaccess Deny line, used by Cpanel’s IP blocker, ONLY works on ips found in REMOTE_ADDR, which in the case of a caching front-end would be false. REMOTE_ADDR would for all accesses be pointing to the caching server:-) The bad IP would never be seen.

    IN YOUR CASE, they ARE running a Nginx caching proxy server in front of you. Your site headers show that. Depending on their setup and other local changes, that could have left any and all “Denies” in .htaccess/Cpanel totally non-functional. Your host should also have told you that.

    In summation.. That IPs are getting as far inside your walls as WordFence has everything to do with your hosting setup, and nothing to do with WordFence. 🙂

    Thread Starter Caleb

    (@crudhunter)

    I should mention, that the only reason I fell over this is_admin() issue (which probably should be a call to wfUtils::is_admin() instead), is that I am hunting code for a type of visits that seems to consistently escape Live Traffic logging (at a minimum). One I haven’t been able to figure out yet.

    I have a window on a monitor showing filtered Apache logs, and I keep seeing visits, from a multitude of IP-addresses, that arrive (and are successful) according to the Apache logs, but those same visits never show up in WordFence’s Live Traffic view running in a second window. Mind you, these are NOT /wp-admin area visits. Just normal scrapers and bots, arriving on for example the front-page.

    At first, I thought it was merely a display or filtering issue, but checking the wfHits table, the visits are not there at all. Nothing registered. Those visits turned invisible. Despite these visitors for sure having arrived on the site(s). As witnessed by the Apache logs.

    Haven’t figured out why yet, but from what I can see it is pretty consistent that all the log-escaping visits seem to have one thing in common. They all arrive from a 301 Redirect.
    As in, the bot tries to hit a http:// version of the site, which is NOT available, and are automatically redirected by .htaccess code to the “https://” secure version.

    Both accesses (1st one hitting the redirect) show in the apache logs, but I would expect the 2nd access (on https) to show in Live Traffic as normal.
    IT DOES NOT. It is not logged at all. I have not been able to reproduce it by simply trying to access the http:// version myself. Something circumspect is happening to these particular accesses and WordFence’s internal tests on whether to log a “Hit” or not.

    Anyway.. Along the way, the call to the wrong “is_admin” came up. But that bug is not actually relevant to the original problem I am hunting, since none of the Live Traffic “invisible” accesses are to the /wp-admin area.

Viewing 15 replies - 16 through 30 (of 218 total)