Caleb
Forum Replies Created
-
Yes..
Your blocking period on the options page is set way higher than the default of only 10 minutes (since your messages claims 14+ hours to unblock, I assume maybe 24 hours?).
If you set the block for 24 hours, that means that “recent” here means that they, according to the message, got blocked 9 hours 37 minutes ago. Never seen since then.But had the blocking time been e.g. 10-30 minutes, these IPs would have fallen off the temporary tracking a long time ago. (to be caught again, if they come back to misbehave.)
They are only hanging around in your lists because of the long blocking time.
So, someone (frequently on a dynamic IP, never to be seen again) arrived at your site, misbehaved, and got themselves blocked.
But they have not returned and attempted access again “since” then.
Key words are “since it was blocked” (read: AFTER being blocked).It does not say that they were never blocked. Merely that they have not been seen and blocked “since then”.
The auto_prepend would normally be only on one place. So if you found it in .htaccess, that would be it.
You really only need to add the
php_value auto_prepend_file nonein the /forum directory’s .htaccess.
You are simply turning off running of the WAF the forum software can run.Well, the call for wordfence-waf.php is obviously there, since it bombs the site, and it only has those three options to be called (.htaccess, php.ini, OR a ‘user ini file’ ), sooo..
If you are running FCGI/PHP-FRM, it will be in the .user.ini file (notice the DOT in front).
Of course, the ‘.user.ini’ is merely the default name, since the actual name for your local “user ini file” is a configurable option in PHP, that could have been changed by your hosting company. But they rarely do. Causes confusion.
Checking phpinfo (from command line or from any plugin with a “Diagnostics” page showing PhpInfo) will confirm if your hosting have strayed. (Since you no longer have a WordFence Diags page 🙂 )
Like[~]$ php --info | grep user.ini user_ini.filename => .user.ini => .user.iniWordFence will have used whatever name is configured in PHP for that file. It is not a hard-coded name. But usually installations do not stray from the default name of ‘.user.ini’.
If you are not running FastCGI/PHP-FPM, then a .user.ini file has no impact (is not used by PHP), and one of the other two options (.htaccess or php.ini) must have been used.
The “auto_prepend” call should be there in your files.. Somewhere. 🙂
Another way, if only certain directories are not allowed to be “indexed” (show file-listing), would be to add a local .htacess file in that directory disallowing it, or to simply create a dummy “index.html” for Apache to find in that directory. 🙂
Then magically that index.html file will be shown instead of a full directory listing..
Showing file-listings is only Apache’s fallback, if there is nothing else to show, and when it is allowed to by the “Options” settings..When you disallow it in .htaccess, users will get an error message if trying.
Which is the normal way to do it.Has nothing to do with WordFence. This happens without WordFence having any knowledge OR a way to control it.
Apache (your web-server) by default, and if it is allowed, will show an index of a directory, if no index.php/index.html or similar exists to show.
Normally, this is turned off in the global configs, unless you have a local option to turn it ON.You need to tell Apache that showing directory content is NOT valid.
Normally done in the global configs, you can also do it by simply entering<IfModule mod_autoindex.c> Options -Indexes </IfModule>in your .htaccess file. That will tell Apache to stop showing directory indexes.
Two different kinds of “protection”.
Most main security is set up to protect your site against hackers or similar badders. Those logging in, trying to infect your system, use it for bad stuff. (as in, get access to your PHP based stuff for their own purposes, such as spamming.)
HTML files do not get “hacked” typically. It is “just” content. Unless someone got direct editing access to it, they cannot “infect it”. Plus, cached files get continually re-generated by the caching plugin, so infecting these files would be useless. It would only last until the next cache-rebuild.
The type of protection you might think about there is protection against scrapers (stealing your content), and that is a whole other thing.
You can easily block IP ranges through simple one-liners in your .htaccess file.
But if you want to rate-limit scrapers, and do more advanced limiting for stuff served up directly by your Web-Server (Apache or NGinx), you would need code and rules that run either IN or BEFORE Apache/NGinx.For Apache, that for example means running such as the Apache ModSecurity module, which can through many rules both protect static pages AND protect the site itself against login hackers and many, many other things that happen further in. (IN fact, most of the firewall rules deployed in WordFence run equivalently as ModSecurity rules.)
ModSecurity rules can also do country blocking, IP blocking, permanent and temporary banning of IPs based on their actions, …, …BUT.. And this is a big BUT.. To be allowed to control your own ModSecurity rules, you MUST be running your stuff on a VPS or Dedicated server.
Since such rules typically can affect server-wide, rather than merely one web-site, and since one single syntax error in a ModSecurity rule-file can prevent the web-server from starting, they cannot allow people on Shared Hosting to control ModSecurity rules. Since all sites on a Shared Hosting server share the same web-server process.Also. although many hosting platforms run some level of ModSecurity (even without you knowing it), they cannot deploy most of these rules, as they would affect some web-sites. For example, WordPress Admin pages will innocently trigger at least a handful of some of the standard used ModSecurity rule-sets, such as OWASP rules, stopping posts from being saved. Unless one is following the Apache and ModSecurity logs directly during deployment, it would merely be seen to a WordPress owner as WordPress failing, without knowing why.
To deploy ModSecurity is a process, where some “rules” have to be selectively turned off for certain admin pages as one find these conflicts. (By looking in Apache/ModSecurity logs.)
Another option, which allows only for IP level blocking is to use firewall rules.
Such as IPtables (typically only available on Dedicated servers), APF (Advanced Policy Firewall) interfacing to iptables (VPS and Dedicated servers), and other simpler IP-blocking available to shared-hosting customers in eg. Cpanel.But since static content (including plugin generated page cache) is merely content being served up by the web-server (and that is its job), only teaching the Apache/Nginx server or a layer EARLIER in the connection process can stop it.
As mentioned, there is no way anything based in WordPress and PHP (run AFTER Apache/Nginx in the connection line) can stop access to static content.
If PHP/WordPress/WordFence is not even called up to run, it cannot take any actions either.On your mobile/non-mobile question. It all depends on your choice of WP theme and choice of caching plugin. On the typical WP themes the HTML code is made to be adaptable on its own. And most caching plugins doing their caching through creating static pages makes NO decisions in that delivery process, except some of them create multiple versions of cached pages. But for the typical caching, through caching WP plugin, caching front-server, or other means there are NO decisions to be made. No call to WP/PHP at all. Heck, for users with a fronting caching Nginx server for example, the site do not even get called at all. The front-caching server pushes out the content without you even seeing the access. 🙂
Cached static pages is also why things like side-bar Widgets adapting to user country/interests/cookies typically do not work unless they are based on Ajax (call-backs to the server) or are pure JavaScript. Any non-Ajax Widget that makes content generation decisions specific to a visitor will fail.
For non-Ajax widgets, if a Russian IP was visiting when the cached page was created, all the subsequent visitors (from anywhere) will share the Russian’s view, because it is now hard-printed into static (cached) HTML. 🙂But.. Caching methods and their many workarounds is a discussion for another group.. Not really related to WordFence. 🙂
As you said. No way to know.
The plugin (and you) cannot know what is valid and trust-worthy to the individual sites, unless it is a direct-connect setup.
Unfortunately that is something the site-owner has to investigate (ask their hosting provider). Unavoidably making their life more complex. 🙂Adding to the complexity is the fact that some site visitors, even in the site-direct-connect cases actually arrive by a valid proxy. Where the address might need to be translated before blocking happens, or thousands of potential visitors could be killed off in a single blocking action.
For example, all mobile visitors arriving from AT&T Wireless and some other ISPs will arrive by proxy, with a standard header carrying the “real visitor” IP address. 🙂
Block the proxy IP, rather than the real visitor, and magically all (or most) mobile visitor’s from that provider would vanish from the site, because the AT&T proxy’s IP-address got “blamed” and blocked. So even in a direct-connect case, it can be unsafe to merely choose “use REMOTE_ADDR only”.Fortunately, not too many bad guys show up on Wireless IP addresses, unless they spoof the headers. No one wants to pay wireless data fees for their hacking activities. 🙂
The Internet is such a cool (and tricky) place, Eeh. 🙂
@doit686868, Well.. 🙂
Actually solving the problem can only be done by the Shield folks providing a way to “lock down” which headers should be accepted and used as valid on individual sites. Which is VERY specific to various hosting setups. (Direct connect, reverse-proxy, caching front, Managed WordPress, …, …)
Things only the local site-owner can know, and frequently cannot be detected by the plugin other than by guessing.. 🙂Apparently in the particular configurations for you and @melodies, the fronting server is passing along one or multiple headers trying to tell y’all about the outside visitor (as they should), but Shield is making the wrong picks from those headers.
The hosting company support folks are right.. There is nothing they can do. They cannot and SHOULD NOT turn off sending those headers, because that would simply end up with the plugin banning all access to your site. The front-server’s IP address would get blocked, since that is physically the only “visitor” you actually have (by REMOTE_ADDR). It would then get blamed for all bad accesses, rather than the “real visitor” listed in the headers. 🙂
Only the plugin owner can fix this.
By making the “simple configuration” less “simple”. 🙂Can I ask a dumb question? Are all these sites yours?
If they are, if you have 10 sites, all running on the same VPS, why are you not running them in a single WordPress Multi-Site setup? Rather than as individual separate WordPress copies, requiring independent WordFence, independent site scans, independent management, …, …?Just curious.
Unfortunately, Wp-Simple-Firewall (AKA Shield) uses a very insecure way of gathering IP addresses. (As a side-effect also allowing anyone wanting to, to spoof it with fake information.)
Most of the below headers should ONLY be used when they are known to be trust-worthy. Such as when a trusted front-end caching- or reverse-proxy is KNOWN to be setting them.
@melodies, In your case with a Nginx front, the hosting company will have put the “real visitor IP” into a header with one of the names below. Such as “HTTP_X_REAL_IP”, which is apparently used in @doit686868’s case. They could have chosen another, but they know which. As a site-owner, one has to pick the header that is trust-worthy AND reliable for that particular hosting setup, if REMOTE_ADDR cannot be used. All others (except REMOTE_ADDR) can be spoofed by a visitor by simply sending that header with a random IP in it. 🙂
What is happening is that because you have front caching (or reverse-proxy) server, the only 100% trustworthy header (REMOTE_ADDR, from the TCP connection) will ALWAYS show that your front-caching server is your visitor, and the real visitor’s IP is then passed along by the fronting server in one of the other HTTP headers.
Wp-simple-firewall grabs whatever it finds first as if it is the real visitor IP. Logs it, might block it if it misbehaves, …, …This is the order of priority it uses for grabbing IPs from headers:
'HTTP_CF_CONNECTING_IP', 'HTTP_X_FORWARDED_FOR', 'HTTP_X_FORWARDED', 'HTTP_X_REAL_IP', 'HTTP_X_SUCURI_CLIENTIP', 'HTTP_INCAP_CLIENT_IP', 'HTTP_FORWARDED', 'HTTP_CLIENT_IP', 'REMOTE_ADDR'Without a configuration to “lock down” which headers should be considered true and valid, the results can get kind of random, plus as mentioned it allows for bad guys to spoof the security by passing along random IPs in the headers, rather than their own.
That is the nature of many types of caching. Both WordPress Caching Plugins or when a front-end caching server/proxy is deployed. The caching functionality turns pages normally generated by WordPress into static pages, where PHP/WordPress/WordFence is not even called up. That is why cached pages are faster. The page is delivered directly by your Web-Server.
If WordFence/WordFence is not called up to work, they obviously cannot test and block access.
Only fix to this is to move certain blocking “left” or “up-stream” in the connection path to stop access directly by the Web- or Caching service Server (typically Apache or Nginx). Not something WordFence (a PHP based WordPress plugin) can fix.
It is simple.. When blocking is done as late in the game as in PHP, no blocking can happen if PHP is not even called up. 🙂
Well.. I have a separate “incident” in this forum about these parameters WordFence “Injects” into our pages for Google to find. And I believe WordFence need to FIX it (stop doing it). Especially since they obviously very frequently see GoogleBot as “Human”, and use GoogleBot to handle “Callbacks” to themselves.. 🙂
The URL removal tool is not for this kind of issue. That will be a battle fought until end of time, since that does not remove Google’s memory of the goofy parameter that it feels needs to be tested on all pages. The URL paths are NOT the problem. The query parameter is.
But I might suggest a different solution to a case like this, where so many have been indexed and where WordFence is no longer in the picture.
Add a couple of lines of code to .htaccess, wp-config, or similar, that checks for the goofy parameter and 301 Redirects (Permanent Redirect) to the same URL as called for, but with the parameter removed.
Similar to a site-move, that should over time automatically make Google’s indexes eliminate all memory of the bad parameter. Google will learn, that any attempt to use the foolish parameter, just to check it’s effect on various pages, will have NO effect, other than being asked to remove it.
It will at the same time eliminate all indexed page versions with that parameter from the indexes, as GoogleBot should silently move URL versions WITH the parameter over to their versions WITHOUT it.
Then just sit back, watch, and wait. It will take time, since GoogleBot might have to revisit all these pages probably multiple times to verify that the 301 Redirect is really permanent, but it WILL happen. 🙂
You might see a page first disappear from the index, and then reappear, since there is a lag between the “old” version disappearing and the “new” version getting officially indexed.Mind you, this works because WordFence is NO LONGER on the site.
If it was, new cases of parameters would still be injected into pages for one (confusing the heck out of GoogleBot), and secondly, it also blocks WordFence from receiving these parameters, and hence interferes with it’s internal accounting. 🙂Just disconnect it in .user.ini, .htaccess, or php.ini, wherever the auto_prepend config was put.
Should have been done before deleting the file.
See https://wordpress.org/support/topic/wordfence-waf-php-2/