Caleb
Forum Replies Created
-
All the tables are created on WordFence install in a simple loop. One after the other. There is nothing special about the 3 tables you list as actually created, and in fact, the tables you end up with in this case are not even “in order”.
The Notifications and KnownFileList tables are the last tables created of all, and PendingIssues somewhere in the middle in the order.
That tells me something.
a) Your database filesystem likely was out of disk-space or iNodes at the time. Because of other things going on in your system something freed up space or inodes in the middle of creating tables. WF managed to create an initial table, then failed on a long list, and then managed to create the two last ones. 🙂
Just a bad situation.b) WordFence does not check any errors returned from any table creation. Once in that “createAll” loop, it continues running even if something fails. Maybe they can improve that some. Creating the necessary tables should be a “create All, or create None” type thing. MySQL does not support DDL type transactions, but that is the principle that would have to be implemented manually.. If a table creation fails when in that loop, then roll-back/undo all the previously created. Start fresh.
But honestly, running out of space in the middle of table creation is such a rare occurrence, that I don’t see that as making much difference in the large picture.
WordPress itself is such a mismatch of code, that this “problem” can show up anywhere. Fixing one plugin out of thousands is of little consequence. 🙂Yes, you can create the table by copying their definitions from another site. No problem.
BUT! The prefixes must be correct. You must adjust the table name prefix to match the site you are creating them in, or they twill never be found again.I notice from your text that the system where they failed run with a WordPress base_prefix of “sbqte_” (a choice you made during the WordPress install). But the tables you list as trying to delete/create are using the prefix “wp_”, which is the DEFAULT WordPress table name prefix. You might be using that on another site?
So.. You can create the tables manually by copying their definitions from another site, but you must adjust the actual table names to match the site you are creating them for. Otherwise WordPress cannot find them. You cannot install “wp_” named tables on a site defined to use the “sbqte_” prefix on tables. The “wp_” named tables will appear from a WordPress perspective as if they were never created in the first place.
I would suggest you set up some simple automation to warn you when your DB filesystem run down on space or inodes.. Like by sending you an email warning when running under 20% or some number. WordFence, WordPress, or any plugin: when you run your Database filesystem out of resources, the results will frequently be indeterminate.
If that happened in the middle of for example saving a post in WordPress, I can guarantee that your options, posts and other tables would likely be out of sync as well. Parts of the post information would be saved, other parts not.
Yes, certainly.. It is a buggy pattern match and change done by WordFence.
Also called “a bug” in the code. 🙂WordFence only touches your CSS and JS links when you turn that option on.
It tries to (only means to) change the “?ver=x.y.z” type parameters on the links, but uses a somewhat simplistic preg_match and code pattern to do so. Your site must be using some odd links that end up running the pattern astray.Why it affects just YOUR site that way (by mis-matching and changing the wrong thing on your particular styling links) I can only say if I knew what your specific links look like. Which depends on theme names and other things. Then I could replay that pattern in my head on your link/theme structure, and see where the matching and code goes wrong.
The problem in your case goes away when you stop trying to “Hide the wordpress version”, because WordFence stops trying to change your CSS/JS links. Stops using the poorly constructed method to find version strings to change.
You don’t need that option anyway.. As it states in WordFence’s own help documentation, they actually suggest you no longer turn that option on. There are so many other ways for bad guys to know that you are using WordPress that “hiding” is kinda useless. 🙂
And it just adds to the time it takes WordFence to run, without any benefit.
Just look at the page source. It is filled with WordPress and plugin unique paths.
Bad bots don’t care what version of WordPress you are running. Only that you are in fact running WordPress, which is very obvious to anyone looking. All they have to do is ask Google or manually click “view source” on a web-page. 🙂BTW.. I was exaggerating when I said “unlikely” to make a difference. 🙂
Just hedging my bets, since I was just making a qualified guess.. 🙂When Godaddy says “Not our problem”, it simply means that their system is doing exactly what they expect it to do. That they do not see anything that they want to fix.
503 error is how they limit your resources on ALL their hosting accounts.
Google GoDaddy 503, and you will see endless hits on people seeing that issue. That is not an unusual way to limit usage for any shared hosting.They also state it directly in their help-pages. See such as
https://www.godaddy.com/help/website-errors-503-service-temporarily-unavailable-5089
Personally, a number of years back I for years had a Dedicated Server fronted by an also private Cisco Firewall with them. This worked fine, because on a dedicated server I control the limitations, they impose none. It is “my” hardware.
My only shared account with GoDaddy could never be used to run any real kind of site at all. It only held a 1-2 page service site.
On sitemaps.
Submitting sitemaps to Google is merely something one has to do to tell them about pages that are harder or slower to find.. Gets them started. But GoogleBot WILL over time automatically find any and all other pages that are naturally linked up on your site. Most sites do not even need a sitemap. Google just follow all the links it finds across your pages, and WILL eventually find all your pages.
Your pages do not disappear from Google search because Google cannot use your sitemap. That is completely irrelevant to what pages go into search.
Your pages disappear from Google because THE INDIVIDUAL POSTS and PAGES are returning 503 errors. Google will not direct their search customers to failing page content. But they wouldn’t dream of sending a searcher to your sitemap file.That is also why your Google Search Dashboard shows all the individual pages as failing with 503s. Why you say thousands of 503s.
That loading your sitemap from your server fails with the same “I am overloaded” error is just an irrelevant fact, but caused by the same reason. That GoDaddy is heavily limiting your resource usage on ALL shared accounts.For all but Dedicated Servers (where you are in control of the entire piece of hardware) GoDaddy has a long history of severely overloading servers with domains and sites (more accounts per server, more profit) and hence having to limit each account’s resource usage to an extreme degree. It is what they do..
Check on the Internet how many domains are assigned to your IP address (meaning that the same server/cluster is servicing both your domain and those others).. I once did for fun.. My shared IP with Godaddy had more than 4400 other websites of varying sizes on it. And each server can have multiple/many IP addresses assigned to it, so multiply up from there. 🙂 All those sites then in addition on the back-end share database servers, which means that they only see a minuscule portion of the database server’s query caches, if any at all. Something that can only be helped with front-end Object Caching.
That is how shared hosting is done. Nothing new there. It depends on the hosting provider how many sites they decide to load on each server.
But independent of provider, this is how they make a profit on selling $3-$6 hosting accounts. By putting many thousands of domains/web-sites on the same server-structure, and not allowing any individual site to steal too much resources from the other thousands of sites sharing the fun. 🙂And if you really have a combined 11,000 pages/links, your account is definitely too large for small shared hosting.
Think about it. Just the amount of hits you get from the Google/Yandex/Baidu/Naver/… search engines (and all the unwelcome bots) wanting to check and recheck all your pages repeatedly can be a serious killer.
If you then add WordFence checking each new connection against “bad stuff” (and hence slowing down) all those hits so they hang on longer fractions of a second, the problem multiplies up heavily.But unless the WordFence folks know of a specific problem, where WordFence makes connections hang around for much too long, your problem seems to be a generic overload one.
So, here is a dumb, unlikely, but “quick to test” suggestion..
If you have the option “Hide WordPress version” switched on in your options page, could you try switching it off and retest?
That option is not really of any use anymore anyway, so turning it off has no negative impact. But it could potentially explain some things.
It could be unrelated.
class-wp-widget-text is WordPress’ standard text widget. Either called alone, or through an Extending widget installed on your site. So that is easy to check out.
Go to your widget panel and disable any Text-based widgets you are using. (“text” here also means that they can contain HTML.)
Temporarily park that/those widget(s) in the disabled area to save and keep their configurations for later.)That would make the DOM problems seize to exist, since you no longer use the text widget. The DOM problem is likely because some invalid HTML was typed or pasted into that text widget’s value field.
With the widget(s) disabled, you would see whether that also have an impact on your 500 errors.
With 403 (Permission denied) errors, it could be WF blocking access, and hence Learning mode or white-listing would a good idea.
However, with 503 (Server overloaded, don’t wanna respond to the requests) we are probably unfortunately into an area that need more close view on server/site configurations and some debugging. Probably a quick thing to fix on a system one can see and touch, but without seeing it, I am merely rattling off some of the many potential problems.
Some (shared) hosting setups limit the number of Apache or PHP worker processes you can use at a time. To protect the shared resources on that server. In some cases down to 3 worker-processes only, which means that there is a severe limit on how many requests can be handled in parallel.
Generally, you hit the 503 error either on real overloads, or they can happen if for example a bug in a plugin cause a never-ending loop. In such a case it never relinquishes it’s PHP worker process for the next web-call to be serviced. You eventually loose them all, as they might be killed off only after hitting the CPU limits set in your Apache or PHP configs.
BUT.. In this case, with a hosting account holding 2100 + 9000 pages/links that Google and all the other bots want to pull on and check out, all it takes to see 503 errors is a general slowness in handling requests (like when you re-enable WordFence with a large number of accumulated blocked IPs and accesses to check against).
The fact that it works with WordFence disabled could either mean that WordFence has a real “never-ending-loop” type bug, which I at this point consider quite unlikely.
Or it could mean that you simply need to reconfigure your hosting setup to actually match the load of a larger/older site-need. Or, see further down below.You don’t mention your hosting setup. But if on a VPS or dedicated server, you would go check how many processes are actually being used, and reconfigure your setup.
With PHP-FPM/FastCGI you would go up the number of PHP worker processes allowed. Same for Apache.. If your site STILL gets exhausted and issue 503s, then maybe you have a looping, never-ending plugin. In which case, upping the # of allowed worker-processes would help little or nothing. It would just eat up the additional workers as well.On a shared hosting setup, you usually cannot just go add more workers, as they cannot allow you such access.
Another distinct problem could be, if you have accumulated a large number of IP blocks over time. Especially if you block many IPs one at a time, rather than blocking off whole ranges. Making WordFence slow down over time. Having to work harder and harder to check incoming accesses (in slow PHP and WordPress).
In that case, you would need to reduce that load. Either just by consolidating all those IP blocks removing overlaps, or by “moving some blocking Left in the path”. Like by moving the majority of your current IP blocking from PHP (WordPress/WordFence) into Apache (block with “Deny” in htaccess), or even further left in the access path into a real firewall. That will remove a lot of load. Spare PHP and WordPress from even getting started on most IP blocks, by allowing Apache to quickly block them early on.
(There is a way to easily transform your accumulated WordFence IP Blocks into Apache or firewall type blocks, should you wanna try that route).Another potential problem that could significantly slow your system response, and hence cause 503 overload errors, is if your dead/expired transients are getting overblown. Without Object Caching, transients (temporary data) are sitting in and clogging up your options table. That can make it grow exponentially. Install a plugin like WP-Optimize. It will show you how many expired transients your site is carrying around, and allow you to clean them out.
I could continue the list of potentials that need to be checked out to get your setup down under it’s overload situation. Better caching configuration come to mind as well.
But all of this is a kind worthless exercise for a system I cannot see or touch. When one is unable to actually check out how it is behaving in the dance that is serving web-requests. Something is obviously stepping on your system’s toes and hitting its bad bunions.The solution might be in WordFence if you can find a pattern in their IPs. Like they all fall within the ownership of a certain proxy or VPN vendor’s ranges. Some scammy vendors, like HideMyAss, iPredator, or others helping scam artists are selling spammer packages of 500-1000 rotating IPs at a time, but they are all owned by that vendor. Or maybe all IPs come out from a certain hosting company, a certain country, …
Or they could be abusing the TOR Onion network, which could make the IPs run around the whole world.You cannot block each IP one at a time, since that may be a never ending task.
Who do these IPs belong to when you look them up? If you are sure you can recognize the accesses that stem from the reverse proxy.
Well.. You are giving up too easy.
You only have two options then. You either find a pattern in the IPs the headers used, and block the fool directly from accessing your site.
Or you do a DMCA report to Google convincing them that the other site is merely stolen content, and they will then kill it off and eradicate it from Google Search. In effect making it useless since it will drive no traffic. They might even kill off the person’s ads account, since if they are using Google ads, Google obviously know exactly who these people are. Down to their bank-account
They will then move on to duplicating someone else’s site, if they are allowed another ad account. But generally speaking, it is on you to take action..
That blocker line in htaccess has nothing whatsoever to do with WordFence. It is a standard line of tests for keywords in User-Agents, that either you or another admin likely added to the htaccess sometime back in history.
It was copied into your .htaccess from one of the many standard tutorial examples using it.. In this case likely lifted off from this blog posting, since it is tagged as Perishable Press’ socalled “5G” list:
https://perishablepress.com/5g-blacklist-2013/
But they did not make up that rule for their “5G” list either..
They merely collected it from one of many other tutorials using it, such as from TutsPlus, where that exact code existed at least a year earlier in their tutorial.As you can see in that user-agent rule, it contains the section “|comodo|” in the list of keywords it tests for. So check your logs and see if the word Comodo exist in the user-agent when checking the SSL..
Cpanel’s AutoSSL is a quite new feature, and did not exist back when these tutorial examples were set up. So they would never have expected it to block valid things like that.
You can just edit the rule and remove the offending word(s), likely just the “comodo” part, allowing them to check their SSLs.
Well.. The fastest would be to simply turn it on for the whole site. In your PHP.ini, .user.ini, or .htaccess file.
Since your failure is in the back-end, it’s not like your front-end visitors would see a difference, since thats not where the failure is.To turn it on Only for the back-end, you would have to turn it on depending on the scripts called.
The simplest and fastest is the ugly way. Temporarily edit the affected PHP file(s) in /wp-admin and add
ini_set(‘display_errors’, ‘1’);
in PHP directly. Adding it to the top of customize.php would be my first step.
Another way would be to set it, without editing the PHP file(s) is to set it from .htaccess on a conditional of the path used. For example turning on for all paths matching ‘^/wp-admin/’
In htaccess it is turned on using a line like
php_flag display_errors onMight wanna add
php_flag display_startup_errors on php_flag html_errors onas well, while at it. Just in case.
in PHP.ini/.user.ini, the format is
display_errors = 1
or
display_errors = OnI guess you’d have to actually debug. 🙂
Do you have display_errors tuned on for that back-end section, so you can see any PHP errors or stack-traces that pop up? Might be needed to tell you what’s going on behind those curtains.
Only a few ways a mirror site can set this up to mirror that closely.
Either their site is running your’s in an iframe, which would be pretty useless to them, as it would be ignored by Google.
An Iframe you can protect yourself against. Add something like<script type="text/javascript"> //<![CDATA[ if (window.top !== window.self) {document.write = '';window.top.location = window.self.location; setTimeout(function(){document.body.innerHTML='';},1);window.self.onload=function(evt){document.body.innerHTML='';};} //]]> </script>in your theme’s header tag.
Or they are more likely running a reverse proxy connecting back to your site, where every action on their site is mirrored by calling up the content from yours.. That’s why it looks as if it is “immediately mirrored”, because in fact it is. They do nothing, you do everything. 🙂 And Google’s bots would not be able to detect it, so they get indexed.
In the reverse proxy they can also on the fly auto-edit some of your content. Like for example change your copyrights into their names.
But without seeing the sites/domains, hard to say.
Not really much WordFence can do about an “evil” reverse proxy, unless you block the IPs, or rather the whole IP range they are calling you on. I would assume that the dynamic IPs are not entirely random or spread all over the world.
Hmm.. Then I guess the 500 errors will have to be investigated especially.
I do know that the automated while-listing of /wp-admin/customize.php is how mine got white-listed.
I specifically remember the puke WordFence did on that URL, where I had to claim it as OK, adding it as an exception. And it is the only URL at all that is white listed in my config.As it states in the listing “Whitelisted by via false positive dialog”.
In the table as
Filter URL: “/wp-admin/customize.php”
Filter Param string: “request.queryString[return]”/wp-admin/customize.php is a standard URL that typically have to be white-listed because of some action it (validly) does that the firewall will otherwise block. The POST parameters are “suspicious”.
Need to white-list that URL on the firewall page.
Can be done manually, but another way is likely to switch WordFence to learning mode, run the customize.php functionality, watch WordFence complain, block it, and ask you what to do. Then declare it a false positive. That will automatically add the right parameters into the white-list table.
@11whyohwhy15, as the page info you pasted in indicated, what the correct number is depends highly on your page content, type of theme, smarts of your widgets, and other factors like Ajax.
For example, if you are using smart widgets, where some might show dependent content based on the visitor (like based on country), then to bust page caches, they frequently use Ajax (call-backs to the server) to call in the localized pieces.
So if you load a page, but that page has 4 widgets each doing just one Ajax call, then what looks to the visitor as a single page is really 5 calls to the server. Not counting the other dynamic things that could be going on.So it is not that a human user can read 240 pages, or would even click that fast if just browsing through. It is that each single click depending on your site design could be multiplied up several times.
Hence, YOU, the site owner is the only one that can determine what the right number is for your site. I think the default is merely set so high that it is unlikely to suddenly block off half the page content because access limits blocks it off in the middle of a page, and generate support calls for THAT reason. 🙂
On determining which accesses are truly human or not. Hard to do reliably.
That depends on how “dumb” the robots are created (most are pretty stupid).
A really well designed robot/crawler can appear VERY human in how they access the site.Heck, forum spammer bots like xRumer and it’s cousins for blog spamming have “long term” planning in them to appear human.
Register on a site one day.. Then post a few automated but completely bogus “replies” over a couple of days to appear like a “real, active forum member”.
THEN START spamming like crazy after gaining forum cred and the site allowing links. 🙂It’s all in the programming.
But most robots are simplistic. They miss out on sending certain headers, so are clearly not from a real browser.. Don’t load CSS/JS or other things, and so despite their agent-strings claiming to be a human browser, they are obviously not.
There are a ton of things that COULD be used for an estimated guess on what is human or not. None are 100%.
Not even Google Analytics is 100%, because any real human visitor, arriving with such tracker blockers as Ghostery will prevent from loading up the Google Analytics JS scripts. So these users can browse as normal, read every interesting page on your site, and Google Analytics would be none the wiser. 🙂 Google depend on their Javascript loading up in that person’s browser.