WordPress.org

Forums

iThemes Security (formerly Better WP Security)
Memory Limit Reached Error (17 posts)

  1. bravewhale
    Member
    Posted 1 year ago #

    I'm noticing (and heard from other visitors) that an error message will come up randomly during browsing of my site. See shot here:

    https://www.dropbox.com/s/3sdx1o66fwagwl5/bw-mem-ex.png

    It's a memory limit reached error. I asked my host, siteground, where I have a cloud vps account, and they upped my memory and said it doesn't look like it's caused by traffic as I don't get a ton. So maybe it's a plugin causing it.

    Is there are any case where the security plugin throws this error mistakenly? I find it hard to believe that a plugin somewhere on the sites I host on this vps are using up so much memory that it stops loading the page and gives an error.

    Thanks for any insight you can provide!

    https://wordpress.org/plugins/better-wp-security/

  2. Placeholder
    Member
    Posted 1 year ago #

    I was having the same problem! PavelCZ1982 seems to have tracked it down, though.

    It is causing random crashes of site when you have got allowed option for scheduled backup of db.

    I disabled scheduled backups and have had no problems since ...

  3. Placeholder
    Member
    Posted 1 year ago #

    Forgot to mention that I installed UpdraftPlus for backups instead. Works like a charm.

  4. bravewhale
    Member
    Posted 1 year ago #

    Wow thank you for letting me know!

    I'm guessing that automatic backups was enabled by default for this plugin. The only place I found a setting for this was under Security > Settings > Database Backups. I now unchecked that options, so hope it fixes it.

  5. joe.toomey
    Member
    Posted 1 year ago #

    I just dealt with this same issue. I imagine there may be some debate about whether this is a bug or not, and it's true that you can typically resolve the problem by increasing the amount of memory allocated to php (first in your php.ini file, and perhaps also necessary to remove an apache RLimitMEM restriction if one is in place.)

    I would argue that the root cause is the fact that the iThemes Security plugin (which is fantastic, by the way) assumes that a sql representation of the entire database can fit in memory in a php string. It does this in better-wp-security/modules/free/backup/class-itsec-backup.php in the function execute_backup(). In this function, it populates a string called $return with the full sql content of the backup of the entire database, and when it's complete it writes that string out to a file and optionally zips the zip.

    I'll repeat that I think this plugin is fantastic. I do think, however, that it's not fair to assume that the entire wordpress database can or should fit in memory. In the case of the site I was investigating, the Redirection plugin had created an enormous log table (whose name began with the wordpress wp_ prefix.) Our site itself typically consumed less than 40MB at steady state, but we were still blowing up php during the backup process with 256MB allocated to php. I confirmed with some debug output that the backup process indeed consumed all of that memory before crashing. This was worsened by the fact that iTheme Security appears to retry failed backups very frequently (in our case every 3-4 minutes), and each of these retries resulted in the same out of memory error, crashing php.

    I think it would be a fairly straightforward refactoring to write the .sql file in chunks rather than storing the entire file contents in memory. This would make this excellent plugin even better, and would help a lot of people who are mystified as to why their modest wordpress sites are blowing up quite large php memory limits.

  6. Placeholder
    Member
    Posted 1 year ago #

    Thanks for sharing the results of your legwork!

  7. joe.toomey
    Member
    Posted 1 year ago #

    I'm not sure if the iThemes Support people monitor all of these threads or not, but I have a question:

    If I were to fix this bug, could I submit a patch to be incorporated into the plugin? Do you have a git repo where I could fork and submit a pull request, or alternatively, could I just post the source here?

    (Note, I'm not going to write and test the code if it's not going to be folded into the production plugin; I already "fixed" the site I was working on by reducing the size of a table that was unnecessarily large. But I do think this is something that should be fixed properly.)

  8. bravewhale
    Member
    Posted 1 year ago #

    I'm not certain how this works for their plugins, but you should probably head over to the iThemes support forums and post this over there. Link to this thread and see what they say. That would be awesome if you could submit your patch!

  9. Tim
    Member
    Posted 1 year ago #

    I can confirm something is definitely hitting the memory limit.

    This is displayed in iThemes Logs:

    Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 128 bytes) in /home/XXXX/public_html/wp-includes/wp-db.php on line 1941

    This is in the error log:

    [28-Jun-2014 18:30:11 UTC] PHP Warning: Cannot modify header information - headers already sent by (output started at /home/XXXX/public_html/wp-content/themes/genesis/lib/structure/header.php:48) in /home/XXXXX/public_html/wp-content/plugins/better-wp-security/core/class-itsec-lockout.php on line 316

    This appears to be a clash between my parent Theme Genesis but may be related. I'd appreciate any thoughts.

  10. joe.toomey
    Member
    Posted 1 year ago #

    Your "Allowed memory size exhausted" error is occurring at the same location mine was (wp-db:1941), which is inside wordpress' get_results() function. get_results() is a utility function in wordpress that executes a query against the database and and returns the results in the format requested. Line 1941 is part of the code path when the caller requests that the query result be returned in an integer indexed array.

    I can see from your error message that you also have PHP configured with 256MB of RAM. Not knowing more about your situation, I can't say for certain if your issue is the same as mine or not. For example, how often do you encounter the out of memory error? If it's very infrequent and seemingly random, you could have a memory leak somewhere in your theme/plugins that slowly exhausts all memory, eventually running out and reporting this error. If, like me, you find that php memory usage is stable and low until iThemes Security tries to do a backup, then you're probably hitting the same error we've been discussing above.

    The way that I went about troubleshooting the error after the normal fixes (increasing php memory and making sure that actually worked, confirming that my memory usage was stable and low over time) was to add some debugging log statements to wp-db.php so I could see what was happening when the error occurred. Here's the code change I made to help narrow down the problem (be sure to make a backup of wp-db.php before trying this so you can easily restore your settings and in case you mess something up when editing the file:

    function get_results( $query = null, $output = OBJECT ) {
    		$this->func_call = "\$db->get_results(\"$query\", $output)";
    
    		if ( $query )
    			$this->query( $query );
    		else
    			return null;
    
    		$new_array = array();
    		if ( $output == OBJECT ) {
    			// Return an integer-keyed array of row objects
    			return $this->last_result;
    		} elseif ( $output == OBJECT_K ) {
    			// Return an array of row objects with keys from column 1
    			// (Duplicates are discarded)
    			foreach ( $this->last_result as $row ) {
    				$var_by_ref = get_object_vars( $row );
    				$key = array_shift( $var_by_ref );
    				if ( ! isset( $new_array[ $key ] ) )
    					$new_array[ $key ] = $row;
    			}
    			return $new_array;
    		} elseif ( $output == ARRAY_A || $output == ARRAY_N ) {
    			// Return an integer-keyed array of...
    			if ( $this->last_result ) {
    				$emited = false;
    				foreach( (array) $this->last_result as $row ) {
    					if ( $output == ARRAY_N ) {
    						// ...integer-keyed row arrays
    						if (!$emitted) {
    						    error_log("Current Mem: " . memory_get_usage() . ", eak mem: " . memory_get_peak_usage());
    							error_log($query);
    							$emitted = true;
    						}
    						$new_array[] = array_values( get_object_vars( $row ) );
    					} else {
    						// ...column name-keyed row arrays
    						$new_array[] = get_object_vars( $row );
    					}
    				}
    			}
    			return $new_array;
    		} elseif ( strtoupper( $output ) === OBJECT ) {
    			// Back compat for OBJECT being previously case insensitive.
    			return $this->last_result;
    		}
    		return null;
    	}

    This is 6 lines of added code; the first declares the $emitted variable to keep track of whether we've already logged for this request, and then 5 lines of the if clause to actually log.

    What this does is print out the current and peak memory consumed by php before we start reading the query results into memory, and also print out the query that was executed. The memory will give you an idea of if you have reasonable memory available before we start reading the results. If the available memory is close to your limit (within a few MB), then the problem is probably elsewhere and you don't really have the available headroom to run a reasonable size query. If, like me, you have tons of free memory before the query is run, then look to see what the query is that runs right before you run out of memory (my log entries were all in php error log, but if yours are split across and iThemes log and the php error log, correlate between the two based on timestamps.) The query that was run immediately prior to the memory error is the one that blew you up.

    In my case it was a SELECT * FROM wp_redirection_logs;. That table was blown up to a huge size b/c the Redirection plugin was misconfigured on my site to never expire log entries. From reading the iThemes security plugin's code, it's clear that the backup action does a SELECT * FROM query on every table in your DB that starts with the wp_ prefix (or other prefix if your site is configured to use a different prefix due to multi-site or something else.) Once you find what the query is, you can start to reason about the cause of your error, and perhaps prune unnecessary DB content to workaround the issue like I did.

    I'd be happy to offer suggestions if you go through these steps and report back.

    --Joe

  11. Tim
    Member
    Posted 1 year ago #

    Fantastic reply, thanks Joe!

    I'm not actually running any db backups with iThemes security as all my databases are backed up twice daily on our server and copied off site. So I don't think it can be to do with the iThemes backup function.

    The out of error message is constant and appears where on the iThemes security Log tab where it should be listing the 404 errors and failed login attempts.

    I wonder if it had just logged too many of these errors and couldn't display them using a query.

    I'll try your debug code and see where I get.

    Tim

  12. Tim
    Member
    Posted 1 year ago #

    Ah... OK well I've got 73,000+ rows in the wp__itsec_log table so there is my problem.

    I'm not that comfortable editing the database directly.

    I have a backup and am quite happy restoring it.

    Is it safe to select all the rows in the table in phpMyAdmin and delete them?

    Once I have got rid of them I can see what is causing all the 404 errors.

    Tim

  13. joe.toomey
    Member
    Posted 1 year ago #

    If you know you are able to restore from your backup (i.e. you've tested your restore process and know that it works), then you should feel okay trying to prune from that table.

    My guts says it should be okay to remove all the records (definitely do not drop the table). Like I said, if you can restore to a backup confidently, everything becomes a lot less stressful.

    We had tens of thousands of 404s as well, and they appear to have been caused by botnet traffic. You can try to figure out the cause before cleaning up the DB by just looking at your DB backup's SQL file in a text editor that can handle large files (for example, notepad++ on windows, TextWrangler on Mac, or just page through with less on linux).

  14. Tim
    Member
    Posted 1 year ago #

    Brilliant, thanks Joe.

  15. Tim
    Member
    Posted 1 year ago #

    Just to complete this thread.

    Have now deleted all rows from the iThemes Security log table in the database. Exhausted memory error has now gone and error logging is working again.

    As Joe mentions it looks like botnet traffic causing the problem.

  16. joe.toomey
    Member
    Posted 1 year ago #

    Glad to hear it, Tim.

  17. joe.toomey
    Member
    Posted 1 year ago #

    I've added these troubleshooting steps to a serverfault answer I posted last week on this issue. I'd appreciate it if you could upvote both the question and the answer if this helped.

    Thanks!
    --Joe

    http://serverfault.com/questions/609416/wordpress-site-has-repeated-php-out-of-memory-error-despite-increasing-php-memor/610805#610805

Topic Closed

This topic has been closed to new replies.

About this Plugin

About this Topic