http://wordpress.org/support/ WordPress › Support User Favorites: tigertech en Thu, 26 Nov 2009 05:46:51 +0000 http://wordpress.org/support/topic/201058#post-1163082 Tue, 04 Aug 2009 21:55:26 +0000 donncha 1163082@http://wordpress.org/support/ <p>Unfortunately it can only do so much, under a huge deluge of comments a server will fail as happened to your site. </p> http://wordpress.org/support/topic/201058#post-1162749 Tue, 04 Aug 2009 17:04:48 +0000 rondelrosario 1162749@http://wordpress.org/support/ <p>My site was bombed by comments last night and was slapped with a hosting suspension. I'm using Version 0.9.6.1.</p> <p>Sorry for digging this up. </p> http://wordpress.org/support/topic/224783#post-944485 Sun, 04 Jan 2009 11:09:14 +0000 mrmist 944485@http://wordpress.org/support/ <p>There have beeen discussions about this before and I expect there will be more later. Basically whichever way it gets set up can cause problems in certain environments. See <a href="http://trac.wordpress.org/ticket/5007">this bug trac ticket</a>. </p> http://wordpress.org/support/topic/224783#post-944411 Sun, 04 Jan 2009 07:40:15 +0000 tigertech 944411@http://wordpress.org/support/ <blockquote><p>I just create an alias for my <a href="mailto:wordpress@mydomain.com">wordpress@mydomain.com</a>. It's just easier and I get replies back to a real account.</p></blockquote> <p>Yes, that works fine. However, the WordPress instructions do not tell people to do this; if creating an e-mail address before installing WordPress is the solution (I hope it's not), they should.</p> <blockquote><p>Then get another anti-spam system.</p></blockquote> <p>As it happens, I'm not complaining about my setup. It works fine for me. I'm complaining on behalf of other people who might not want to replace their anti-spam system just so they can get this message... especially since their anti-spam systems are doing what they're supposed to (detecting forged sender addresses on a message -- in this case, wordpress@&lt;domain_name&gt;).</p> <blockquote><p>Also, how exactly do you check the validity of a sender address? Sender domain is easy to check, but the sender within the domain?</p></blockquote> <p>Many inbound anti-spam systems do this by making a "callback" to the sending server and performing "HELO / MAIL FROM &lt;&gt; / RCPT TO &lt;address&gt; / QUIT" steps, checking that "RCPT TO &lt;address&gt;" works.</p> <p>And many outbound mail systems have a list of valid "From" addresses that exist on the system, rejecting others (this is a good way of stopping compromised scripts from spewing spam).</p> <blockquote><p>If it defaults to the user ID of the web server process (such as apache2 or www-data) how is that better than <a href="mailto:wordpress@yourdomain.com">wordpress@yourdomain.com</a>?</p></blockquote> <p>It's <strong>far</strong> better. The default mail() address on any properly configured system should be a valid address that's allowed to send mail. wordpress@&lt;domain_name&gt; probably isn't.</p> <p>Again, it's simply not okay to make up fake e-mail addresses these days. It's not 1988 any more. Forged e-mail is one of the biggest problems on the Internet, and more and more systems detect and block it in all sorts of ways, both on the outgoing and incoming ends.</p> <p>Scripts that send mail should either use the default address or ask the user for a valid address. </p> http://wordpress.org/support/topic/224783#post-922683 Fri, 12 Dec 2008 17:41:41 +0000 jdembowski 922683@http://wordpress.org/support/ <p>I just create an alias for my <a href="mailto:wordpress@mydomain.com">wordpress@mydomain.com</a>. It's just easier and I get replies back to a real account.</p> <blockquote><p>anti-spam systems that check the validity of the sender address will fail (I've actually seen this happen).</p></blockquote> <p>Then get another anti-spam system. Whitelisting names is just another way to guarantee that mail does not get delivered. Also, how exactly do you check the validity of a sender address? Sender domain is easy to check, but the sender within the domain?</p> <blockquote><p>When PHP is using sendmail (which most are), it's better to not specify the sender address at all; most hosting environments are (or should be) configured with a reasonable working default.</p></blockquote> <p>That would fall under the idea of DIAW. If a mail server gets a badly formed address, it may append the domain for the sender, but you need to have something. If it defaults to the user ID of the web server process (such as apache2 or www-data) how is that better than <a href="mailto:wordpress@yourdomain.com">wordpress@yourdomain.com</a>? </p> http://wordpress.org/support/topic/224783#post-922648 Fri, 12 Dec 2008 17:03:54 +0000 tigertech 922648@http://wordpress.org/support/ <p>At the end of the WordPress installation process, it sends an e-mail message containing the password to the new administrator.</p> <p>However, WordPress sends this message from "wordpress@&lt;domain name&gt;". That address may not exist (in fact, it probably doesn't), which can cause problems: in particular, anti-spam systems that check the validity of the sender address will fail (I've actually seen this happen).</p> <p>WordPress should not use a fake, probably nonexistent sender address like this. When PHP is using sendmail (which most are), it's better to not specify the sender address at all; most hosting environments are (or should be) configured with a reasonable working default. </p> http://wordpress.org/support/topic/201058#post-870510 Mon, 06 Oct 2008 17:21:24 +0000 donncha 870510@http://wordpress.org/support/ <p>That's an impressive change in the load of that server! I think the admin page needs to reorganised so that advanced options like this are further down the page and out of sight of users who don't need this functionality. </p> http://wordpress.org/support/topic/201058#post-868671 Fri, 03 Oct 2008 19:37:53 +0000 tigertech 868671@http://wordpress.org/support/ <p>I wanted to followup with the results of testing this.</p> <p>It works extremely well. <a href="http://www.tigertech.net/temp/wp-super-cache-graphs.html">Here are some graphs showing the CPU usage and load average of the busy WordPress site that started all this</a>. (This is on a 3 GHz dual Xeon server with 4 GB of RAM, and the site receives around 100,000 - 150,000 page views a day.)</p> <p>Just so it's clear, the site was always using WP Super Cache. The problem was that the site sometimes gets hundreds of comments per hour in response to a single post, and when that happened, the supercache file was not available a large portion of the time (because it was being rebuilt). Every comment posted led to dozens of simultaneous slow non-cached runs of the full WordPress script until one finished and the cached file was recreated.</p> <p>The only solution for these load spikes was for the site owner to enable "lockdown" mode when he noticed a comment storm happening. That fixed the load spike, but as far as he was concerned, broke the functionality of his site -- new visitors couldn't see any comments that had been posted, and the comments were the main reason many were visiting.</p> <p>The new code solves this. I first used my custom patched version, and when WordPress 0.8.2 came out, used that version unmodified and enabled the experimental feature in the config file. The CPU usage and load dropped remarkably, as you can see from the graphs. There have been no more 503 errors, and the site owner says that neither he nor his visitors have noticed the occasional few-second delay before comments appear in the cached version of the files. He reports no other problems at all, and he has not needed to enable lockdown since the change.</p> <p>While not everyone needs this code, people who do currently need lockdown mode would probably benefit tremendously from using this method instead.</p> <p>I hope this is useful -- I'd be glad to answer any questions anyone has about my experience with it. Donncha, thanks for being so willing to listen and to spend time reworking the code to include this idea. </p> http://wordpress.org/support/topic/201058#post-863462 Sat, 27 Sep 2008 08:23:37 +0000 donncha 863462@http://wordpress.org/support/ <p>The needs-rebuild code is in the latest release but it's disabled by default. Check out the sample config file for the right option. </p> http://wordpress.org/support/topic/201058#post-862792 Fri, 26 Sep 2008 09:39:37 +0000 donncha 862792@http://wordpress.org/support/ <p>Tigertech - Instead of modifying that function, I modified the post_change function so it renames the 2 index.html files directly. The prune_super_cache() function doesn't need the recursive parameter when called elsewhere so it doesn't seem necessary to modify it. </p> http://wordpress.org/support/topic/201058#post-862312 Thu, 25 Sep 2008 18:59:48 +0000 tigertech 862312@http://wordpress.org/support/ <blockquote><p>tigertech - sort of applied your patch to trunk and it's working.</p></blockquote> <p>Great! Yours is much cleaner (and correct-er); thanks.</p> <p>I tried it, and it works as intended, but I did notice a bug: all the supercache files in the cache directory are getting renamed to ".needs-rebuild", instead of just the ones affected by an edit/comment.</p> <p>This is happening because it now uses prune_super_cache to delete/rename files that are associated with meta files (instead of just unlinking them). When it finds that the home page cached item needs to be pruned, it calls "prune_super_cache ('/$cachepath/supercache/$hostname/', true, true)" to prune that page -- but prune_super_cache is recursive, so it actually affects every file below that directory (i.e., everything).</p> <p>One way to fix it is to add a flag that can make prune_super_cache non-recursive in that special case, like this:</p> <p> <a href="http://www.tigertech.net/patches/wp-supercache-20080925111227.patch">http://www.tigertech.net/patches/wp-supercache-20080925111227.patch</a></p> <p>With that small patch applied, it seems to work as I'd expect. I'm testing it on the busy site I mentioned, and will report back in a few days as to whether it eliminated their need for lockdown mode during busy comment posting periods.</p> <p>Thanks again for all the effort -- without WP Super Cache, some of the sites we host simply wouldn't work, even if we threw clusters of powerful machines at them. </p> http://wordpress.org/support/topic/201058#post-861957 Thu, 25 Sep 2008 11:15:29 +0000 donncha 861957@http://wordpress.org/support/ <p>tigertech - sort of applied your patch to trunk and it's working. You had forgotten to check if the user regenerating the supercache was an anon one. Try it out here: <a href="http://svn.wp-plugins.org/wp-super-cache/trunk" rel="nofollow">http://svn.wp-plugins.org/wp-super-cache/trunk</a> </p> http://wordpress.org/support/topic/201058#post-861352 Wed, 24 Sep 2008 18:15:40 +0000 tigertech 861352@http://wordpress.org/support/ <blockquote><p>Because rename() in PHP fails on Windows servers if the destination file exists. To my mind that would have left a (very small) window of time where no file existed to serve since to be cross-platform compatible we would have to delete the .html before renaming the .tmp -&gt; .html</p></blockquote> <p>Hmmm, okay. That's true, but this seems like a lot of extra effort to go to (and a lot of complicated extra rewrite rules) just to avoid very occasionally running an extra copy if a new request arrives before the unlink/rename.</p> <p>I actually came up with a simpler patch, at <a href="http://www.tigertech.net/patches/wp-supercache-mini-lockdown.patch">http://www.tigertech.net/patches/wp-supercache-mini-lockdown.patch</a>. I want to test it on a high-volume site that normally requires lockdown mode, though, for a while, before suggesting that this approach is a useful general solution. I'll report back what happens. </p> http://wordpress.org/support/topic/201058#post-846475 Sat, 06 Sep 2008 12:20:20 +0000 Murmatron 2 846475@http://wordpress.org/support/ <p>OK, I've been busy hammering a subset of my dev-site with multiple wget sessions at a rate of about 10 requests/sec (that's much more than the PHP engine can serve, and much less than Apache can serve statically - it seemed a to be a fair balance) with very short cache expiry times whilst browsing and commenting and removing posts etc.etc. and didn't get any junk or lag so I'll post a link to <a href="http://murmatrons.armadillo.homeip.net/features/experimental-eaccelerator-wp-super-cache/2#R057">a patch</a> for anyone else brave enough to try it. </p> http://wordpress.org/support/topic/201058#post-845993 Fri, 05 Sep 2008 17:42:43 +0000 Murmatron 2 845993@http://wordpress.org/support/ <blockquote><p>why is an extra file extension necessary?</p></blockquote> <p>Because rename() in PHP fails on Windows servers if the destination file exists. To my mind that would have left a (very small) window of time where no file existed to serve since to be cross-platform compatible we would have to delete the .html before renaming the .tmp -&gt; .html</p> <p>Working properly on Windows is important to me.</p> <p>That's the only reason. </p> http://wordpress.org/support/topic/201058#post-845955 Fri, 05 Sep 2008 16:50:03 +0000 tigertech 845955@http://wordpress.org/support/ <p>Murmatron 2 wrote:</p> <blockquote><p>Page generation renames .expired to .wip (short for 'write in progress') which is served by rewrite rules to parallel requests during page generation.</p></blockquote> <p>Just to make sure I understand: why is an extra file extension necessary? Can't you just rename it back to .html and keep the RewriteRules as-is? Then you can skip step 5, too. </p> http://wordpress.org/support/topic/201058#post-845636 Fri, 05 Sep 2008 05:37:05 +0000 Murmatron 2 845636@http://wordpress.org/support/ <p>tigertech, I was thinking along similar lines but decided against it because of this:<br /> <code>&lt;!--nextpage--&gt;</code><br /> Comments usually appear at the bottom of every page of a paged post. Taking the human trigger out of that process means that you may actually increase the server load by having to automatically regenerate all the pages of a paged post when everyone is busy posting comments on just the first or the last.</p> <p>I've got a working version already that pretty much guarantees only one load of the php engine for each supercache page by doing this:</p> <ol> <li>Don't delete expired pages; rename them to .expired</li> <li>Page generation renames .expired to .wip (short for 'write in progress') which is served by rewrite rules to parallel requests during page generation.</li> <li>Write .tmp rather than .html so that partial .html isn't served (the rewrite rules prefer .html to .wip)</li> <li>Rename .tmp -&gt; .html</li> <li>Delete .wip</li> </ol> <p>Unfortunately, I'm having problems making a patch against supercache since the version I'm running is more 'different' than TortoiseMerge can cope with now. :\ </p> http://wordpress.org/support/topic/201058#post-845408 Thu, 04 Sep 2008 22:08:52 +0000 tigertech 845408@http://wordpress.org/support/ <p>While thinking about this, I had a third idea.</p> <p>The ideal solution would be to make the full WordPress code run just once after a comment is posted, but to do so as soon as possible after the comment (so that the comments aren't outdated for other visitors).</p> <p>Perhaps it could be engineered to forcibly make exactly that happen. Something like:</p> <p>Don't delete (or touch) the supercached file at all when a new comment is posted. Instead, fire off a separate "fake" standard (non-cookie, non-comment) page load using cURL or something.</p> <p>That fake page load would use a magic cookie value that mod_rewrite notices to bypass the cache, but WP Super Cache would ignore it and treat the request as cacheable, thus generating a brand new copy of the supercache file.</p> <p>The overhead is one extra fake page request, but in return, there never needs to be a moment in which the supercached file is missing, so there's no huge load spike for a few seconds after the comment is posted. Additionally, the delay before the new comment appears for other people is simply the length of a single standard page load -- much better than lockdown mode. </p> http://wordpress.org/support/topic/201058#post-844181 Wed, 03 Sep 2008 12:45:07 +0000 Murmatron 2 844181@http://wordpress.org/support/ <p>I'm 75% there, but I can't settle on a form of rewrite rules that minimize the performance hit</p> <p>Here's the sequence.<br /> phase2 wp_cache_ob_callback<br /> 1. rename .obsolete to .wip (.wip is served by rewrite) so no<br /> other anon clients will load php<br /> 2. create .tmp<br /> 3. write file<br /> 4. rename .tmp -&gt; .html<br /> 5. delete .wip</p> <p>If a file has been expired, there's will always be a file for rewrite to serve and I think the .tmp -&gt; .html sequence also addresses the possibility of partially written files in <a href="http://wordpress.org/support/topic/201085">this thread</a>.</p> <p>.obsolete files are generated in prune_super_cache and wp_cache_post_change by renaming instead of unlinking and aren't served by rewrite rules</p> <p>The sequence and logic seems sound. I've got this working on my mutant version of supercache just fine, but the most important and trickiest bit is to get Step 1. moved out of ob_callback to before ob_start so that only the first anon client (unless we're very unlucky) will load php</p> <p>Stay tuned - It might take a couple of days</p> <p>I was about to hit 'Post', but I got thinking... It may not be that bad. I can probably rename the .obsolete files on fairly cheap pre-conditions in phase1 and then undo it in ob_callback if it turns out to have been inappropriate to do it. </p> http://wordpress.org/support/topic/201058#post-844128 Wed, 03 Sep 2008 10:26:55 +0000 donncha 844128@http://wordpress.org/support/ <p>I like the second idea too, but locking is always a problem on high traffic sites or pages. If you're willing to do a patch I'd love to see it as it would be useful! </p> http://wordpress.org/support/topic/201058#post-843691 Tue, 02 Sep 2008 21:30:57 +0000 tigertech 843691@http://wordpress.org/support/ <p>While I was envisioning it do the "check and sleep" thing before it makes a database connection, I guess that would be hard to guarantee.</p> <p>More generally, I agree that the second idea does seem better, because it avoids intentionally delaying anything. Intentional delays would probably just cause something else horrible to happen (even if the database didn't fill up, the apache process table could easily fill up while waiting). </p> http://wordpress.org/support/topic/201058#post-843654 Tue, 02 Sep 2008 20:50:09 +0000 Otto42 843654@http://wordpress.org/support/ <p>The .obsolete trick is probably the better approach. Because having all those concurrent processes basically waiting is going to rapidly kill the database as it runs out of connections. </p> http://wordpress.org/support/topic/201058#post-843648 Tue, 02 Sep 2008 20:43:27 +0000 tigertech 843648@http://wordpress.org/support/ <p>Summary: I had two ideas that seem like they might significantly improve the performance of WP Super Cache under heavy comment posting load, minimizing or eliminating the need for lockdown mode. Feedback would be appreciated.</p> <p>Details: One of our customers has a fairly busy WordPress / WP Super Cache site that has an extremely large number of comment posts. It's not unusual to see posts garner 500 comments within three hours, for example (a new comment every 20 seconds), and posts with more than 1000 comments aren't unheard of.</p> <p>The site uses a great many WordPress plugins. An uncached page load can take several seconds, even on an otherwise lightly loaded (90% idle CPU) 3 GHz dual Xeon server using FastCGI.</p> <p>With this kind of comment posting, even WP Super Cache can't really keep up unless it's in lockdown mode. The customer is unhappy with that because of the delay in comment posting.</p> <p>So I analyzed this. Consider what happens when a comment is posted if lockdown mode is not enabled: As a first step, the supercached file is removed. The blog then regenerates the supercached file the next time it's viewed.</p> <p>However, if the page is getting several views per second, this causes a problem. The instant the supercached file is removed, the load on the server shoots up, because several concurrent requests for the uncached page arrive and start running. Within a couple of seconds, there might be half a dozen copies of the script running, all competing with each other for resources to display the page (and generate the cached file as a side effect). And there are more requests arriving every second, making things worse and worse. Sometimes, the resource contention can cause the first WordPress copy to take ten seconds or more to regenerate the cached page, and even after the cached page is first generated, the server is almost certain to be "pegged" for twenty seconds or more finishing up the requests that arrived.</p> <p>Then, 20 seconds later, the whole thing starts again with a new comment post. The end result is that more than half the time, there's no cached page available, and the server will pretty much be 100% busy running WordPress scripts despite the caching, leading to errors as it runs out of CPU time.</p> <p>Lockdown is the traditional solution to this. But what's interesting is that the performance problem is not caused by the *first* post-comment visitor requesting a new page and regenerating the cache. The server could handle regenerating the new cached page once every 20 seconds as a new comment arrives, no problem at all. Rather, it's caused by the fact that there might be *dozens* of new uncached requests for that page before the supercached copy is regenerated.</p> <p>If WordPress could be made to run the code to regenerate the cached page only once per comment post, this shouldn't be a problem (and almost nobody would need lockdown mode).</p> <p>I had two separate ideas about how to make this happen. First of all, it might be possible for WP Super Cache to realize that a page is in the process of being regenerated by another concurrent process, and simply wait a few seconds for the result of that page to appear in the cache directory, and send that back instead of running another full WordPress instance. This could be implemented as follows:</p> <p>When a request for a "to be cached" page starts, WP Super Cache would put a special temporary file with a funny name in the appropriate supercache directory. At the beginning of each request, WP Super Cache would check for the existence of this file. If the file is present and is less than, say, 15 seconds old, the code would go into a loop where it sleeps for, say, a tenth of a second and looks to see if the cached file is now in the directory. If so, use it. If we've slept for more than what would be 15 seconds since the temp file was created, assume that something is wrong and break out of the loop and run normally.</p> <p>A second idea is to not delete the "obsolete" supercached file when a new comment is posted. Instead, rename it to a special name -- say, append ".obsolete" to it. The .htaccess rules won't find the file under this special name, so the next request will start regenerating the supercached file. However, that request could check to see if there is a ".obsolete" file present in the directory, and whether that file is less than, say, 30 seconds old. If so, rename it back (remove the ".obsolete") so that subsequent requests that arrive for the page before our process finishes will still use the original cached file via the .htaccess rule. Again, our process will be the only one regenerating the new file, running no more than once every 30 seconds.</p> <p>This second idea isn't perfect, because it delays the appearance of the most recent comments (like lockdown mode) if two people do visit the page at once. However, the average delay is the same as the average time between requests for the page, and the maximum delay would be 30 seconds (or whatever), which is much less than lockdown mode.</p> <p>So those are my ideas. Any thoughts? I suspect I might be missing something obvious. If either of these sounds like it would work, I can come up with a patch and test it. </p>