Supercache can be ten times faster by using sane rewrite syntax

The difference in the speed of the .htaccess is huge* – somewhere between 10 and 100 times. That matters when it hits the cache, meaning it doesn’t run any PHP. On the other hand. when it doesn’t hit the cache and it runs thousands of lines of PHP, the rewrite isn’t significant compared to the PHP.

I haven’t benchmarked it, but knowing the C code that it runs, there’s a huge difference in how many lines of code are run, and how many times key lines are repeated.

I’ll be actually testing a new installation, with the skip rules, in the next 2-36 hours so I can confirm there are no typos. Shortly after posting above, I did realize I had one error. This:

!^.*[^/]$

Should become:
/*$

I was wrong when I said:
/$

The difference being the case where it’s empty, there’s no slash or any other characters. (Just mydomain.com)

* The “huge” speed difference mostly matters only if you care about speed / load, of course. It’s not nearly the slowest thing in WordPress, by a long shot, but people use WP Supercache because speed / load matters to them. In an application like MS Word, other slowness would dwarf this difference.

I’ll try to remember to follow up after this new site is tested. I’ve worked on the WPSC .htaccess before, but didn’t save it.

blau

(@blau)

Very interesting because more speed means also lower server load. I’m using php cache right now mainly because I can send 304s.

(@jondaley)

blau: note that the documentation/comment for the 304 responses isn’t really right. Your site will send 304s (provided your server is configured correctly) when using mod_rewrite, because it is a static file, so apache will send a 304 for it just like any other static file on your site. I was just wondering about that supercache warning, and was thinking it should be rewritten better.

(@jondaley)

raymor: It’d be great to see your skip rules. I’ve looked through the supercache rules before, and wondered about the performance implications.

I’m not convinced your analysis of /*$ is correct though. Doesn’t that match anything, whether it ends with a / or not? I’m thinking the rule is trying to say match anything that ends with a slash, OR match ^$ I have no idea what scenario the double-slash rule is trying to match.

(@biranit)

While on the matter, I was wondering about a few things and would like to make some constructive suggestions:

1) I think whether or not there’s support for HTTPS should be set up in the plugin’s settings page. If the site has no support for HTTPS – then there should be no reason to include the rewrite rules for HTTPS support.

2) Since I selected to treat known users as anonymous, then there is no point in the rewrite condition that checks the %{HTTP:Cookie}

3) Since I selected mobile support, then there is no reason why there shouldn’t be rewrite rules for those supported browsers, such as

RewriteRule ^(.*) “/wp-content/cache/supercache/%{SERVER_NAME}/$1/index-mobile.html” [L]

RewriteRule ^(.*) “/wp-content/cache/supercache/%{SERVER_NAME}/$1/index-mobile.html.gz” [L]

The situation right now is that there are a) too many conditions the htaccess must go through; and b) too many rejection cases then get treated by (the MUCH slower) PHP, instead of serving him the html immediately.

Given the rise in mobile traffic (30% of our site visits are from smartphones nowadays) – the latter issue (a lack of rewrite rules for mobile) is a very painful issue for me.

I wish I knew rewrite well enough to write an alternative, but I don’t… So I’m here 🙂

Thanks,

Bira

(@biranit)

OK, so following my previous post, I decided to give it a try and test an .htaccess version on my test server.

1) I incorporated raymor’s suggestions
2) I removed the https support
3) I removed the cookie condition test
4) I added rule for the mobile cached page

This is what I ended up with:

# BEGIN WPSuperCache
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
#If you serve pages from behind a proxy you may want to change 'RewriteCond %{HTTPS} on' to something more sensible
AddDefaultCharset UTF-8

RewriteCond %{REQUEST_URI} /*$
RewriteCond %{REQUEST_URI} !//
RewriteCond %{REQUEST_METHOD} !POST
RewriteCond %{QUERY_STRING} !=
RewriteCond %{HTTP:X-Wap-Profile} !^[a-z0-9\"]+ [NC]
RewriteCond %{HTTP:Profile} !^[a-z0-9\"]+ [NC]
RewriteCond %{HTTP_USER_AGENT} !^.*(2.0\ MMP|240x320|400X240|AvantGo|BlackBerry|Blazer|Cellphone|Danger|DoCoMo|Elaine/3.0|EudoraWeb|Googlebot-Mobile|hiptop|IEMobile|KYOCERA/WX310K|LG/U990|MIDP-2.|MMEF20|MOT-V|NetFront|Newt|Nintendo\ Wii|Nitro|Nokia|Opera\ Mini|Palm|PlayStation\ Portable|portalmmm|Proxinet|ProxiNet|SHARP-TQ-GX10|SHG-i900|Small|SonyEricsson|Symbian\ OS|SymbianOS|TS21i-10|UP.Browser|UP.Link|webOS|Windows\ CE|WinWAP|YahooSeeker/M1A1-R2D2|iPhone|iPod|Android|BlackBerry9530|LG-TU915\ Obigo|LGE\ VX|webOS|Nokia5800).* [NC]
RewriteCond %{HTTP_user_agent} !^(w3c\ |w3c-|acs-|alav|alca|amoi|audi|avan|benq|bird|blac|blaz|brew|cell|cldc|cmd-|dang|doco|eric|hipt|htc_|inno|ipaq|ipod|jigs|kddi|keji|leno|lg-c|lg-d|lg-g|lge-|lg/u|maui|maxo|midp|mits|mmef|mobi|mot-|moto|mwbp|nec-|newt|noki|palm|pana|pant|phil|play|port|prox|qwap|sage|sams|sany|sch-|sec-|send|seri|sgh-|shar|sie-|siem|smal|smar|sony|sph-|symb|t-mo|teli|tim-|tosh|tsm-|upg1|upsi|vk-v|voda|wap-|wapa|wapi|wapp|wapr|webc|winw|winw|xda\ |xda-).* [NC]
RewriteCond %{HTTP:Accept-Encoding} gzip
RewriteCond %{HTTPS} !on
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/supercache/%{SERVER_NAME}/$1/index.html.gz -f
RewriteRule ^(.*) "/wp-content/cache/supercache/%{SERVER_NAME}/$1/index.html.gz" [L]

RewriteCond %{REQUEST_URI} /*$
RewriteCond %{REQUEST_URI} !//
RewriteCond %{REQUEST_METHOD} !POST
RewriteCond %{QUERY_STRING} !=
RewriteCond %{HTTP:X-Wap-Profile} !^[a-z0-9\"]+ [NC]
RewriteCond %{HTTP:Profile} !^[a-z0-9\"]+ [NC]
RewriteCond %{HTTP_USER_AGENT} !^.*(2.0\ MMP|240x320|400X240|AvantGo|BlackBerry|Blazer|Cellphone|Danger|DoCoMo|Elaine/3.0|EudoraWeb|Googlebot-Mobile|hiptop|IEMobile|KYOCERA/WX310K|LG/U990|MIDP-2.|MMEF20|MOT-V|NetFront|Newt|Nintendo\ Wii|Nitro|Nokia|Opera\ Mini|Palm|PlayStation\ Portable|portalmmm|Proxinet|ProxiNet|SHARP-TQ-GX10|SHG-i900|Small|SonyEricsson|Symbian\ OS|SymbianOS|TS21i-10|UP.Browser|UP.Link|webOS|Windows\ CE|WinWAP|YahooSeeker/M1A1-R2D2|iPhone|iPod|Android|BlackBerry9530|LG-TU915\ Obigo|LGE\ VX|webOS|Nokia5800).* [NC]
RewriteCond %{HTTP_user_agent} !^(w3c\ |w3c-|acs-|alav|alca|amoi|audi|avan|benq|bird|blac|blaz|brew|cell|cldc|cmd-|dang|doco|eric|hipt|htc_|inno|ipaq|ipod|jigs|kddi|keji|leno|lg-c|lg-d|lg-g|lge-|lg/u|maui|maxo|midp|mits|mmef|mobi|mot-|moto|mwbp|nec-|newt|noki|palm|pana|pant|phil|play|port|prox|qwap|sage|sams|sany|sch-|sec-|send|seri|sgh-|shar|sie-|siem|smal|smar|sony|sph-|symb|t-mo|teli|tim-|tosh|tsm-|upg1|upsi|vk-v|voda|wap-|wapa|wapi|wapp|wapr|webc|winw|winw|xda\ |xda-).* [NC]
RewriteCond %{HTTPS} !on
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/supercache/%{SERVER_NAME}/$1/index.html -f
RewriteRule ^(.*) "/wp-content/cache/supercache/%{SERVER_NAME}/$1/index.html" [L]

RewriteCond %{REQUEST_URI} /*$
RewriteCond %{REQUEST_URI} !//
RewriteCond %{REQUEST_METHOD} !POST
RewriteCond %{QUERY_STRING} !=
RewriteCond %{HTTP:Profile} !^[a-z0-9\"]+ [NC]
RewriteCond %{HTTP_USER_AGENT} ^.*(2.0\ MMP|240x320|400X240|AvantGo|BlackBerry|Blazer|Cellphone|Danger|DoCoMo|Elaine/3.0|EudoraWeb|Googlebot-Mobile|hiptop|IEMobile|KYOCERA/WX310K|LG/U990|MIDP-2.|MMEF20|MOT-V|NetFront|Newt|Nintendo\ Wii|Nitro|Nokia|Opera\ Mini|Palm|PlayStation\ Portable|portalmmm|Proxinet|ProxiNet|SHARP-TQ-GX10|SHG-i900|Small|SonyEricsson|Symbian\ OS|SymbianOS|TS21i-10|UP.Browser|UP.Link|webOS|Windows\ CE|WinWAP|YahooSeeker/M1A1-R2D2|iPhone|iPod|Android|BlackBerry9530|LG-TU915\ Obigo|LGE\ VX|webOS|Nokia5800).* [NC]
RewriteCond %{HTTP:Accept-Encoding} gzip
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/supercache/%{SERVER_NAME}/$1/index-mobile.html.gz -f
RewriteRule ^(.*) "/wp-content/cache/supercache/%{SERVER_NAME}/$1/index-mobile.html.gz" [L]

RewriteCond %{REQUEST_URI} /*$
RewriteCond %{REQUEST_URI} !//
RewriteCond %{REQUEST_METHOD} !POST
RewriteCond %{QUERY_STRING} !=
RewriteCond %{HTTP:Profile} !^[a-z0-9\"]+ [NC]
RewriteCond %{HTTP_USER_AGENT} ^.*(2.0\ MMP|240x320|400X240|AvantGo|BlackBerry|Blazer|Cellphone|Danger|DoCoMo|Elaine/3.0|EudoraWeb|Googlebot-Mobile|hiptop|IEMobile|KYOCERA/WX310K|LG/U990|MIDP-2.|MMEF20|MOT-V|NetFront|Newt|Nintendo\ Wii|Nitro|Nokia|Opera\ Mini|Palm|PlayStation\ Portable|portalmmm|Proxinet|ProxiNet|SHARP-TQ-GX10|SHG-i900|Small|SonyEricsson|Symbian\ OS|SymbianOS|TS21i-10|UP.Browser|UP.Link|webOS|Windows\ CE|WinWAP|YahooSeeker/M1A1-R2D2|iPhone|iPod|Android|BlackBerry9530|LG-TU915\ Obigo|LGE\ VX|webOS|Nokia5800).* [NC]
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/supercache/%{SERVER_NAME}/$1/index-mobile.html -f
RewriteRule ^(.*) "/wp-content/cache/supercache/%{SERVER_NAME}/$1/index-mobile.html" [L]
</IfModule>

# END WPSuperCache

I turned on debug for WPSC and also tested response time with the default WPSC .htaccess and the revised one I did. The results are a HUGE increase in speed – especially for mobile and known users (which we treat as anonymous).

I plan to use this on my live server, and would love to see something even better incorporated in the next WPSC version – since I have little clue what I’m doing and so may have missed some stuff that could be added or removed.

Many thanks,

Bira

p.s.

Donncha, I think you should also add “Windows Phone” to the list of mobile devices 🙂

(@jondaley)

I don’t have anything special for mobile yet, so I also removed a bunch of lines:

# BEGIN WPSuperCache
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
AddDefaultCharset UTF-8

RewriteCond %{REQUEST_URI} !^.*[^/]$
RewriteCond %{REQUEST_URI} !^.*//.*$
RewriteCond %{REQUEST_METHOD} !POST
RewriteCond %{QUERY_STRING} !=
RewriteCond %{HTTP:Accept-Encoding} gzip
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/supercache/%{SERVER_NAME}/$1/index.html.gz -f
RewriteRule ^(.*) “/wp-content/cache/supercache/%{SERVER_NAME}/$1/index.html.gz” [L]

RewriteCond %{REQUEST_URI} !^.*[^/]$
RewriteCond %{REQUEST_URI} !^.*//.*$
RewriteCond %{REQUEST_METHOD} !POST
RewriteCond %{QUERY_STRING} !=
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/supercache/%{SERVER_NAME}/$1/index.html -f
RewriteRule ^(.*) “/wp-content/cache/supercache/%{SERVER_NAME}/$1/index.html” [L]
</IfModule>
# END WPSuperCache

(@biranit)

By the way, whay is HTTP:Profile? I couldn’t find any documentation on it.

Also, why is it necessary to have two HTTP_USER_AGENT strings for mobile? The second, with the mobile prefixes, is quite redundant and a little out of date. It’s kind of a nightmare maintaining it.

(@biranit)

Just as a follow-up:

This bit, which I took from @raymor, isn’t working well:

RewriteCond %{REQUEST_URI} /*$
RewriteCond %{REQUEST_URI} !//
RewriteCond %{QUERY_STRING} !=

If you have permalinks for posts using post_name, then links with /?p= will not redirect to /post-name/ and you’ll effectively get two cache versions of the same post.

I reverted back these three RewriteCond to the way they were originally in the WPSC plugin, so right now that part of my .htaccess is:

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
AddDefaultCharset UTF-8

RewriteCond %{REQUEST_URI} !^.*[^/]$
RewriteCond %{REQUEST_URI} !^.*//.*$
RewriteCond %{REQUEST_METHOD} !POST
RewriteCond %{QUERY_STRING} !.*=.*
RewriteCond %{HTTP:X-Wap-Profile} !^[a-z0-9\"]+ [NC]
RewriteCond %{HTTP:Profile} !^[a-z0-9\"]+ [NC]
RewriteCond %{HTTP_USER_AGENT} !^.*(2.0\ MMP|240x320|400X240|AvantGo|BlackBerry|Blazer|Cellphone|Danger|DoCoMo|Elaine/3.0|EudoraWeb|Googlebot-Mobile|hiptop|IEMobile|KYOCERA/WX310K|LG/U990|MIDP-2.|MMEF20|MOT-V|NetFront|Newt|Nintendo\ Wii|Nitro|Nokia|Opera\ Mini|Palm|PlayStation\ Portable|portalmmm|Proxinet|ProxiNet|SHARP-TQ-GX10|SHG-i900|Small|SonyEricsson|Symbian\ OS|SymbianOS|TS21i-10|UP.Browser|UP.Link|webOS|Windows\ CE|Windows\ Phone|WinWAP|YahooSeeker/M1A1-R2D2|iPhone|iPod|Android|BlackBerry9530|LG-TU915\ Obigo|LGE\ VX|webOS|Nokia5800).* [NC]
RewriteCond %{HTTP_user_agent} !^(w3c\ |w3c-|acs-|alav|alca|amoi|audi|avan|benq|bird|blac|blaz|brew|cell|cldc|cmd-|dang|doco|eric|hipt|htc_|inno|ipaq|ipod|jigs|kddi|keji|leno|lg-c|lg-d|lg-g|lge-|lg/u|maui|maxo|midp|mits|mmef|mobi|mot-|moto|mwbp|nec-|newt|noki|palm|pana|pant|phil|play|port|prox|qwap|sage|sams|sany|sch-|sec-|send|seri|sgh-|shar|sie-|siem|smal|smar|sony|sph-|symb|t-mo|teli|tim-|tosh|tsm-|upg1|upsi|vk-v|voda|wap-|wapa|wapi|wapp|wapr|webc|winw|winw|xda\ |xda-).* [NC]
RewriteCond %{HTTP:Accept-Encoding} gzip
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/supercache/%{SERVER_NAME}/$1/index.html.gz -f
RewriteRule ^(.*) "/wp-content/cache/supercache/%{SERVER_NAME}/$1/index.html.gz" [L]

RewriteCond %{REQUEST_URI} !^.*[^/]$
RewriteCond %{REQUEST_URI} !^.*//.*$
RewriteCond %{REQUEST_METHOD} !POST
RewriteCond %{QUERY_STRING} !.*=.*
RewriteCond %{HTTP:X-Wap-Profile} !^[a-z0-9\"]+ [NC]
RewriteCond %{HTTP:Profile} !^[a-z0-9\"]+ [NC]
RewriteCond %{HTTP_USER_AGENT} !^.*(2.0\ MMP|240x320|400X240|AvantGo|BlackBerry|Blazer|Cellphone|Danger|DoCoMo|Elaine/3.0|EudoraWeb|Googlebot-Mobile|hiptop|IEMobile|KYOCERA/WX310K|LG/U990|MIDP-2.|MMEF20|MOT-V|NetFront|Newt|Nintendo\ Wii|Nitro|Nokia|Opera\ Mini|Palm|PlayStation\ Portable|portalmmm|Proxinet|ProxiNet|SHARP-TQ-GX10|SHG-i900|Small|SonyEricsson|Symbian\ OS|SymbianOS|TS21i-10|UP.Browser|UP.Link|webOS|Windows\ CE|Windows\ Phone|WinWAP|YahooSeeker/M1A1-R2D2|iPhone|iPod|Android|BlackBerry9530|LG-TU915\ Obigo|LGE\ VX|webOS|Nokia5800).* [NC]
RewriteCond %{HTTP_user_agent} !^(w3c\ |w3c-|acs-|alav|alca|amoi|audi|avan|benq|bird|blac|blaz|brew|cell|cldc|cmd-|dang|doco|eric|hipt|htc_|inno|ipaq|ipod|jigs|kddi|keji|leno|lg-c|lg-d|lg-g|lge-|lg/u|maui|maxo|midp|mits|mmef|mobi|mot-|moto|mwbp|nec-|newt|noki|palm|pana|pant|phil|play|port|prox|qwap|sage|sams|sany|sch-|sec-|send|seri|sgh-|shar|sie-|siem|smal|smar|sony|sph-|symb|t-mo|teli|tim-|tosh|tsm-|upg1|upsi|vk-v|voda|wap-|wapa|wapi|wapp|wapr|webc|winw|winw|xda\ |xda-).* [NC]
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/supercache/%{SERVER_NAME}/$1/index.html -f
RewriteRule ^(.*) "/wp-content/cache/supercache/%{SERVER_NAME}/$1/index.html" [L]

RewriteCond %{REQUEST_URI} !^.*[^/]$
RewriteCond %{REQUEST_URI} !^.*//.*$
RewriteCond %{REQUEST_METHOD} !POST
RewriteCond %{QUERY_STRING} !.*=.*
RewriteCond %{HTTP:Profile} !^[a-z0-9\"]+ [NC]
RewriteCond %{HTTP_USER_AGENT} ^.*(2.0\ MMP|240x320|400X240|AvantGo|BlackBerry|Blazer|Cellphone|Danger|DoCoMo|Elaine/3.0|EudoraWeb|Googlebot-Mobile|hiptop|IEMobile|KYOCERA/WX310K|LG/U990|MIDP-2.|MMEF20|MOT-V|NetFront|Newt|Nintendo\ Wii|Nitro|Nokia|Opera\ Mini|Palm|PlayStation\ Portable|portalmmm|Proxinet|ProxiNet|SHARP-TQ-GX10|SHG-i900|Small|SonyEricsson|Symbian\ OS|SymbianOS|TS21i-10|UP.Browser|UP.Link|webOS|Windows\ CE|Windows\ Phone|WinWAP|YahooSeeker/M1A1-R2D2|iPhone|iPod|Android|BlackBerry9530|LG-TU915\ Obigo|LGE\ VX|webOS|Nokia5800).* [NC]
RewriteCond %{HTTP:Accept-Encoding} gzip
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/supercache/%{SERVER_NAME}/$1/index-mobile.html.gz -f
RewriteRule ^(.*) "/wp-content/cache/supercache/%{SERVER_NAME}/$1/index-mobile.html.gz" [L]

RewriteCond %{REQUEST_URI} !^.*[^/]$
RewriteCond %{REQUEST_URI} !^.*//.*$
RewriteCond %{REQUEST_METHOD} !POST
RewriteCond %{QUERY_STRING} !.*=.*
RewriteCond %{HTTP:Profile} !^[a-z0-9\"]+ [NC]
RewriteCond %{HTTP_USER_AGENT} ^.*(2.0\ MMP|240x320|400X240|AvantGo|BlackBerry|Blazer|Cellphone|Danger|DoCoMo|Elaine/3.0|EudoraWeb|Googlebot-Mobile|hiptop|IEMobile|KYOCERA/WX310K|LG/U990|MIDP-2.|MMEF20|MOT-V|NetFront|Newt|Nintendo\ Wii|Nitro|Nokia|Opera\ Mini|Palm|PlayStation\ Portable|portalmmm|Proxinet|ProxiNet|SHARP-TQ-GX10|SHG-i900|Small|SonyEricsson|Symbian\ OS|SymbianOS|TS21i-10|UP.Browser|UP.Link|webOS|Windows\ CE|Windows\ Phone|WinWAP|YahooSeeker/M1A1-R2D2|iPhone|iPod|Android|BlackBerry9530|LG-TU915\ Obigo|LGE\ VX|webOS|Nokia5800).* [NC]
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/supercache/%{SERVER_NAME}/$1/index-mobile.html -f
RewriteRule ^(.*) "/wp-content/cache/supercache/%{SERVER_NAME}/$1/index-mobile.html" [L]

I have to say, this has markedly improved the speed of our mobile version – and the hits ratio on the cache is much higher.

Thanks,

Bira

(@jondaley)

I’m not sure what “If you have permalinks for posts using post_name, then links with /?p= will not redirect to /post-name/ and you’ll effectively get two cache versions of the same post.” means.

Why wouldn’t they redirect? A URL with a GET parameter is not going to be cached at all, so I don’t see how you would end up with two cached pages?

Thread Starter

raymor

(@raymor)

I’m not convinced your analysis of /*$ is correct though. Doesn’t that match anything, whether it ends with a / or not? I’m thinking the rule is trying to say match anything that ends with a slash, OR match ^$ I have no idea what scenario the double-slash rule is trying to match.

I think you’re right, I applied the not operator (!) in the wrong order. It says “don’t match it if it ends with something other than a slash”, doesn’t it?

The old pattern was:
! ^.*[^/]$

^.* = Start with any characters, or none. That’s meaningless, so we’ll remove it and we have:
![^/]$

! means “don’t match”. [^] means “any character other than a slash”. $ means “at the end”. So we have:

Don't match any character other than a slash, at the end

Getting rid of the negative:
Match if it's a slash at the end, or nothing at all at the end.

So I now think the best we can do is to just get rid of the meaningless ^.* and use:
![^/]$

Does that sound right?

(@biranit)

@jondaley there is an option in WPSC’s advanced settings: “Don’t cache pages with GET parameters.”

I have that option UNchecked, because some of our pages actually use GET parameters.

So yes, if the htaccess rewrite conditions are broken, a /?p= won’t redirect — and in effect will create a second cache version of the same post. Probably the GET cache is done in PHP or legacy, but nevertheless, it is a second cache served for the same post.

(And never mind that I just don’t want two different URLs shared around, etc)

(@biranit)