Support » Plugin: W3 Total Cache » Amazon Cloudfront Invalidations

  • I have this plugin running on two sites that use Amazon Cloudfront for the CDN. On May 3, one site started sending invalidations to Cloudfront. At first there were anywhere from a few to 1000 invalidations per day. Starting May 19, there were 7000 invalidations per day. Amazon charges $0.005 per invalidation after 1000, so I ended up with a $500 bill for Cloudfront for May.
    I have disabled the CDN for now, so it stopped. As far as I can tell, W3 Total Cache was the only plugin updated around that time, and the only one that should affect CDN.
    I saw another post which said to use the generic setting instead of the Cloudfront setting, so I will give that a try. Is there anything else I can do to make sure this doesn’t happen again?

    Thanks!

Viewing 8 replies - 16 through 23 (of 23 total)
  • I just got a $200 December bill due to this. This is a big chunk for my small time side hustle and there’s no way I’m going to bill my clients for this.

    I’ve used W3TC + Cloudfront for quite some time and I’ve never seen any warnings about excessive costs due to automated invalidations.

    Seriously, seriously unhappy.

    jcconnell

    (@jcconnell)

    I just received a $250 bill overnight!! This is insane. I’m pleading with Amazon to not charge me for this – I just can’t afford it.

    Please could someone from W3TC comment and mention specifically how to avoid this in the future:

    – What actions caused the invalidation expense
    – What actions should be avoided to not incur these expenses again in the future
    – What configuration should be selected to minimize this
    – etc etc

    Plugin Support Marko Vasiljevic

    (@vmarko)

    Hello,

    W3 Total Cache can purge the CDN cache by sending invalidation requests or just telling it there are new files to be cached. The CDN will understand what to do with it. We built a new option “Only purge CDN manually” because of the problems with invalidation requests on CloudFront causing huge bills. If it’s enabled, you can click the purge CDN button in W3 Total Cache to send invalidation requests, otherwise, all changes will be detected automatically and an invalidation request will be sent immediately.

    jcconnell

    (@jcconnell)

    I see that Cloudfront supports wild card (*) invalidations – any plans to implement that?

    So activating “Only purge CDN manually” is the answer here? Once this is enabled, in what scenario would we want to actually purge the CDN manually?

    jcconnell

    (@jcconnell)

    Could you please confirm that if I enable “Only purge CDN manually” that it will not incur any additional charges?

    jcconnell

    (@jcconnell)

    The bill is now up to $450! I haven’t done a single thing to the site in between now and my first post here. How can I disable this immediately?

    Plugin Support Marko Vasiljevic

    (@vmarko)

    As said in the previous post select “Only purge CDN manually” and save all settings.

    I had a similar experience as the others described and I’ve learned why this happening. For those of you confused about the surprising AWS bills allow me to clarify, and please correct me if I’m wrong.

    W3TC is like a doorway to your AWS account specifically the Cloudfront service. You can think of the billing system for Cloudfront as a machine that copies and counts all of the files in your /wp-content/cache/object folder when you first deploy to it.

    Then it distributes a copy of those static files aka objects to its global network of datacenters. As of posting this, the free tier comes with 2,000,000 requests per month.

    https://aws.amazon.com/cloudfront/pricing/

    If you open up chrome inspector’s Network tab on your website’s homepage, you can see the total number of http requests made in the lower left corner. So each time you or someone else comes along and visits your homepage that’s how many requests they are making to your account, then their browser will hopefully cache all known files from there until they expire. As the visitor continues to navigate around your website each new http request they make such as an image will also be counted. Even though the total number of http requests is multiplied per unique visitor, the high cost probably isn’t happening because of that, its more likely to come in the form of invalidation requests.

    After the Cloudfront is setup and deployed any changes to your css, js, images or fonts (to name a few) will be flagged as “invalid” meaning a new version of that file is available for Cloudfront to redistribute.

    Here’s where the expensive bills are likely coming from.

    Every time you purge the CND cache you are telling W3TC to signal Cloudfront that each path specified is invalid and that a request should be made.

    Path. Not file.

    The free tier only comes with 1,000 invalidation paths.

    The following counts as 3 paths.
    images/apples.jpg
    images/cat.png
    images/something.jpg

    Using a wildcard it counts as 1 path.
    images/*

    Things that WILL cause invalidation requests:
    – Pressing the “purge CDN completely” button inside of W3TC.
    – Purging files manually.

    Things that MAY cause invalidation requests (if CDN pulls):
    – Creating or updating public files.
    – Updating plugins. (css, js, fonts, img)
    – Updating core.
    – htaccess cache expiration timers?

    • This reply was modified 10 months, 3 weeks ago by gjones604.
Viewing 8 replies - 16 through 23 (of 23 total)
  • The topic ‘Amazon Cloudfront Invalidations’ is closed to new replies.