WordPress.org

Ready to get started?Download WordPress

Forums

W3 Total Cache
CDN Offloading Works Initially; Stops Working by Next Day (32 posts)

  1. nsm2
    Member
    Posted 3 years ago #

    When I first install/configure W3TC to use Amazon CloudFront as a CDN, everything works fine (well, mostly, but we'll deal with the minor issues later.) The majority of images/css/js files are offloaded to the CDN. The files are uploaded to the CDN and the URLs are being rewritten in the HTML to point to the new location.

    Sometime later (sorry, but I don't have a specific timeframe), the site reverts back to being almost entirely served locally. A few items are still offloaded, but most are not. The files are still located on the CDN, but the URLs are no longer being rewritten. They point to the local copies.

    What could be going on? Where could I look for further info? The "Unsuccessful File Transfer Queue" has a large number of entries, but they all read "Last Error: Object already exists", which doesn't suggest to me a true problem. The Delete Queue is empty. The wp-content/w3tc/log/ folder is empty.

    Thanks for your help!

  2. mjmcgowan
    Member
    Posted 3 years ago #

    Have the same issue, i enabled "Force over-writing of existing files" and exported everything again, it started to work as normal, a couple of days later when i checked it was being served locally again.

    Also looking for a solution to be able to serve slideshow images from the CDN.

  3. nsm2
    Member
    Posted 3 years ago #

    mjmcgowan: Are you using CloudFront or a different CDN?

  4. Rafal Lukawiecki
    Member
    Posted 3 years ago #

    I have the same issue with CloudFront.

  5. mjmcgowan
    Member
    Posted 3 years ago #

    yes im using cloudfront

  6. mraburn
    Member
    Posted 3 years ago #

    I installed W3 Cache 2 days ago and see it behaving the same way. If I deactivate the plugin then reactive, it starts serving the CDN content from Cloudfront.

    It seems to only be the JS and CSS minify and *Other files like from NextGen Gallery that stop being called from the CDN, all normal media like blog images are still being called from the CDN.

    I can also open a different browser on the same machine and get the CDN calls to work, then go to a blog post and the same calls are now being called locally, not from the CDN.

    Something is not right for sure.

  7. Frederick Townes
    Member
    Plugin Author

    Posted 3 years ago #

    Has anyone checked their unsuccessful transfer queue to make sure it's empty when problems arise?

  8. nsm2
    Member
    Posted 3 years ago #

    As I said in the OP, there are many entries in the Unsuccessful File Transfer Queue, but they all say "Object already exists".

  9. Rafal Lukawiecki
    Member
    Posted 3 years ago #

    I can also add, as I have also mentioned in this as-yet unanswered post, that the transfer queue fills up with plenty of "Object already exists" errors. Deleting the queue helps, but the problem comes back.

    An expensive workaround is to turn on the "Force overwriting" option. When selected, the CDN keeps on working as expected, but at the cost, for my small blog, of an additional 20,000-50,000 uploads to S3 a day, for files which have not changed at all. Amazon, of course, charges for all of those dearly.

    Could it be that somehow W3 Total Cache thinks it needs to upload files which have not been changed at all, even in their cached state?

  10. mraburn
    Member
    Posted 3 years ago #

    I had a bunch of files in my cue yesterday when checking this post and had removed them from the que... they were all File Already exists errors. They are back in the que now.

  11. ruditrg
    Member
    Posted 3 years ago #

    It seems W3 Total Cache by W3 EDGE doesn't detect the images files both when I set to automatically Import external media library attachments and when I import media library attachments (images) to CDN. Now I have to copy all the files to cdn manually. Please help me to solve the problem. Thanks

  12. nsm2
    Member
    Posted 3 years ago #

    ruditrg: Did it work at any point?

    If it worked initially but then stopped working after some time, then let us know any patterns you've seen in which files work and which fail. Also let us know what error messages you've seen in the Unsuccessful File Transfer Queue (although everybody seems to be having the same experience: all entries say "Object already exists".

    If however uploading files to the CDN via the plugin has never worked, then please start a new forum topic. This topic is specifically on the issue of functioning W3TC CDN setups that inexplicably fail after some time.

  13. ruditrg
    Member
    Posted 3 years ago #

    The problem just appeared yesterday, 1st Feb. Upload queue is empty (No Unsuccessful file transfer queue) and I got no error message, just uploading files to the CDN via the plugin not worked.

  14. mraburn
    Member
    Posted 3 years ago #

    I've been checking settings to help see if something pops up that can clarify what is happening.

    I do know in the CDN setup when I was installing W3 that I get an ERROR when trying the test upload. I figured I was stuck when I installed but went ahead and did all the manual uploads of initial media, minify etc, and it worked.

    Maybe this test failure reflects something. Can you guys check yours and see.

    This happens on Auto, SSL and non SSL in the options for connecting to Cloudfront.

  15. Rafal Lukawiecki
    Member
    Posted 3 years ago #

    In my case, there were no errors on set-up, and the Test Upload worked fine. I have even deleted all the files of the plug-in, and I have reinstalled it, from scratch, resetting all settings.

    The CDN works at first (except for a few files it always misses, which I think is a bug, which I described here, but which I cannot figure out a way to report as a bug yet).

    After an hour or a few, however, the files are no longer being served from CDN, progressively fewer and fewer get served, eventually only the favicon seems to come from the CDN. The "Upload queue" fills with errors "Object already exists". Clearing the queue seems to fix the problem temporarily. The only permanent workaround I found was to "Force overwriting" but it is costly as it generates 10k-20k extra S3 uploads of entirely unchanged files at least, and even 70k on some days.

    I am wondering if this is specific to S3/CloudFront or also an issue with Rackspace. I might move to Rackspace for a test run, has anybody who had this issue with CloudFront tried Rackspace yet?

    I am prepared to do more testing or answer other questions to help resolve this bug.

  16. nsm2
    Member
    Posted 3 years ago #

    I mostly ditto your experiences:
    * Test CloudFront Upload always works fine.
    * I removed and reinstalled the plugin from scratch. I *think* that wasn't effective.
    * What did do the trick was creating a new S3 bucket and CF distribution and repopulating it.
    * I've cleared the Upload queue, checked "Force over-writing of existing files", and manually initiated a re-upload via the "exporting media library to CDN", "Upload includes files", and "Upload theme files" buttons. That seems to have done the trick for 90% of the files. (NOTE: "Upload minify files" always shows 0 files. Is this a related problem?)

    My config: (should I post my w3-total-cache-config.php file? [other than passwords, obviously])
    Running on an EC2 instance using EBS storage.
    * General Settings:
    ** All cache types are enabled. Everything is set to APC, except the CDN is CF.
    ** Enable file locking: I've tried both ways. Currently unchecked.
    * CDN Settings:
    ** General: All six items are checked.
    ** Advanced:
    *** Enable changed files auto-upload: checked
    *** Auto upload interval: I had experimented with lower values (600), but restored to its original value of 3600.
    *** Re-transfer cycle interval: ditto. currently back to 900.
    *** Re-transfer cycle limit: 25
    *** filetypes: Appended ";*.pdf" to all three fields
    *** never touched the rest.

    Frederick and team: Please let us know what further info would be helpful in resolving this.

  17. mraburn
    Member
    Posted 3 years ago #

    Does anyone know where we are at on this one?

  18. nsm2
    Member
    Posted 3 years ago #

    Currently, I'm using Rafal's workaround of using Forced Overwriting. I want to investigate it further by stepping through the source code, but don't have time right now.

  19. Frederick Townes
    Member
    Plugin Author

    Posted 3 years ago #

    Can anyone with this issue please try the development version of this plugin?

  20. mraburn
    Member
    Posted 3 years ago #

    I am giving it a spin now, I will let you know in a bit, I will decrease the wait time for upload checks and see what results I get using this. Thanks!

  21. Rafal Lukawiecki
    Member
    Posted 3 years ago #

    I was very impressed with many aspects of the development version of W3TC, like new approach to minification. When I tried it two days ago, however, I found that CDN was not redirecting several files at all, despite having copied them (no errors) to S3. Still, it seemed like it was moving in a good direction.

  22. mraburn
    Member
    Posted 3 years ago #

    I have the Dev version installed here...
    http://www.stangnet.com/

    It seems to be working OK, I notice there are feature changes from the current version so I will assume nothing. :)

    Minify has changed... so I see where I do not specify the files I want to have minified and then the 'upload to CDN' is disabled for minify. There is the 'logged in user will not get minify', which enabled and when I view the source the CSS and some JS IS called from the CDN not just combined. IF I enable minify for 'logged in users' then I see the CSS and JS calls being made in a combined URL but it is LOCAL, not the CDN.

    All media and 'other' media seems to be being called from the CDN like normal.

    Anything else I should be looking for?

  23. mraburn
    Member
    Posted 3 years ago #

    I found the new settings for the Minify to have Auto or Manual mode of combined files.

    The AUTO mode seems to be the one that makes the Local server call.

    In Minify Manual mode it then calls the main CSS and JS files from the CDN, however I cannot now add the files that I want to have included in the minify from other JS and CSS that come with other plugins.

    I know this is a DEV version, Im just reporting what I see so that it may help.

  24. ctimmermans
    Member
    Posted 3 years ago #

    I've got the exact same issues with cloudfront + w3 total cache. It makes the plugin hardly useable, as after a day or 2 all the requests are redirected to my usual server (not cloudfront).

    Should you want additional details, let me know :).

  25. mjmcgowan
    Member
    Posted 3 years ago #

    Hi Can anyone confirm that using the dev version will solve this issues?

    Frederick, have you identified the issue ?

  26. mraburn
    Member
    Posted 3 years ago #

    Im using the DEV version and the CDN is working properly accept for the CSS and JS calls in the header.

    It looks like he is using a new method to combine all the files with one URL call, but that call stays local, not from the CDN.

    All images, media and the NextGen gallery are being pulled from the CDN.

  27. pneumotube
    Member
    Posted 3 years ago #

    I'm using W3TC for about 7-8 months already. CloudFront upload always worked just fine, I did it many times. The plugin always worked flawlessly.

    Today I needed to do a theme files export which unfortunately failed. Custom files / includes files uploading doesn't work as well (just gets stuck or I get a message "Status: request failed"), CloudFront upload testing stays on "Testing..." and never changes. Unsuccessful file transfer queue is empty.

    All static files are served from the CloudFront though, and it seems that main plugin functionality works fine.

    I definitely did not make any major changes to the WP/W3TC settings lately.

  28. sibyy
    Member
    Posted 3 years ago #

    Going back to the initial subject.

    I'm having the same issue as well.

    W3 total cache was working fine with Cloudfront since yesterday.

    However images uploaded yesterday (which have also been automatically uploaded in Cloudfront as well) are not redirected to Cloudfront but are served from the server.

    i.e. whereas all other images are served from static.domain.com the new images are served from http://www.domain.com.

    I've tried a number of things and I've verified I can access the images on from http on Cloudfront but noting has worked.

  29. ctimmermans
    Member
    Posted 3 years ago #

    When do the devs (Frederick Townes) expect to release a version that fixes this bug? Or is it low priority? As this is very important for us to know what way we are heading the coming months.

    Thanks

  30. mediadecay
    Member
    Posted 3 years ago #

    Having the same problem. Newer uploaded files fail to be served from Cloudfront.

Topic Closed

This topic has been closed to new replies.

About this Plugin

About this Topic