UpdraftPlus not moving all files to remote server

  • Resolved drmikegreen

    (@drmikegreen)


    11 years ago

    I have set up UpdraftPlus to save backups to Google Drive. It is consistently leaving two or three of the zipped files on the website server, rather than transferring them to Google Drive.

    One of the files it failed to transfer today was backup_2015-06-16-2214_Church_of_Our_Savior_ba059152a552-uploads4.zip. It created backup_2015-06-16-2214_Church_of_Our_Savior_ba059152a552-uploads4.zip.tmp, but left it on the website server and did not copy it to Google Drive.

    I have tried to reduce the size of zip files (it is now down to 254 MB) and made sure there is enough free space on Google Drive. That has not helped.

    Here is the portion of the log file where this file was created:

    0395.310 (1) backup_2015-06-16-2214_Church_of_Our_Savior_ba059152a552-uploads4.zip.tmp: size is now: 15.60 Mb
0395.335 (1) Zip: backup_2015-06-16-2214_Church_of_Our_Savior_ba059152a552-uploads4.zip.tmp: 100 files added (on-disk size: 15977.3 Kb)
0395.416 (1) Adding batch to zip file (UpdraftPlus_BinZip): over 200 Mb added on this batch (211.1 Mb, 2528 files batched, 103 (171) added so far); re-opening (prior size: 15977.3 Kb)
0416.761 (1) backup_2015-06-16-2214_Church_of_Our_Savior_ba059152a552-uploads4.zip.tmp: size is now: 220.60 Mb
0416.769 (1) A useful amount of data was added after this amount of zip processing: 22.1 s (normalised: 21 s, rate: 9773.5 Kb/s)
0416.772 (1) Performance is good - but we will not increase the amount of data we batch, as we are already at the present limit (time=22.12025308609, normalised_time=20.954520968279, max_time=124, data points known=1, max_bytes=209715200)
0416.784 (1) Adding batch to zip file (UpdraftPlus_BinZip): possibly approaching split limit (38.9 Mb, 3 (174) files added so far); last ratio: 0.9709; re-opening (prior size: 225873.8 Kb)
0427.949 (1) Zip size is at/near split limit (258.1 Mb / 254 Mb) - bumping index (from: 3)
0720.325 (2) Opened log file at time: Tue, 16 Jun 2015 22:26:26 -0400 on http://church-savior.com
0720.328 (2) UpdraftPlus WordPress backup plugin (https://updraftplus.com): 1.10.3 WP: 4.2.2 PHP: 5.4.19 (Linux p3nlhg852.shr.prod.phx3.secureserver.net 2.6.32-531.1.2.lve1.2.54.el6.nfsfixes.x86_64 #1 SMP Wed Apr 2 14:06:52 MST 2014 x86_64) MySQL: 5.0.96-log Server: Apache safe_mode: 0 max_execution_time: 900 memory_limit: 256M (used: 27.7M | 28.5M) multisite: Y mcrypt: Y LANG:  ZipArchive::addFile: Y
0720.332 (2) Free space on disk containing Updraft's temporary directory: 1313971.9 Mb
0720.336 (2) Backup run: resumption=2, nonce=ba059152a552, begun at=1434507266 (720s ago), job type=backup, previous check-in=123s
0720.341 (2) Scheduling a resumption (3) after 300 seconds (1434508286) in case this run gets aborted
0720.377 (2) Checking if we have a zip executable available
0720.380 (2) Zip engine: found/will use a binary zip: /usr/bin/zip
0720.383 (2) Creation of backups of directories: had begun; will resume
0720.421 (2) Beginning creation of dump of plugins (split every: 254 Mb)
The hosting service is GoDaddy and it is shared hosting.

    The next record referring to this file is:

    1857.481 (5) backup_2015-06-16-2214_Church_of_Our_Savior_ba059152a552-uploads4.zip: uploads: Note: This file was not marked as successfully uploaded, but does not exist on the local filesystem (/home/content/57/9267657/html/wp-content/updraft/backup_2015-06-16-2214_Church_of_Our_Savior_ba059152a552-uploads4.zip)
1857.485 (5) Recording as successfully uploaded: backup_2015-06-16-2214_Church_of_Our_Savior_ba059152a552-uploads4.zip (3c1a4ef845f867c8fd0f7255769762e9)

    Then there are nearly 20 more records that mention this file, which all read something like:
    `2160.194 (6) backup_2015-06-16-2214_Church_of_Our_Savior_ba059152a552-uploads4.zip: uploads: This file has already been successfully uploaded’

    And near the end of the log:
    6028.004 (23) uploads: 1434248067: over retain limit (3); will delete this file (backup_2015-06-13-2214_Church_of_Our_Savior_b7c31727c3e7-uploads4.zip)
    Then some lines later:
    6028.034 (23) Delete file: backup_2015-06-13-2214_Church_of_Our_Savior_b7c31727c3e7-uploads4.zip, service=googledrive
    Finally:
    6034.674 (23) backup_2015-06-13-2214_Church_of_Our_Savior_b7c31727c3e7-uploads4.zip: Deletion failed: file was not found

    Thanks in advance for your help.

    https://wordpress.org/plugins/updraftplus/

Viewing 7 replies - 1 through 7 (of 7 total)
  • Plugin Author David Anderson / Team Updraft

    (@davidanderson)

    11 years ago

    Hi,

    Please can you provide a link to your complete log file for the problematic backup? (You can download it from the “Existing Backups” tab of your UpdraftPlus settings page).

    It’ll be too long to paste into the forum here, but you can download it to your computer, and share it with Dropbox, or paste it in pastebin.com, or any similar service, and post the link here.

    Best wishes,
    David

    Thread Starter drmikegreen

    (@drmikegreen)

    11 years ago

    OK. That log file is now at:
    https://www.dropbox.com/s/u493k1oznkgonzf/log.ba059152a552.txt?dl=0

    Plugin Author David Anderson / Team Updraft

    (@davidanderson)

    11 years ago

    Hi Mike,

    Thank you. The case is very interesting!

    Firstly, the good news: your uploaded backup set was fine; it was complete without the uploads4.zip.tmp file. The issue was solely a jump in the numbering of the zip files – i.e. cosmetic only. The files that were in the temporary uploads4 zip (which would be automatically deleted the next day when UD backs up) actually went into uploads5. This also caused the log message about an unexpectedly missing file, and the failed attempts to delete such files – but this was also harmless, since the backup was fully contained in the actually-existing/uploaded files.

    However, the jump in numbering does cause confusion, so I’d like that fixed.

    The cause is to do with the fact that UD has to be ready for the web hosting server to kill it off at any time. There’s no prior warning of being killed in PHP – your process just dies, and next time you run, you have to try to examine your state to see where things were…. and to be very careful to do operations in the right order, so that the state is always recoverable.

    In this particular case, it appears from my testing, in which I could not reproduce the issue no matter where I killed the process, and my reading of the log, that on your particular web hosting company, it is possible for a filesystem operation (in this case a rename) to be apparently successful (i.e. control returns to UD to continue), but for that rename to actually not go through. Very near the time of being killed, it seems that a later database (i.e. MySQL) operation *can* go through, but a filesystem operation from earlier, did not.

    I believe we’ve now found a way to detect and deal with this previously-thought-to-be-impossible situation.

    Please can you update to the developer version, following these (easy) instructions …

    https://updraftplus.com/faqs/devversion/

    … and then try a few backups (which I appreciate will mean coming back after a few hours and running a new one, given the size of your site), to see what happens? (You may still see messages about previous zips not deleted from Google – since they weren’t uploaded, they can’t now be deleted). If you see anything wrong, then please share the log files so that I can take a look.

    David

    Thread Starter drmikegreen

    (@drmikegreen)

    11 years ago

    I think that fixed the problem, David. Thanks very, very much!

    I ran the backup manually twice, then let the next scheduled ones run. I now find no files being left on the website server.

    Here are the three most recent log files for your perusal:

    1. https://dl.dropboxusercontent.com/u/7287395/log.acc1c6248978.txt
    2. https://dl.dropboxusercontent.com/u/7287395/log.acc1c6248978.txt
    3. https://dl.dropboxusercontent.com/u/7287395/log.acc1c6248978.txt

    The first of these is from my second manual backup. The second and third are from the subsequent scheduled, daily backups. I have Updraft set to split the archives every 512 MB.

    Plugin Author David Anderson / Team Updraft

    (@davidanderson)

    11 years ago

    Hi,

    Thank you. You’ve posted the same link three times – but I can see in that log the tweak I made taking effect, so it looks like it’s worked. Thanks again!

    David

    Thread Starter drmikegreen

    (@drmikegreen)

    11 years ago

    My apologies for not checking what I posted more carefully. 🙁

    Thanks again for fixing this issue. Will you be including this fix in an update anytime soon?

    Plugin Author David Anderson / Team Updraft

    (@davidanderson)

    11 years ago

    Will you be including this fix in an update anytime soon?

    Hi Mike,

    The next release should be in a couple of weeks or so.

    David

Viewing 7 replies - 1 through 7 (of 7 total)

The topic ‘UpdraftPlus not moving all files to remote server’ is closed to new replies.