Hi,
I've have an automated daily backup push to Amazon's S3 bucket up and running for a year without any issues, however in the last day it is suddenly throwing an error (there has been no configuration changes to the backup).
The backup logs show that all the files are successfully compressed into tar.gz files and my S3 bucket shows that upload started but didn't fully complete. The run time was 34:46 and the final size was 12.20GB.
The backup logs show very little information about what might have caused the problem:
"Uploading archive to Amazon's S3 service .. .. upload failed! Upload failed :"
The odd thing is I have the backups configured to "Continue with other features and servers" when an error occurs, and the format is set to one file per server, so even if it did encounter any problems it should have continued with the next file.
I tried to manually run the backup but experienced the same issue.
Is there some other log I can review that will give more information about what is causing the error?
Any light you can shed on this issue would be great.
Hey blueliquiddesigns,
I am having the same issue,
I have a server with about 120 domains and we do both full and incremental backups. some of them have started failing with the same "Uploading archive to Amazon's S3 service .. .. upload failed! Upload failed :" issue, no obvious connection between the domains, and the affected domains vary each day.
Did you work out your issue?
Yes. I increased the number of retries from 3 to 10. You can do this from System Settings -> Virtualmin Configuration -> Backup and Restore -> Number of times to re-try FTP or S3 uploads.
OK so here's the workaround I've adopted to get the backups copying reliably to S3:
I've created an S3 bucket for site backups, mounted it as an NFS volume using S3Fuse and changed the automated backup path to be local pointing at that mount point. There are plenty of notes on the web about obtaining, installing and setting up S3Fuse and creating an S3 mount point. I confirm that the backups are now running reliably.