Backup Failed - Amazon's S3 service - Empty response to HTTP request

Hi

We are getting random backup fails, never the same virtual server and not everyday and not always on the same server.

Getting

    Uploading archive to Amazon's S3 service ..
    .. upload failed! Empty response to HTTP request. Headers were :

.. completed in 9 minutes, 47 seconds

Have set "Number of times to re-try FTP or S3 uploads" to 10 as per other Issues and that reduced the number of fails but has not completely resolved the issue.

Cheers Greg

Status: 
Needs work

Comments

Are you backing up to a bucket in a region close to your Virtualmin system?

Hi Jamie

Yes the EC2 Instances we are backing up are in Sydney as the S3 Bucket is also in Sydney.

Cheers Greg

This is unfortunately a problem that a few other users have reported, and for which we haven't found a solution yet - sometimes the S3 service simply fails an upload with zero information sent back in response :-(

I am hoping that an update to use a newer S3 client library in Virtualmin which is currently underway will fix the issue though.

Thanks Jamie

We will wait for the update, luckily it seems to pick different Virtual Servers each day to fail on so we are getting on average 6 out of our 7 daily backups for each one.

Cheers Greg

Ok, the next release of Virtualmin (4.16) will completely work around this problem by using the AWS command line tool, if installed. See http://aws.amazon.com/cli/ for instructions on how to download it.

Jamie helped me out with the same issue. I had my attempts to contact AWS set by default to 3. Since upping it to 12, I have had 0 fails, I used to get multiple fails a day...

"You may be seeing an interruption due to a transient network error - one possible work around is to increase the number of times Virtualmin re-tries an upload, at System Settings -> Virtualmin Configuration -> Backup and restore -> Number of times to re-try FTP or S3 uploads."

Automatically closed -- issue fixed for 2 weeks with no activity.

Automatically closed -- issue fixed for 2 weeks with no activity.

Status:
Closed (fixed)
»
Needs work

You should add awscli installation to install.sh script to avoid problems with s3

Once installed, all works...

Ilia's picture
Submitted by Ilia on Wed, 01/08/2020 - 06:47

Did you install it using pip3 install awscli command?

Ilia's picture
Submitted by Ilia on Wed, 01/08/2020 - 06:57

Assigned: Unassigned »

Actually it works as well with yum install awscli (CentOS 7 and 8), and on Debian and Ubuntu works apt-get install awscli;

Joe, could we add this to the install.sh script as a dependency?

Yes, exactly I've used apt install awscli