All vhosts are backed up under the same S3 account but randomly one of them fails

76 posts / 0 new
Last post
Tue, 02/24/2015 - 06:07 (Reply to #51)
eddieb

wrong place for reply

Fri, 10/17/2014 - 04:30
monsieurQ

This thread isn't too old to add to I hope, but I've been experiencing irregular backup issues since July I think. Always "no buffer space available" errors after 10-15 of 20 vhosts. A 2nd box with even more vhosts backs up smooth as butter.

Uploading archive to Amazon's S3 service .. .. upload failed! HTTP connection to s3.amazonaws.com:443 for /backup1/10-12-2014/domain.com.tar.gz failed : Failed to connect to s3.amazonaws.com:443 : No buffer space available

I've tried all sorts of experimenting with the backup settings - even splitting the entire schedule into two different nights. Nothing makes a difference.

My question is simply, does a timeline exist for the rollout of Jamie's S£ updates as mentioned in the last post?

Many thanks!

Fri, 10/17/2014 - 09:14
Junon

I am getting exactly the same issue as in this thread on occasion as well. I just recently replaced one of my backup servers with S3. It made sense in terms of price and reliability. Except now I have this issue.

Wed, 10/29/2014 - 18:19
risyasin

Hi. i had the same problem with D.O. kvm ubuntu 12.04 image. Virtualmin 4.11 latest. found this thread so i have changed retry number to 10 but only solved 2 more vhosts. success was 11 of 20 always. so i kept examining the logs & i thought it's about filesize of vhost backup and if it is bigger than chunk size that problem appears more. so i have changed chunk size to 100Mb. now i don't get that error. several times had backup. all fine now. please try & confirm.

-- yasin inat

Wed, 10/29/2014 - 22:12 (Reply to #55)
RealGecko

Can you please note where chunk size is changed?

Thu, 10/30/2014 - 04:21
monsieurQ

@risyasin Many thanks for that, I'll give it a try. I had changed chunk size previously to 25mb without much benefit.

@RealGecko Virtualmin > System Settings (bottom left) > Virtualmin Configuration > Backup & Restore > S3 upload chunk size

Thu, 10/30/2014 - 05:29
risyasin

@monsieurQ: Just be sure your chunk size is larger than your vhost backup file size. ( I believe that is the problem, chunking the backup file size to upload makes problem on S3 side. i think it's S3 bc i am also developer related with S3 & l respect JamieCameron more than Amazon :) )

@RealGecko: monsieurQ was faster than me :)

-- yasin inat

Thu, 10/30/2014 - 05:44 (Reply to #58)
RealGecko

Thanks monsieurQ and risyasin, Ive increased chunk size from default 5MB to 100MB. However my backup sizes are much larger than 5MB and fails happen only from time to time. Ill monitor the way things going for the next week and report any results.

Thu, 10/30/2014 - 06:06
monsieurQ

@risyasin "l respect JamieCameron more than Amazon :)"

Well said ;)

One of my vhosts is 500mb. Is there any reason I could/should not set chunk size to say 750mb for example? Would this have a negative effect on other vhosts?

@RealGecko Thanks - please do report back any progress

Wed, 11/19/2014 - 03:41 (Reply to #60)
RealGecko

Well, nothing changed. Some vhosts still fail from time to time. And it is wierd that "small" backups fail as even as "large" backups do. Nothing to do with the size I presume.

Wed, 11/19/2014 - 10:23 (Reply to #61)
monsieurQ

Yea, I'm the same. I've noticed however that disabling the backups completely results in webin not crashing once a week as was previously the case. This is clearly some kind of resources issue - at least on my machine.

Fri, 11/07/2014 - 03:59
monsieurQ

Having previously got the "No buffer space available" error with up to 10 vhosts failing, significantly upping the chunk size has resulted in only 2-3 vhosts failing on average with the "upload failed! Empty response to HTTP request" error.

I know this particular error is what most of you have been getting anyway but this is actually progress for me!

Thu, 11/27/2014 - 09:29
aburnsni

I found this post on the aws forums

https://forums.aws.amazon.com/thread.jspa?messageID=584023

I moved my buckets from "Ireland" to "US standard" 3 days ago and since then I've not had one failure.

Thu, 11/27/2014 - 10:11
monsieurQ

Ironically I'm actually based in Ireland but interestingly I also use Hetzner in Germany to host. However one virtualmin box always backs up fine to Ireland while the other (almost identical: more vhosts) consistently has problems doing the same. I'm seriously tempted to move buckets though after reading that

Thu, 11/27/2014 - 10:12
eddieb

Don't bother. I've always been on US buckets. Same problem.

Thu, 11/27/2014 - 10:24
monsieurQ

Ha! @eddieb Thanks for responding so quickly; I was poring over the following AWS CLI command when your reply came in:

aws s3 sync s3://oldbucket s3://newbucket --source-region us-west-1 --region us-west-2

..I don't think I will bother now.

Tue, 02/24/2015 - 05:14
jatitoam

A few months after the last post, I'm getting this exact issue. Did anyone got it solved?

Thanks.

Tue, 02/24/2015 - 05:57
monsieurQ

I migrated my most problematic box to entirely new hardware thinking this was fundamentally a hardware resources issue and it has made little difference. If anything the issue has got worse because it also periodically crashes Webmin itself.

Tue, 02/24/2015 - 06:08
eddieb

Eric, does it use the s3cmd library? It recently got updated but the failures persisted.

Alternatively, this doesn't solve the problem for everyone, but enabling Google Cloud Storage backups would be nice.

Mon, 04/13/2015 - 03:37
RealGecko

Well, I switched away from AWS to Rackpace. Works like a charm. No more failure for more than two weeks :)))

Wed, 04/15/2015 - 06:57
amityweb

I added Rackspace Cloud Backup yesterday so only have 1 day of Full and Inc backups, but they all worked fine. The last Full backup with S3 only backed up one site, so its still unreliable.

So I will use Rackspace from now on.

The only thing I dont like is Rackspace splitting the files up, adding .00000000 etc at the end. This post states there is a bug restoring them, but looks like it will be fixed in the next release https://www.virtualmin.com/node/27253

Fri, 04/17/2015 - 04:14
monsieurQ

I've stuck with the Amazon S3 backups and I've noticed that since the recent upgrades which utilise Amazon's AWS the old errors occur rarely now. This is progress, however what's happening now is that Webmin/virtualmin itself crashes and occasionally MYSQL tables crash on sites that have been backed up. This makes backing up like a mini-drama seeing what carnage is generated!

Can I ask anyone who is successfully backing up to amazon to share their chunk size setting? (Backup & Restore / Cloud Storage Providers / Amazon / )

Fri, 04/17/2015 - 10:38
CapstoneWorks

Way back when this problem occurred, we moved to a 100MB chuck size and the problem completely disappeared. The only other change we made was to a core file referenced in another post – I think it increased the “retries”, but I’d have to dig that information up, and that file may have been overwritten by the latest updates anyway. Our 100MB chuck size persists and seems to work 100% of the time.

Settings are found under: Backup & Restore / Cloud Storage Providers / Amazon /

Fri, 04/17/2015 - 11:15
monsieurQ

@CapstoneWorks Many thanks - I'll try that and see if it helps.

Sat, 06/27/2015 - 19:31
devmark

i moved to 100MB chuck size , it works find but some backups are over 100MB . i have tried to set 200MB, it still failed. any ideas?

Pages

Topic locked