Use Backblaze B2 API to send backups to BackBlaze B2

I have been using BackBlaze B2 to store backups of my non virt servers and it rocks. I request virt add B2 support as a backup destination. They ahve an API, and they are much cheaper than any of your other listed cloud storage providers. If you'd like i can find the api...but i am not a developer..:)

Status: 
Fixed (pending)

Comments

btw for ubuntu operating systems you could leverage Duplicati 2.0 which is already present in the base packages of the distros IIRC.

Do they have a command-line tool as well, or just a published HTTP API?

I'll look into this - it's certainly possible to implement.

has this been implemented?

Any timetable?

Sorry, we unfortunately don't have an ETA.

I was just about to provide an example of how to use Duplicity in the meantime until we're able to add feature into Virtualmin, but reading your comments above it sounds like you're already quite familiar with that.

That would be my recommendation for now, to use Duplicity from cron until that feature is completed.

no worries. I have a workaround in place that doesn't involve going outside of virt(experience has taught me going outside of virt leads to bad things with virt). I make heavy use of the current backup feature so I found a different way. However once B2 is directly implemented it will allow me to greatly reduce my complexity in the hosting backend..:)

RJM Web Design's picture
Submitted by RJM Web Design on Thu, 08/01/2019 - 13:11 Pro Licensee

I just use rclone to copy the entire contents of the backup directory to Backblaze as a post-backup command (using the Virtualmin backup GUI). It hasn't failed yet.

TECHNICALLY i already am. however it would be nice for virt to be able to do it directly. However the system i have setup gives me my three layer backup system for my webhosting clients so it's not a huge priority for me at the moment..:)

RJM Web Design's picture
Submitted by RJM Web Design on Thu, 08/01/2019 - 14:47 Pro Licensee

I think I have seven-day retention set up on that bucket. I forget offhand. It's so cheap, it doesn't pay NOT to be redundant about it.

I’d love to see support in Virtualmin for other S3 API-compatible storage services like B2, SwiftStack, Eucalyptus, Scality, Wasabi, etc..

This has been requested in other posts, e.g.:

VirtualminPro BackBlaze B2 Cloud Storage
https://www.virtualmin.com/node/67311

Add Support for Backblaze B2 Cloud Storage
https://www.virtualmin.com/node/52481

Amazon S3 Compatible Alternative
https://www.virtualmin.com/node/22964

Virtualmin should work with S3-compatible APIs already.

To use an S3 compatible backup, you can go into Backup and Restore -> Cloud Storage Providers -> Amazon S3, and there, select the "S3-compatible server hostname", and add in the correct hostname offered by your provider.

That being said, I didn't think Backblaze was S3 compatible.

However, you can use the above with any service offering S3 compatible backups.

RJM Web Design's picture
Submitted by RJM Web Design on Sat, 02/22/2020 - 05:28 Pro Licensee

Backblaze B2 isn't S3 compatible. They have their own API, and there are many third-party software packages that can interface with it. I find rclone to be the fastest and easiest to implement if one has a passing acquaintance with the terminal, but there are others.

I run rclone as a post-backup command and it's been working flawlessly for seven months. Running it as a post-backup command also means that Webmin waits until the remote transfer is completed before sending the confirmation email and reports the total time rather than just the local backup time, which is good because I know that the entire job has completed successfully.

The big advantage to Backblaze for server backup is that uploads are inexpensive. Downloads, not so much; but they're not so expensive that you have to worry about taking out a loan if you ever need to use one. They're also reliable, which is why I can't comment on their support (I've never needed it). They're pretty ideal for server backup because you can store absurd numbers of versions quite inexpensively.

In summary, yes, I think Backblaze would be a worthy addition to the Webmin remote destination options. If my Perl didn't suck so much, I might just code it myself. But rclone is an easy-to-use, free alternative that also works well and takes about five minutes to install and configure. But B2 is not S3 compatible.

Maybe someone who actually knows what they're doing in Perl can start with rclone and write a GUI wrapper specific to Backblaze B2, and there would be your integration.

Richard

Thanks for the writeup, Richard! I’m glad to know B2 isn’t S3-compatible. I guess I just assumed that.

andreychek, I had no idea I could change the hostname under Backup and Restore -> Cloud Storage Providers! Thanks for the info.

Status: Active » Fixed (pending)
Status:
Fixed (pending)
»
Closed (fixed)

backblaze now has S# api for itself.

RJM Web Design's picture
Submitted by RJM Web Design on Fri, 05/29/2020 - 19:36 Pro Licensee

I've been using rclone to back up to B2, and because it works fine, I'll probably leave it that way until it no longer does. This is still a good thing, though. It's easier to set up an S3 connection than to set up rclone.

Starting backup of 1 domains to Amazon S3 bucket virtualmin-backups ..

Failed to list S3 buckets : The V2 signature authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256

This is what I get when I try the custom url for backblaze....

RJM Web Design's picture
Submitted by RJM Web Design on Wed, 09/02/2020 - 05:58 Pro Licensee

Status:
Active
»
Needs review

That's a bit bizarre. I believe that's the error one would get when trying to use the AWS V2 signature version to access a bucket in an endpoint that only supports V4. I have no idea why you'd get it trying to access Backblaze unless they're trying to emulate AWS region-per-region.

Richard

RJM Web Design's picture
Submitted by RJM Web Design on Wed, 09/02/2020 - 08:09 Pro Licensee

Ah, okay. That makes sense.

I don't see that option in the Virtualmin > Backup and Restore > Cloud Storage Providers > Amazon S3 dialogue. Is V4 the default?

Just curious, mind you. I'm still happily using rclone sync.

Richard

no idea..i have virt sending its backups to an offsite FreeNAS machine that then uploads to B2...:)

This is so disappointing.... wish they would add B2 to its backup sources.

If it works that's fine Jamie.

Ok - support for backups to backblaze is in progress

Awesome!! Thanks so much Jamie :)

Status:
Needs review
»
Active

Changed status to active

Status:
Active
»
Fixed (pending)

Ok, after several days of hacking this is done for inclusion in the next release.

If anyone wants to beta test it now, please let me know.

Ok - which Linux distro are you running there?

Ok, I send a module debian package to you via email

Shoot -- you have the virtualmin GPL verson with this or is it going to be a Pro option only?

Looks like you missing the Endpoint setting which BackBlaze requires.

Likely pro-only for now.

What Endpoint setting are you referring to?

Damn... I installed this on a GPL install...

When you create a bucket BB also gives you the S3 Endpoint to use EG; S3 Endpoint: s3.us-west-002.backblazeb2.com