Virtualmin Backup and Amazon AWS S3 Storage Access - Intial Setup

2 posts / 0 new
Last post
#1 Mon, 10/13/2014 - 14:43
rgleason

Virtualmin Backup and Amazon AWS S3 Storage Access - Intial Setup

Virtualmin Backup and Amazon AWS S3 Storage Access
==============================================

Sign up for a free 5gb S3 one year Amazon Storage account
http://aws.amazon.com/s3/

AWS Account ID: xxxx-xxxx-xxxx
Canonical User ID: (long string)
AM users sign-in link: https://xxxxxxxxxxxx.signin.aws.amazon.com/console
----------------------------
Backing up with Virtualmin and S3 Amazon
http://smashtech.net/2009/06/03/backing-up-with-virtualmin-and-s3/
You use your 12-digit account ID to reference your account programmatically and in other contexts.
You use your canonical user ID to configure Amazon S3 access control lists (ACLs).

There are Different types of access keys for the Account and Users
http://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html

Create your AWS account access keys (for Individual Users)
http://docs.aws.amazon.com/IAM/latest/UserGuide/IAMBestPractices.html
Any request you make for an AWS resource requires credentials...permission to access the resource.
Use access keys (access key ID and secret access key) to give you access to all your resources

Therefore, protect your AWS account credentials like any other secret in these ways:

If you don't already have a set of access keys for your account, don't create them unless you absolutely need them. Instead, use your account password to sign in to the AWS Management Console and create an IAM user for yourself that has administrative privileges—see the next section.

=========
AWS - IAM Security account credentials issued (You must sign up for these) :

user name 1
Access Key ID: (long string of characters assigned)
Secret Access Key: (longer string of characters assigned)

======== To configure yourserver.com Virtualmin Backup and Restore settings for S3 servers.
You need from your account at Amazon and the bucket & pathnames that you have created.
Account Id:
Access Key:
Secret Key:
Bucket and Path:
yourserver.com/backup-full-sun/%d-%m-%Y Delete 7 days
yourserver.com/backup-incre/%d-%m-%Y Delete 7 days

=========

Backing up with Virtualmin and S3 Amazon
http://smashtech.net/2009/06/03/backing-up-with-virtualmin-and-s3/

How to configure virtualmin
https://www.virtualmin.com/node/9127#comment-91465

Okay, so I figured out how to make this work with S3. Essentially, it will create directories using the strftime parms, but only one level deep.

So the example above (and how I've always done mine) of "backups/%Y-%m-%d" will not work (because / is an invalid character).
However, if I set it up as follows, it creates one backup per server, in a yyyy-mm-dd directory:
File under bucket: %Y-%m-%d [x] do strftime style substitutions
Backup format [x] one per server [x] create destination directory

Amazon S3 Compatible Alternative (for services that are Amazon-like, not Amazon)
https://www.virtualmin.com/node/22964

Using Virtualmin 3.93, you can set that by going into System Settings -> Virtualmin Configuration -> Backup and Restore, and adding your desired hostname into the "S3-compatible server hostname" field.
Will that do what you're after?

Virtualmin > Backup and Restore > Amazon S3 Buckets
Provide access to your buckets and allows you to set when to move files to Glacier (cheaper storage)
and when to delete them. Very nice.

=================
Result
Uploading archive to Amazon's S3 service ..
.. done
11 servers backed up successfully, 0 had errors. 6 Virtualmin configuration settings backed up successfully.
Backup is complete. Final size was 1.16 GB.
Deleting backups from backup-full-sun/%d-%m-%Y in Amazon S3 bucket web1.tgp-architects older than 7 days ..
.. found 12 backups, but none were old enough

In AWS Control Panel found
All Buckets /mydomainbucket/backup-full-sun/13-10-2014
In this folder find many *.tar.gz files for multiples types of files (3 per Url) for each of 6 websites.
===================

For deleting see
http://docs.aws.amazon.com/AmazonS3/latest/dev/manage-lifecycle-using-ja...

Mon, 10/13/2014 - 14:50
rgleason

Peter got me started on this after we got the Backups working properly with two backup schedules, one weekly for full backup and one daily for incremental backups.

Topic locked