Incremental Backups

Seems there should be an ability to do incremental backups. I realize for Windows VMs this may be a challenge but for Linux it may not be too bad. I sent Joe a script written by a buddy Chris Wik which does this. I have 17 VMs which are spread across 4 hosts which all tie into FreeNAS via their own dedicated jumper directly to the NAS. It takes nearly 9 hours to complete the backups every night.



I suppose that maybe this could be done with rsync in theory - however it would preclude compression of the backup, so the end result might be that the backup is larger and takes longer!

Steffan's picture
Submitted by Steffan on Sat, 09/03/2016 - 23:01 Pro Licensee

It turns out to be faster and takes up less space. Did you look over that script I emailed to Joe and you a while back? I don't know exactly what he uses but its fast. That's why Chris Wik does it that way at ANU internet. I'm happy to beta test anything you want.

How would it take less space though? Because the backup isn't compressed, the size of a full backup would be larger.

Steffan's picture
Submitted by Steffan on Sun, 09/04/2016 - 00:05 Pro Licensee

Compress it.

Courtesy of Chris Wik. Perhaps it'll help.

Here's my script, sold as seen, no guarantees :-) It has been working great for us for many years with only minor tweaks here and there. It requires the perl-Linux-LVM and gcc packages, the former can be found in rpmforge.

The config file contents are as follows, I put it in /etc/backup.conf but you could change that to suit:

$::dry_run   = '';           # change to '' for live run, '-n' for dry
$::stats    = '';       # change to '--stats' for stats output
$::snap_pc  = 5;
$::sleep    = 5;
$::backup_host  = '';
@::exclude      = ('coop2_backups');
@::vgs      = ('Xen','FastXen');
Should be mostly self explanatory. Put the name of your volume group(s) in the vgs array, any LVs you want to exclude go in the exclude array. backup_host is the hostname or IP of your backup server.
You'll need a folder /mnt/backup which is where it'll mount the LVM snapshots.
#!/usr/bin/perl -w -P
#include "/etc/backup.conf"
use Linux::LVM;
# Find LVMs
foreach $vg(@vgs) {
    print "Scanning VG $vg...\n\n";
    foreach $lv(`ls /dev/mapper/$vg-*`) {
        # Get LV info
        %lv = get_lv_info($lv);
        @lv_name = split(/\//, $lv{'lv_name'});
        $lv_name = pop @lv_name;
        ###print "Name\t\t\t" . $lv_name . "\n";
        # Check if it's in the exclude list
        $match = 0;
        foreach $ex(@exclude) {
             if($lv_name =~ $ex) { $match = 1; }
        if($match == 1) { next; }
        # Find out if it has an ext filesystem
        $type   = `file -s -L $lv`;
        if($type =~ /ext/) { $fs = 'ext'; } else { $fs = 'other'; }
        ##print "Filesystem\t\t" . $fs . "\n";
        # If it's ext, continue, otherwise go to next in list
        if($fs ne 'ext') {
            ##print "Not an ext fs\n";
        } else {
            print "Name\t\t\t" . $lv_name . "\n";
        # How big is the LV?
        ##print "LV Size\t\t\t" . $lv{'size'} . $lv{'size_unit'} . "\n";
        $snap_size = ($lv{'size'} / 100 * $snap_pc) . $lv{'size_unit'};
        ##print "Snapshot size\t\t" . $snap_size . "\n";
        # Create snapshot
        $lvcreate = "/sbin/lvcreate -s -L " . $snap_size . " -n backup_snap /dev/" . $vg . "/" . $lv{'lv_name'};
        ##print "Creating LV\t\t" . $lvcreate . "\n";
        print `$lvcreate`;
        # Mount snapshot
        $mount = "mount /dev/$vg/backup_snap /mnt/backup";
        ##print "Mounting snapshot\t" . $mount . "\n";
        print `$mount`;
        # Check for rsync.exclude file
        if(-r '/mnt/backup/etc/rsync.exclude') { $rsync_exclude = '/mnt/backup/etc/rsync.exclude'; }
        else { $rsync_exclude = ''; }
        # Perform backup
        $rsync_args = "$dry_run $stats -aH --numeric-ids --delete --ignore-errors --exclude-from=".$rsync_exclude." --password-file=/etc/rsync.scrt";
        $rsync = "rsync $rsync_args /mnt/backup rsync\@".$backup_host."::backups/".$lv_name;
        ##print "Perform backup\t\t" . $rsync . "\n";
        print `$rsync`;
        # Unmount snapshot
        $umount = "umount /mnt/backup";
        ##print "Unmounting snapshot\t" . $umount . "\n";
        print `$umount`;
        # Remove snapshot
        $lvremove = "/sbin/lvremove -f /dev/$vg/backup_snap";
        ##print "Removing snapshot\t" . $lvremove . "\n";
        print `$lvremove`;
        # Remove leftover symlinks in /dev/mapper (required on CentOS 6)
        print `rm -f /dev/mapper/*-real /dev/mapper/*-cow`;
        # sleep before next backup
        ##print "Sleeping $sleep secs...\n";
        sleep $sleep;

This assumes you are running an rsync server, if you want to use ssh for transport change $rsync_args accordingly. If you're using a public network for transferring the backups, use SSH!

Ah, so you're doing the rsync at the filesystem level rather than of the VM disk image?

That would save some space because the backup no longer has to include empty blocks, but for a full filesystem I'd expect that it would be larger than a compressed disk image.

Steffan's picture
Submitted by Steffan on Mon, 09/05/2016 - 18:26 Pro Licensee

Chris has been using it and it seems to work well. This could afford a good revert to point in time feature rather than the whole image from the nightly backup.

Steffan's picture
Submitted by Steffan on Mon, 09/26/2016 - 15:24 Pro Licensee

Jamie, has there been any thought given to how to speed this up? It's taking over 9 hours to do these backups. Any solutions you suggest?

I have some ideas, but haven't built anything yet.

Another option if you just want a way to do fast restores on the same system is to use snapshots, which are fully supported in Cloudmin already and are instantaneous.