Restoring large sites from Rackspace Cloud files fails.

5 posts / 0 new
Last post
#1 Mon, 05/13/2013 - 12:20
tom-pps

Restoring large sites from Rackspace Cloud files fails.

First, I would like to apologize that my first post is a support request but with great software I have not run into any issues prior and I have found most answers to any questions I have in search and in the documentation. So really first and foremost I would like to thank the developers and the community.

I am having an issue with restoring large sites from Rackspace cloud files, which I believe is related to a possible Bug in Virtualmin's restore feature for Rackspace. I believe that while restoring sites Virtualmin does not take into account that large files are segmented in Rackspace Cloud files.

Server A runs a daily scheduled backup to Rackspace cloud files.
Server B runs a weekly cron to restore the backups from Rackspace cloud files.
Domain 1 is ~30MB
Domain 2 is ~3GB

Server A & Server B a re kept updated regularly and are running CentOS 6.4
virtualmin-release-1.0-5.gpl.rhel.noarch
virtualmin-base-1.0-63.rh.noarch
wbm-virtualmin-init-2.4-1.noarch
wbm-virtualmin-dav-3.5-1.noarch
wbm-virtualmin-registrar-2.2-1.noarch
wbm-virtualmin-htpasswd-2.6-1.noarch
wbm-virtualmin-awstats-4.9-1.noarch
wbm-virtualmin-mailman-6.2-1.noarch
wbm-virtualmin-svn-5.0-1.noarch

Server A runs a scheduled backup daily to Rackspace Cloud Files to virtualmin_backup/%Y-%m-%d with these settings checked.
[X] Do strftime-style time substitutions on file or directory name
[X] Transfer each virtual server after it is backed up
[X] One file per server
[X] Create destination directory?
[X] Continue with other features and servers
[X] Full (all files)
Backups run as expected.
Cloud files Show these files
Domain 1
- domain1.tar.gz.info
- domain1.tar.gz.dom
- domain1.tar.gz
Domain 2
- domain2.tar.gz.info
- domain2.tar.gz.dom
- domain2.tar.gz
- domain2.tar.gz.00000000
- domain2.tar.gz.00000001
- domain2.tar.gz.00000002
- domain2.tar.gz.00000003
- domain2.tar.gz.00000004 (-> etc)

Server B runs the weekly cron which uses the following line to restore files from Rackspace:


/usr/sbin/virtualmin restore-domain --source rs://[USERNAME]:[API-KEY]@virtualmin_backup/`date -d 'last friday' +%Y-%m-%d` --reuid --all-features --skip-warnings --all-domains

, but this fails.


Checking for missing features ..
.. WARNING - The following features were enabled for one or more
domains in the backup, but do not exist on this system. Some
functions of the restored domains may not work : Mail for domain, Mailman

Starting restore..
Downloading archive from Rackspace Cloud Files ..
.. done

Extracting backup archive files ..
.. extraction of /tmp/.webmin/[NUMBERS]_restore-domain.pl/domain2.tar.gz.00000006 failed : /bin/tar: This does not look like a tar archive
/bin/tar: Skipping to next header
/bin/tar: Exiting with failure status due to previous errors

Restore failed!

According to Rackspace, by downloading domain2.tar.gz this automatically concatenates the segments. So there is no need to download and extract the .00000000 files.

- Please advise me on my findings and let me know if I should report this as a Bug.
- Is there anyone else who has experienced this and has found a work around?
- I am a developer in various languages and might be able to create a patch for this issue, but I am not familiar with the architecture of this software. Where should I be looking to create a patch for this functionality?

TLDR;
Virtualmin should not download the segments (.00000000 files) from Rackspace cloud files and try to extract and restore them. How do I fix this?

Tue, 05/14/2013 - 10:05
andreychek

Howdy,

I'm going to ask Jamie to take a look at this... if it's indeed a bug, he'll be able to work with you to get that sorted out.

-Eric

Tue, 05/14/2013 - 14:32 (Reply to #2)
JamieCameron

This is a Virtualmin bug that is triggered when restoring a domain large enough to be split into parts on rackspace cloud files.

The work-around is to restore from the domain's tar.gz file explicitly, instead of the whole directory. So in your case, you'd need to use a command like :

/usr/sbin/virtualmin restore-domain --source rs://[USERNAME]:[API-KEY]@virtualmin_backup/\`date -d 'last friday' +%Y-%m-%d\`/mydomain.com.tar.gz --reuid --all-features --skip-warnings --all-domains

''

Wed, 05/15/2013 - 09:21
tom-pps

Great Thanks for the tip, Shall I open a bug for this or will you be handling that?

EDIT
I'd also like to note this bug also affects the auto deletion of old backups. It's not as bad, because it's not a fatal error, it just reports the that deletion failed because the file was not found if the main file was deleted prior.

Wed, 05/15/2013 - 15:03
andreychek

No need to open a bug, Jamie's already handled correcting that in the next Virtualmin version.

He'll take care of the deletion issue too... thanks for reporting it!

-Eric

Topic locked