CPU Load Spiking During Backups

Our server CPU load is spiking excessively during nightly backups, going over 8 at times. This has led to Apache and/or MySQL lockups and needing to restart those services. We have a Quad Core, 8GB, 2x1TB server with only about 30 accounts and 50GB of backup files, so I am confused as to why we are having this problem. Anything we can check or do we need to alter our backups? We are running daily backups of all files and services.

Status: 
Active

Comments

Yeah, running backups for all your Virtual Servers can be a pretty intensive operation. The act of tarring and compressing 50GB of files, and dumping the contents of your databases, takes a lot of resources.

One thing that may help a bit is to change the "nice" level the backup process runs at.

To do that, log into Virtualmin, and go into Webmin -> Webmin -> Advanced Configuration, and look at "CPU priority for scheduled jobs". You can try bumping that up to something like a 20, which means it runs with a low priority, which should reduce your system load.

You may also want to make sure no other large cron jobs are running at the same time as your backups.

Also on that same "Advanced Options" page you can adjust the "IO priority for scheduled jobs" , which will reduce the disk IO impact of large backups on other processes.

Joe's picture
Submitted by Joe on Tue, 09/28/2010 - 22:41 Pro Licensee

What Jamie didn't mention is that we've had some experience with the IO priority being a serious issue, despite our systems being quite beefy (not quite as beefy as yours, but still sizable). Once the disk IO contention was straightened out, our backups became manageable...but before that we were seeing the systems almost completely freeze. Disks just aren't keeping up with the speed of all the rest of hardware these days.

This is a great tip, should it be added to a FAQ somewhere?