Backup and HDD consumption problem

6 posts / 0 new
Last post
#1 Thu, 06/11/2015 - 06:59
vld

Backup and HDD consumption problem

Hello,

I have a server where home folder (where all the sites are) is about 50GB in size, /var/log is around 25GB (I have to keep a lot of logs) and /backup folder is also around 25GB.

Server currently has around 80GB of free space on the disk, backup type is set to incremental and it is kept for 3 days.

My problem is that when this backup is in progress, home folder begins to grow and eventually becomes twice as large (100+ GB), as the backup process is coming to an end. When the backup is finished /home folder returns to his original size of 50GB.

I'm guessing that backup script first does full backup of ALL sites, than compares it to existing backup and keeps only the diff, but this spike in HDD usage is becoming a big problem. Is there any way to change the way this is done, perhaps to do this process one site at a time instead of everything at once?

During the last backup HDD usage was 94% and it's a matter of days when there will be no more disk space to finish the process.

This is becoming a bigger and bigger problem each day and I could really use some help. :)

Thanks!

Thu, 06/11/2015 - 07:35
Diabolico
Diabolico's picture

Can i ask why you need 25GB of logs?

- I often come to the conclusion that my brain has too many tabs open. -
Failing at desktop publishing & graphic design since 1994.

Thu, 06/11/2015 - 08:04
vld

Clients requested it.

Thu, 06/11/2015 - 10:25
andreychek

Howdy,

The incremental backups use a feature in tar that only adds newly created files to it. It's possible though that during the backup process, logs are being copied to the homedir that are causing it to grow large -- those logs would be deleted after the backup is finished.

Do you need to backup the logs though?

You could always enable the backup option to not backup logs, which may help with what you're seeing.

On the backup screen, in the Features and Settings section, you could always uncheck the "Include log files" option.

Another option that could potentially help, is to set the " Transfer each virtual server after it is backed up" option under "Additional destination options" in the backup screen. That causes it to clear out temp data used during the backups quicker.

-Eric

Thu, 06/11/2015 - 11:41
vld

Thanks for your reply Eric, option "Transfer each virtual server after it is backed up" is already checked.

I suppose I don't need to backup the logs. I guess it could be the source of the problem but I'm still not sure how the /home folder becomes 50GB bigger when there are only 25GB of logs.

Anyway, I unchecked this option and I will see tomorrow is there any difference. :)

Sat, 06/13/2015 - 03:49
vld

Just to let you know your suggestion worked! :)

I unchecked the option to backup the logs and the backup was done a lot faster and without excessive disk consumption.

Thank you for your help.

Topic locked