[SOLVED] Why does just one Virtualmin scheduled backup hang when all others are OK?

10 posts / 0 new
Last post
#1 Fri, 11/11/2016 - 03:01
otamate

[SOLVED] Why does just one Virtualmin scheduled backup hang when all others are OK?

I have 3 Virtualmin production servers running scheduled backups to 2 office servers. They are all set up the same except each backup is set for different days, so they are dotted across the week randomly. Each VM instance will cycle, performing a full backup to both trarget servers. They are all set for 1am. They all work fine and I get the success emails from them apart from one in particular, which starts then just does not complete. When i check "running backups" it shows that one as still running.

As a test, I changed the schedule for that one to daily and after 4 days the job failed with "too many running backups - limit is 3". When I check the running jobs, I see the 3 are running. What can I check with this job to see why it is behaving differently to the others please? I'd also point out all this is duplicated for the Webmin config backup, which runs at 3am, and also the other two scheduled Vritualmin backups to the problem target also always work. This means I know I can dismiss FTP / permission etc problems. I make use of the %Y/%m/%d directory name substitution and the target directories are created. When the backup which is hanging targets the other server, the one which works, with identical params, it takes 2 hours. When I check the directory structure of one of these hung backups, I see the folder has been created and partially populated in alphabetic order of the VM host names, but stopped at some random point. It seems there is no log to check because the backup neither fails nor succeeds - but please correct me if I'm wrong there. Thanks.

Fri, 11/11/2016 - 10:32
Masplus

Can you post the log of this backup, you can get them at:

  • Virtualmin > Backup and Restore > Backup Logs

For a first impression it seems be a problem with I/O of your server or your backup destination server, because it takes too long to make theese backups.

Sun, 11/13/2016 - 03:35 (Reply to #2)
otamate

No log is created - the backup doesn't finish. Is there a time limit to backups?

Tue, 11/15/2016 - 11:31 (Reply to #3)
Masplus

Yes, but the error is caused by the system instead of virtualmin. Check if you have Input/Output errors. Try to make that backup locally, in the same server.

You can check /var/log/syslog to check if there is a problem ( CPU, RAM, I/O, etc )

Wed, 11/16/2016 - 05:05
otamate

Thanks. Did you mean the fault is likely to be on the source or destination server? The target server is Fedora, the source is Ubuntu. So there's no /var/log/syslog on the target, but I just checked for hardware errors and nothing showed up. Would welcome a more likely place to look on the destination (Fedora) server than journalctl. Remember when the source is either of the other two all completes normally, and when the destination from the same source is the other office server, also Fedora, all completes normally. It smells like some comms issue, since the offices are in separate cities.

Wed, 11/16/2016 - 06:24 (Reply to #5)
Masplus

If you can, try to make that backup locally in the same machine ( not at the Fedora ), to check if that finishes correctly.

To take more information....

  • The VirtualHost that you want to make backup has any special configuration ? It's a normal website ? ( Apache + MySQL, for example )
  • The total disk quota usage for this virtualhost is too big ? If it's too big you may check /tmp to verify if that backup can be exported there ( VirtualMin backups can't work directly to the destination server, always make backup locally and then uploads it to the destionation server ).
Wed, 11/16/2016 - 06:39
otamate

Hi Masplus - We may be onto something here. First, its a full backup of all servers, so there isn't one particular one I can deal with. But - I did just notice the destination server which fails has a 2G /tmp folder and the one which works has a 4G one. I'm assuming the source server, the one running virtualmin and performing the backups, must be OK since it always works with the 4G one, i.e. it's /tmp folder must therefore be correct?

Wed, 11/16/2016 - 07:07 (Reply to #7)
Masplus

It seems to be the problem, but that will be printed on the log system or in the backup log like "no space left on the device". If you have a partition with more free space, you can assign a new /tmp directory for backups and all the virtualmin temporary stuff.

Go to: https://server:10000 > WebMin section > WebMin menu > WebMin Configuration > Advanced options > Temporary files directroy.

Also, you can assign more space in actual /tmp partition.

Fri, 12/02/2016 - 11:55
otamate

I have solved this! After exhausting everything else, I changed from FTP to SSH and it is completing the backups properly now. I also get the advantage of security, so I actually went through all the other schedules making the same change, which continued as before.

Mon, 12/12/2016 - 03:30 (Reply to #9)
Masplus

Perfect ! Yes, in previous versions of Webmin+Virtualmin FTP backups didn't work correctly. So all of our backups were configured with SSH, wich is faster, more secure and allows to use ( without problems ) a non-standart port.

If your problem is solved, you can mark the post as solved. Thank you for your feedback, this post will help other users.

Topic locked