Scheduled backups are not deleted on remote sftp server

23 posts / 0 new
Last post
#1 Wed, 11/14/2012 - 05:08
gyorgy.chityil@...

Scheduled backups are not deleted on remote sftp server

Hello,

With the latest versions (3.95, 3.94) Virtualmin stopped deleting old backups. It used to work before! I am attaching a screenshot of the settings, which all seem correct.

Listing backup directories gives me

sftp> ls -la
drwxr-xr-x    4 u46155   u46155          4 Nov  5 11:58 .
drwxr-xr-x    3 u46155   u46155          3 Apr 21  2012 ..
drwxr-xr-x    2 u46155   u46155         59 Nov  2 05:54 02-11-2012
drwxr-xr-x    2 u46155   u46155         54 Nov  3 08:04 03-11-2012

My screenshot does not seem to appear as an attachment, but here it is: https://www.virtualmin.com/system/files/Edit%20Scheduled%20Backup_0.png

Thu, 11/15/2012 - 23:58
JamieCameron

What output do you get in Virtualmin when you try to run this backup?

Also, make sure that the user you are connecting to the SSH server as doesn't have an scponly shell - this prevents Virtualmin from listing the destination directory.

''

Sat, 11/17/2012 - 04:13
gyorgy.chityil@...

Thanks for coming back. I am not sure how to check whether virtualmin uses scponly for backups. I thought virtualmin runs with root?

Here is the full output of a test run. It fails to create the directory because the drive is already full. The drive has plenty of disk space, but due to the failed deletion it becomes full (100 GB drive, one backup is 25 GB)

Starting backup of 1 domains to /backup/%d-%m-%Y on FTP server xxx.xx.xx, local file /backup/daily/%d-%m-%Y ..

Creating backup for virtual server mygreatsite.com ..
Copying virtual server configuration ..
.. done
Backing up Cron jobs ..
.. none defined.

Copying records in DNS domain ..
.. done

Saving mail aliases ..
.. done

Saving mail and FTP users ..
.. done

Backing up mail and FTP user Cron jobs ..
.. none to backup

Backing up Dovecot control files ..
.. none found

Copying Apache virtual host configuration ..
.. done

Copying Apache log files ..
.. done

Copying Logrotate configuration ..
.. done

Dumping MySQL database booksbon_bonanza ..
.. done

Backing up Webmin ACL files ..
.. done

Creating TAR file of home directory ..
.. done

.. completed in 20 seconds
Uploading archive to FTP server xxxx.xxxxxxx.de ..
.. upload failed! STOR /backup/17-11-2012/mygreatsite.com.tar.gz failed : /backup/17-11-2012/mygreatsite.com.tar.gz: No such file or directory

Backup failed! See the progress output above for the reason why.

Before the drive became full I received errors like

Uploading archive to FTP xxx.xx-xxxxxx.de ..
.. upload failed! Upload incomplete - file size is 7676658117, but sent 7039603712
Sat, 11/17/2012 - 12:06 (Reply to #3)
JamieCameron

It looks like that backup didn't complete at all .. which will prevent deletion of old backups from happening, as Virtualmin won't attempt to clean up old backups unless it knows that the most recent one was successful.

''

Sat, 11/17/2012 - 12:15
gyorgy.chityil@...

I think the backup completed since it says ". completed in 20 seconds" The ftp upload does not complete. I can see the backup in my local backup location (I have two, one locally, and the second one on the remote ftp)

Sat, 11/17/2012 - 12:43 (Reply to #5)
JamieCameron

The creation of the backup tar file completed, but it didn't get uploaded to the FTP server .. so the overall status was incomplete.

''

Sat, 11/17/2012 - 12:59 (Reply to #6)
gyorgy.chityil@...

Yep, the reason is not completed is because the server is full, because the previous backups were not deleted.

So, the issue is the old backups are not deleted, and the server runs out of space. :)

Sat, 11/17/2012 - 13:27 (Reply to #7)
JamieCameron

Try manually deleting some old backups from the server to free up space, running the backup again in Virtualmin, and let us know what it outputs..

''

Sat, 11/17/2012 - 15:24 (Reply to #8)
gyorgy.chityil@...

Yep, been doing that for a few weeks now. :)

I delete old backups, and the backups run again, but not deleted until the disk space runs out.

Tried completely wiping the drive, so no old folders are present, still old ones are not removed.

Sat, 11/17/2012 - 16:10 (Reply to #9)
JamieCameron

I'd be interested to see the full backup output when the remote drive isn't full, to see if it attempts to purge old backups, and if so what goes wrong.

''

Mon, 11/19/2012 - 00:54 (Reply to #10)
gyorgy.chityil@...

Thanks, I enabled reporting for successful backups as well, and this time it seems to have worked. Not sure why. Here is the output:

One weird thing I noticed that it reported some directories having the same age in days, even though they have different dates , eg Deleting directory /backup/daily/11-11-2012, which is 8 days old .. Deleting directory /backup/daily/10-11-2012, which is 8 days old ..

.. completed in 1 minutes, 15 seconds

Uploading archive to FTP server xxxx.xxxxxxxxx.de ..
.. done

19 servers backed up successfully, 0 had errors.

Deleting backups from /backup/%d-%m-%Y on FTP server xxxx.xxxxxxxxxx.de older than 2 days ..
    Deleting FTP file /backup/12-11-2012, which is 6 days old ..
    .. deleted 27.33 GB.

    Deleting FTP file /backup/14-11-2012, which is 4 days old ..
    .. deleted 27.77 GB.

.. deleted 2 old backups

Deleting backups from local file /backup/daily/%d-%m-%Y older than 5 days ..
    Deleting directory /backup/daily/13-11-2012, which is 6 days old ..
    .. deleted 27.60 GB.

    Deleting directory /backup/daily/09-11-2012, which is 10 days old ..
    .. deleted 26.66 GB.

    Deleting directory /backup/daily/11-11-2012, which is 8 days old ..
    .. deleted 1.80 GB.

    Deleting directory /backup/daily/12-11-2012, which is 6 days old ..
    .. deleted 27.33 GB.

    Deleting directory /backup/daily/10-11-2012, which is 8 days old ..
    .. deleted 27.78 GB.

.. deleted 5 old backups
Mon, 11/19/2012 - 09:49 (Reply to #11)
JamieCameron

The number of days old it displays the backup as is based on the timestamp on the directory, not the file name. So if files were added to or deleted from the directory, the timestamp could change.

''

Mon, 11/19/2012 - 09:50
gyorgy.chityil@...

Ok thanks, will keep an eye on this and report back with some concrete workable material if it fails.

Wed, 11/21/2012 - 00:24
gyorgy.chityil@...

Deletions started failing again, and it seems the script is not recognizing / finding old backups, even though they are there. Here is the output from the backup command. In 1 or 2 days the remote sftp server will be full again:

Uploading archive to FTP server xxxx.xxxx-xxxxxxxx.de ..
.. done

19 servers backed up successfully, 0 had errors.

Deleting backups from /backup/%d-%m-%Y on FTP server xxxx.xxxxx-xxxxxx.de older than 2 days ..
.. no backups to delete were found

Deleting backups from local file /backup/daily/%d-%m-%Y older than 5 days ..
    Deleting directory /backup/daily/15-11-2012, which is 6 days old ..
    .. deleted 27.88 GB.

.. deleted 1 old backups
Wed, 11/21/2012 - 16:14 (Reply to #14)
JamieCameron

Could you post the listing of the directory /backup from the FTP server? Ideally obtained by using the ftp command to connect to the FTP server. The issue may be that the dates as shown by FTP aren't in the format Virtualmin expects..

''

Wed, 11/21/2012 - 23:33 (Reply to #15)
gyorgy.chityil@...

Thanks, here is the list

sftp> dir
19-11-2012   20-11-2012   21-11-2012   22-11-2012

or

sftp> ls -la
drwxr-xr-x    6 hidingmyftpuserid   hidingmyftpuserid          6 Nov 21 22:56 .
drwxr-xr-x    3 hidingmyftpuserid   hidingmyftpuserid          3 Apr 21  2012 ..
drwxr-xr-x    2 hidingmyftpuserid   hidingmyftpuserid         59 Nov 19 04:10 19-11-2012
drwxr-xr-x    2 hidingmyftpuserid   hidingmyftpuserid         59 Nov 20 08:10 20-11-2012
drwxr-xr-x    2 hidingmyftpuserid   hidingmyftpuserid         59 Nov 21 06:09 21-11-2012
drwxr-xr-x    2 hidingmyftpuserid   hidingmyftpuserid          2 Nov 21 22:56 22-11-2012
Thu, 11/22/2012 - 00:45 (Reply to #16)
JamieCameron

Thanks, that looks fine.

How many days do you have the backup configured to delete from the FTP server after though?

''

Thu, 11/22/2012 - 00:46 (Reply to #17)
gyorgy.chityil@...
Thu, 11/22/2012 - 12:41 (Reply to #18)
JamieCameron

In your comment from 2012-11-21 23:33 , it looks like you are using sftp to list the directory.

What if you use the regular ftp command? In that case, the listing format is controlled by the server .. that's what I'm interested to see.

''

Thu, 11/22/2012 - 13:14 (Reply to #19)
gyorgy.chityil@...

Ah, ok. I didn't have ftp installed on my server, so after installing this is what I have:

ftp> dir
227 Entering Passive Mode (xxx,xxx,xx,xxxx).
150 Opening ASCII mode data connection for file list
drwxr-xr-x   2 hidingmyftpuserid   hidingmyftpuserid         59 Nov 19 04:10 19-11-2012
drwxr-xr-x   2 hidingmyftpuserid   hidingmyftpuserid         59 Nov 20 08:10 20-11-2012
drwxr-xr-x   2 hidingmyftpuserid   hidingmyftpuserid         59 Nov 21 06:09 21-11-2012
drwxr-xr-x   2 hidingmyftpuserid   hidingmyftpuserid         36 Nov 22 07:29 22-11-2012
226 Transfer complete
Fri, 11/23/2012 - 00:40 (Reply to #20)
JamieCameron

Again, that looks fine..

You might want to wait a few days till Virtualmin 3.96 comes out, as I have fixed some bugs in this area .. it's possible that the new version will solve this problem.

''

Fri, 11/23/2012 - 00:41 (Reply to #21)
gyorgy.chityil@...

Lovely! Much appreciated, look forward to that version. :)

Wed, 11/28/2012 - 02:09 (Reply to #22)
gyorgy.chityil@...

Success!

Reporting that the remote files were deleted with Webmin 3.96! The backup failed though because the disk was full already (I was lazy to delete it), so the upload failed, but from tomorrow it will be working fine as there is enough space now...Many thanks.

Uploading archive to FTP server xxxxxxx.xxxxxxx.de ..
.. upload failed! STOR /backup/28-11-2012/mysitename-m.com.tar.gz failed : /backup/28-11-2012/nysitename-m.com.tar.gz: No such file or directory

Deleting backups from /backup/%d-%m-%Y on FTP server xxxxxx.xxxxxx.de older than 2 days ..
    Deleting FTP file /backup/19-11-2012, which is 9 days old ..
    .. deleted 28.49 GB.

    Deleting FTP file /backup/20-11-2012, which is 8 days old ..
    .. deleted 28.73 GB.

    Deleting FTP file /backup/21-11-2012, which is 7 days old ..
    .. deleted 28.96 GB.

    Deleting FTP file /backup/22-11-2012, which is 6 days old ..
    .. deleted 13.81 GB.

.. deleted 4 old backups