Integrating ZFS with Virtualmin

10 posts / 0 new
Last post
#1 Fri, 08/23/2019 - 05:45
danboid

Integrating ZFS with Virtualmin

I recently configured a Virtualmin server using ZFS. I couldn't find any guides for integrating ZFS with Virtualmin so I thought I'd share my notes here.Unfortunately all the formatting will be lost but its better than nothing right?

Configuring a ZFS Virtualmin server

This page outlines the key steps required to setup a new (Ubuntu) Virtualmin server with users home directories stored on a ZFS drive. The main advantage to doing this is that each user will have access to automatic snapshots of all the files in their home directory. Virtualmin doesn't officially support ZFS but it can be integrated nicely if you follow this guide. This guide assumes you have at least two drives available - one for the OS and another to store /home which will be formatted as ZFS.

Creating a ZFS /home directory

Ubuntu 19.10 and later versions will have support for creating ZFS on root installations but I used Ubuntu 18.04 with /home located on an ext4 drive to install Virtualmin. ZFS would be preferable for the root drive too but it doesn't really matter what filesystem is used for the main OS drive.

Let's assume the drive we are going to use for the ZFS /home directory is /dev/sdc. First we'll install the ZFS utils, create a Solaris partition and create a ZFS pool on that called zhome. All these commands need to be run as root or with sudo:

# apt update && apt upgrade
# apt install zfsutils-linux
# sgdisk --zap-all /dev/sdc
# sgdisk --new=1:0:0 --typecode=1:BF00 /dev/sdc
# zpool create -o ashift=12 -o autoexpand=on -o autoreplace=on -O atime=off -O compression=lz4 zhome /dev/sdc1

Next we'll create a dataset called home on our new pool, copy the existing /home into it, delete the old /home and mount the new one:

# mkdir /tmp/home
# rsync -avh /home/ /tmp/home/
# rm -rf /home/
# zfs create -o sharesmb=off zhome/home
# zfs set mountpoint=/home zhome/home
# rsync -avh --ignore-existing /tmp/home/ /home/

After these steps, it is probably worthwhile rebooting to check all is well and that your /home has been successfully relocated onto the ZFS disk.

Set hostname and FQDN

Before downloading and running the install script for Virtualmin, it is recommended to ensure the hostname (in /etc/hostname ) and Fully Qualified Domain Name are set correctly. Setting the FQDN is as simple as adding a line such as the following to /etc/hosts :

127.0.1.1 yourdomain.ac.uk yourdomain

Running the Virtualmin install script and Install Wizard

The Virtualmin install script expects to be run as root on a clean install of Ubuntu LTS, Debian or RHEL. Refer to the website for download instructions but it is likely to be:

# wget http://software.virtualmin.com/gpl/scripts/install.sh
# sh ./install.sh

After the install script has completed, it will give a warning about not being able to set quotas on /home. This can be safely ignored because we will set users disk quotas in a custom shell script that will be run when creating new users/domains. Before logging into Virtualmin and running its install wizard, you will need to ensure at least incoming TCP ports 10000 (for the web interface) and 53 (for DNS) are open. You can use your normal Linux credentials to login to vmin. When the wizard prompts you for the nameserver, enter the FQDN of the virtualmin box.

Setting up DNS

In order for Virtualmin to be able to create new subdomains for each user, you first need to have a wildcard domain set up with your domain registrar that points at the public IP of the virtualmin server. In addition to that, there are a couple of settings to configure in webmin.

Under Webmin, go to Networking -> Network Configuration -> Hostname and DNS Client and enter 127.0.0.1 and the address of your DNS registrars DNS server (168.63.129.16 for Azure) into the DNS server text entry boxes and save the settings. Also under Webmin, go to Servers -> BIND DNS server and under 'Existing DNS zones' click 'Create master zone' then type the FQDN of the vmin server into the 'Domain name / Network' and 'Master server' fields before clicking 'Create' to add the zone.

Integrating ZFS

As previously mentioned, Virtualmin doesn't officially support ZFS so we have to add the missing functionality. The following script will create a new ZFS dataset and apply a quota to it for every new user/domain created either via the web interface or with a script/command line. It will also destroy the dataset when the domain/user is deleted. Note that destroying the dataset will fail if the user is logged into their home directory via SSH at the time of deletion. I store this script under /usr/local/bin/vmin-zfs-domain.sh:

#!/bin/sh
if [ "$VIRTUALSERVER_ACTION" = "CREATE_DOMAIN" ]; then
  /sbin/zfs create zhome/home/$VIRTUALSERVER_USER
  /sbin/zfs set refquota=200M zhome/home/$VIRTUALSERVER_USER
fi
if [ "$VIRTUALSERVER_ACTION" = "DELETE_DOMAIN" ]; then
  /sbin/zfs destroy -r zhome/home/$VIRTUALSERVER_USER
fi

Change the refquota value to the amount of disk space you wish for users to have by default. This can be modified later per user/dataset using zfs set. After you have created that script and set it to be executable, we need to tell Virtualmin when to run it by going to System Settings -> Virtualmin Configuration -> Actions upon server and user creation and pasting the full path to the script in the field next to 'Command to run before making changes to a server' before clicking Save.

SSL certs

Virtualmin has integrated support for adding and renewing Let's Encrypt SSL certificates. First you need to install the required packages with:

# apt install letsencrypt python3-certbot-apache

You must have at least one user/domain created under virtualmin before you can add the certs because you need to use a users public_html directory for the Let's Encrypt domain ownership verifcation process to work. Once both of these requirements are met, go to Webmin -> Webmin Configuration -> SSL encryption -> Let's Encrypt then choose the 'Other directory' option for 'Website root directory for validation file' and enter the path to the users public_html dir eg /home/dan/public_html . Set 'Months between automatic renewal' to 2 and then click 'Request Certificate' .

MySQL

The MySQL server only allows access from localhost by default. To allow users to login to the MySQL server from anywhere you must edit /etc/mysql/mysql.conf.d/mysqld.cnf , find the bind-address line and change its value from 127.0.0.1 to 0.0.0.0 then restart the mysql server. It is also required to open incoming port 3306 for the VMs network interface under Azure.

Disable unwanted Virtualmin plugins and features

There are a few virtualmin plugins that are enabled by default that you may want to disable before adding users. Webalizer and awstats are log file analysers which both create files in every users ~/public_html directory. Most users are unlikely to use these so best to disable them to avoid the confusion. The DAV Login plugin should also be disabled to increase security unless it is being used.

Additional configuration

I have only really documented the parts of setting up Virtualmin here that I didn't think were well documented online, or not documented at all as was the case with the ZFS integration. You will also likely want to create and configure at least one Server Template and one Account Plan to restrict what users can and cannot do before you start adding users, creating them using your templates and plans. You will also want to set up regular scrubs of the pool via a cron job and install zfs-auto-snapshot .

Fri, 08/23/2019 - 09:03
scotwnw

Been looking to do something similar. I use zfs manager in webmin to create mount points/file systems/quotas manually. Haven't delved into moving /home mount point to zfs, except on my home desktop. This is awesome and simple to do. And all user's files could be compressed on the fly. Huge win for space/speed for all those php text files and mail folders.

AS far as users handling their own snapshots. Thats not possible in this setup. They would need cmd line access and be able to run zfs commands. That requires sudo/root. I have been playing with custom cmds in webmin for users to delete/restore snapshots. But a user has to understand what delete does, what restore does and the difference between dates/times on the server vs the user time zone. FYI, delete snapshot means merge into the original files, meaning orig files now include all that was in the snapshot, and then the snapshot itself is deleted. To me, delete a snapshot sounds like it should mean delete all files in the snapshot and revert to the way it was before the snapshot. Which is the exact opposite of what it actually does. So its definitely confusing for users.

Fri, 08/23/2019 - 09:10 (Reply to #2)
danboid

Hi scotnw

Whilst users cannot create their own snapshots (unless they are granted sudo rights) or use zfs rollback, they don't need to as zfs-auto-snapshot handles the creation of snapshots and they can restore individual files and directories (or all of them) from the various snapshots created of their dataset from within the ~/.zfs/snapshot folder contained in the root of every home directory / dataset.

Fri, 08/23/2019 - 10:32
scotwnw

Hmm. I can not find said hidden .zfs folder in my zfs 'file systems' root. I have auto snapshots on as well. Been racking my brain forever trying to get users access to snaps. Guess I have more reading to do. Thx

Fri, 08/23/2019 - 10:54
danboid

That's odd. Are you running a very old version of ZoL?

Fri, 08/23/2019 - 11:38
scotwnw

I running default zfs version on Ubuntu 16.04 and Ubuntu 18.04. I dont find .zfs folder on either. Not in mount point paths nor in pool paths. Nor at at all on the whole machine with 'locate .zfs'. Has to be something simple I'm missing. I'll feel like an idiot when I find it. lol

Sat, 09/07/2019 - 10:04 (Reply to #6)
effgee

Try just cd .zfs in shell in a folder that should have snapshots.

Its super hidden and does not show up in directory listing even when including hidden files.

Mon, 09/09/2019 - 08:46
scotwnw

That did it. I knew I was stupid, didn't know I was that stupid. lol. I was doing 'ls -a' and not seeing .zfs, so assumed not there. Thx

Wed, 10/02/2019 - 08:06
danboid

I have updated the ZFS DESTROY_DOMAIN command in the domain creation script to read

/sbin/zfs destroy -r zhome/home/$VIRTUALSERVER_USER

Previously it was missing the -r option so destroying the dataset would fail if it had any snapshots.

I also added some notes on enabling external MySQL server access and disabling unneeded plugins.

Sun, 10/06/2019 - 18:37
calport

Thanks danboid. I am posting this to be kept informed about further comments. Also I have an account with rsync.net and they use ZFS; it would be interesting to see if I can remotely mount a drive and put the /home directory on rsync.net infrastructure. It might not offer the fastest web hosting but it does open up other possible services which can be offered by combining virtualmin with rsync.net storage.