Hey Chris,
The
very first Virtualmin installation three years ago was a fully redundant configuration. It wasn't easy, but it's gotten easier since then. ;-)
There are at least three orthogonal issues you have to deal with:
[*]Failure detection and takeover[/*]
[*]Synchronization of configuration and data[/*]
[*]Notification of problems[/*]
Webmin has help for you on a couple of these, though you need an extra software component.
The first issue is failover. For this, I used Heartbeat, and I have zero complaints about it. It works beautifully, simply, reliably, and it has a nice Webmin module. You can get it here:
http://linux-ha.org/download/index.htmlSynchronization of configuration and data can be done in roundabout ways with Webmin...but I think the ideal choice is rsync. You can configure it to run over ssh with a public key on the sending machine, so it can happen via a cronjob. On the system I setup, I ran rsync every hour. This is probably an area we ought to address in Webmin, but it's a pretty wide-open task and there are dozens of ways to go about it, depending on your focus. It's also easy to screw things up on the secondary machine--many configuration files are specific to the box, even if the boxes are identically configured and are expected to behave identically after a failover. Network details can't change, for example. I believe I had two or three subdirectories of /etc that I excluded from my rsync process, and otherwise included all of /etc, all of /home, and the /var/named directory (depending on OS and configuration, this may be in /etc or elsewhere). I'll think on doing something like this for Webmin. For a very specific purpose (like a Virtualmin hosting server), it's probably easy enough to select the right bits and then use the remote function API to copy the right stuff over on schedule, possibly using rsync to reduce the size of transmissions (now we're getting fancy!).
The final bit is notifications, since you need to know about failover events so you can fix whatever went wrong on the primary. I believe the System and Server Status module covers this nicely. Both servers can monitor each other, and send notifications in the event of trouble. Once when I didn't want to use heartbeat, I used this module all by itself to detect failure and takeover and IP automagically. Worked just fine, but seems like a "when all you've got is a hammer" kind of solution. ;-)
This would be a good area for a meta-module (like Virtualmin...stepping out of the traditional role of Webmin modules that address only one service or configuration file and actually perform actions on many modules in order to ease a specific task--more complicated to write and much more specific in function, but more useful in the circumstances for which it was written). Maybe some third party will step up to the plate and tackle this one. It's a big job, though, and Jamie and I have our work scheduled out for at least two or three months.
