Load-balancing and fail-over has a significant appeal to most sys-admins in the context of client service and disaster recovery - if there were a module that implemented web services (80/443/etc) I would consider investing in it before I work on building out our new systems.

Joe mentioned the concept of a "hot swap" server being on the horizon, but naturally, the greatest appeal is High-Availability AND load-balancing. Right now just getting the topology of our network solid - but as this develops and if I actually get it to work, then I'll be sure fill people in.

In the meantime, unison is a good suggestion and I'll look into it.
http://www.cis.upenn.edu/~bcpierce/unison/index.html

Thanks!

Load-balancing and fail-over has a significant appeal to most sys-admins in the context of client service and disaster recovery - if there were a module that implemented web services (80/443/etc) I would consider investing in it before I work on building out our new systems.

Joe mentioned the concept of a "hot swap" server being on the horizon, but naturally, the greatest appeal is High-Availability AND load-balancing. Right now just getting the topology of our network solid - but as this develops and if I actually get it to work, then I'll be sure fill people in.

In the meantime, unison is a good suggestion and I'll look into it.
http://www.cis.upenn.edu/~bcpierce/unison/index.html

Thanks!

TonyShadwick wrote:
Unison would be a much better choice than rsync for this. :)

That said, I had started developing the Apache Clustering module and stopped, for lack of time, and apparent lack of interest. If there's sufficient interest I'll look into finishing it...
Any chance you'd be interested in finishing it or providing your code? I'm very interested in getting something like this done, but if some work has already been made, I don't want to reinvent the wheel.

(bump)
I would love it if someone who has created a "hot-swap" setup would share a few details of how they have done it. I've been using rsync and I've not had any luck with how to configure the file transfer and which config files where to transfer over...

Thanks for any help you can offer!

So far, this is how I've done it.

Use 4 servers. 2 are NFS/file servers in a heartbeat/drbd setup. Plenty of details on that, although howtoforge has lots of good guides that I recommend. (not associated with them)

The 2 NFS servers are "backend", the other 2 are "frontend"

Setup a private IP to be shared on the NFS servers with failover. Frontend servers use this IP for mounting directories, that way if the primary NFS server dies, the secondary immediately takes over.

Frontend servers serve HTTP, HTTPS, MySQL, SMTP, POP3, IMAP, etc... These are load balanced using heartbeat and ldirectord. Again, plenty of articles on that. Static IPs for SSL domains need to be shared.

The following directories are mounted on the Frontend servers from the NFS servers - essentially "shared":
/home
/var/lib/php
/etc/postfix
/etc/apache/sites-available
/etc/apache/sites-enabled

You could probably also share the /etc/webmin and /etc/usermin folders, although I haven't looked into that fully.

Setup MySQL to run on both Frontend servers in a circular replication. Good notes here http://www.onlamp.com/pub/a/onlamp/2006/04/20/advanced-mysql-replication.html

All 4 of my servers are the same in terms of horsepower - CPU, RAM, HDD, etc. But you could probably use slower/older machines for the NFS/Backend servers.

I tried doing this setup on just 2 servers, but had problems with NFS Client and NFS Server running on the same machines. Whenever trying to simulate a failover, the client wouldn't release files/folders for DRBD.

Need to make sure both Frontend servers are setup in the Webmin Cluster modules. Then whenever you add a domain or user, you need to refresh the users/groups, and then re-synchronize them on both servers. Then restart apache, restart postfix. Also setup domain transfers in Bind on both machines too.

So far, this is how I've done it.

Use 4 servers. 2 are NFS/file servers in a heartbeat/drbd setup. Plenty of details on that, although howtoforge has lots of good guides that I recommend. (not associated with them)

The 2 NFS servers are "backend", the other 2 are "frontend"

Setup a private IP to be shared on the NFS servers with failover. Frontend servers use this IP for mounting directories, that way if the primary NFS server dies, the secondary immediately takes over.

Frontend servers serve HTTP, HTTPS, MySQL, SMTP, POP3, IMAP, etc... These are load balanced using heartbeat and ldirectord. Again, plenty of articles on that. Static IPs for SSL domains need to be shared.

The following directories are mounted on the Frontend servers from the NFS servers - essentially "shared":
/home
/var/lib/php
/etc/postfix
/etc/apache/sites-available
/etc/apache/sites-enabled

You could probably also share the /etc/webmin and /etc/usermin folders, although I haven't looked into that fully.

Setup MySQL to run on both Frontend servers in a circular replication. Good notes here http://www.onlamp.com/pub/a/onlamp/2006/04/20/advanced-mysql-replication.html

All 4 of my servers are the same in terms of horsepower - CPU, RAM, HDD, etc. But you could probably use slower/older machines for the NFS/Backend servers.

I tried doing this setup on just 2 servers, but had problems with NFS Client and NFS Server running on the same machines. Whenever trying to simulate a failover, the client wouldn't release files/folders for DRBD.

Need to make sure both Frontend servers are setup in the Webmin Cluster modules. Then whenever you add a domain or user, you need to refresh the users/groups, and then re-synchronize them on both servers. Then restart apache, restart postfix. Also setup domain transfers in Bind on both machines too.

So far, this is how I've done it.

Use 4 servers. 2 are NFS/file servers in a heartbeat/drbd setup. Plenty of details on that, although howtoforge has lots of good guides that I recommend. (not associated with them)

The 2 NFS servers are "backend", the other 2 are "frontend"

Setup a private IP to be shared on the NFS servers with failover. Frontend servers use this IP for mounting directories, that way if the primary NFS server dies, the secondary immediately takes over.

Frontend servers serve HTTP, HTTPS, MySQL, SMTP, POP3, IMAP, etc... These are load balanced using heartbeat and ldirectord. Again, plenty of articles on that. Static IPs for SSL domains need to be shared.

The following directories are mounted on the Frontend servers from the NFS servers - essentially "shared":
/home
/var/lib/php
/etc/postfix
/etc/apache/sites-available
/etc/apache/sites-enabled

You could probably also share the /etc/webmin and /etc/usermin folders, although I haven't looked into that fully.

Setup MySQL to run on both Frontend servers in a circular replication. Good notes here http://www.onlamp.com/pub/a/onlamp/2006/04/20/advanced-mysql-replication.html

All 4 of my servers are the same in terms of horsepower - CPU, RAM, HDD, etc. But you could probably use slower/older machines for the NFS/Backend servers.

I tried doing this setup on just 2 servers, but had problems with NFS Client and NFS Server running on the same machines. Whenever trying to simulate a failover, the client wouldn't release files/folders for DRBD.

Need to make sure both Frontend servers are setup in the Webmin Cluster modules. Then whenever you add a domain or user, you need to refresh the users/groups, and then re-synchronize them on both servers. Then restart apache, restart postfix. Also setup domain transfers in Bind on both machines too.

So far, this is how I've done it.

Use 4 servers. 2 are NFS/file servers in a heartbeat/drbd setup. Plenty of details on that, although howtoforge has lots of good guides that I recommend. (not associated with them)

The 2 NFS servers are "backend", the other 2 are "frontend"

Setup a private IP to be shared on the NFS servers with failover. Frontend servers use this IP for mounting directories, that way if the primary NFS server dies, the secondary immediately takes over.

Frontend servers serve HTTP, HTTPS, MySQL, SMTP, POP3, IMAP, etc... These are load balanced using heartbeat and ldirectord. Again, plenty of articles on that. Static IPs for SSL domains need to be shared.

The following directories are mounted on the Frontend servers from the NFS servers - essentially "shared":
/home
/var/lib/php
/etc/postfix
/etc/apache/sites-available
/etc/apache/sites-enabled

You could probably also share the /etc/webmin and /etc/usermin folders, although I haven't looked into that fully.

Setup MySQL to run on both Frontend servers in a circular replication. Good notes here http://www.onlamp.com/pub/a/onlamp/2006/04/20/advanced-mysql-replication.html

All 4 of my servers are the same in terms of horsepower - CPU, RAM, HDD, etc. But you could probably use slower/older machines for the NFS/Backend servers.

I tried doing this setup on just 2 servers, but had problems with NFS Client and NFS Server running on the same machines. Whenever trying to simulate a failover, the client wouldn't release files/folders for DRBD.

Need to make sure both Frontend servers are setup in the Webmin Cluster modules. Then whenever you add a domain or user, you need to refresh the users/groups, and then re-synchronize them on both servers. Then restart apache, restart postfix. Also setup domain transfers in Bind on both machines too.

Sorry, didn't mean to post that 4 times. And it won't let me edit now. =/

I have an very similar setup as follows:

7 virtual servers as follows: / Virtualmin GPL with some customizations

2 - linux-ha/drbd redundant content servers making web content available via nfs
2 - front-end webservers load balanced via ldirectord, mounting above nfs content
2 - linux-ha/drbd redundant mysql DB servers
1 - Developer server with all the user accounts, Virtualmin, also mounting nfs content.

/etc/apache2 and /var/lib/php4 and /var/lib/php5 are also shared mountpoints on the apache servers. (ie. no synching necessary)

Works well, but I am facing memory shortages on the webservers at 150 domains right now which I need to resolve.
Also, because of the customizations I have made to VirtualMin, I am always nervous about upgrading when new releases come out, or even about purchasing the commercial version.

I would be interested in comments regarding performance / limitations from anyone else with a similar setup.