Using Shared Storage for High Availability (NFS or FC)

Hello Support,

Please i need your help setting my environment up. I have provisioned two Webmin/Virtualmin Instances with the same content on both of them and Cloudmin Pro on a seperate machine. One is supposed to act as a primary and the other as a secondary. They are both Virtual Machines provisioned on a VMware Esxi and they are installed on a FC SAN. These two machines use their own storage currently and considering their sizes, it will be a waste of storage space having them act independently, since they already exist on a SAN. How do I configure these machines to use the storage of the Primary alone for both VMs? I already created an NFS Share of the root partition on the primary and added it to the secondary successfully. Do i boot the secondary from this nfs share of the root directory (this seems not to work as it indicates another machine is using the storage) or am I supposed to share just the home directory between them so that Cloudmin replication can work well? If i'm sharing the home directory, how do i set the secondary server to use the home directory share as its default directory? Also, i'm having issues accessing the NFS share from the Secondary while FirewallD is running. I need to turn off FirewallD before i can access it and i've added the NFS, mountd and rpcinfo services and ports, yet it isn't accessible until i turn off FirewallD. Is there something else i need to do?

Thanks

Status: 
Active

Comments

You could export /home from the primary via NFS, and configure the secondary to mount the same directory - then use Virtualmin's replication feature to sync up other domain data.

However, if the primary machine goes down, the NFS export would also be unavailable. So this wouldn't really protect you against downtime ..

Hi JamieCameron,

Thanks for the response and suggestion. I've created the NFS Share on the SAN and successfully mounted the /home directory to it on both the primary and secondary Virtualmin Servers.

Regards

Ok .. and do you have any MySQL databases? If so, they should also be stored on the SAN for reliability.

Hi jamie,

No we don't yet. I'll however change the location to the SAN as well as you have suggested.

Thanks.

Hi Jamie,

Is there a way to achieve an Active/Passive failover system with my setup as opposed to using Active/Active option that DNS Roundrobin load balancing provides? We want the Primary system to serve requests and if anything happens to it, it should failover automatically to the Secondary.

I await your response on this.

Thanks

Yes, you can use Cloudmin's DNS Roundrobin feature to have a DNS entry that points to whichever of the primary or secondary is up.

Okay Jamie,

So how do I set up the DNS Roundrobin feature on Cloudmin to make the DNS entry resolve only to the Primary when both Systems are up and then failover to the Secondary when the Primary is down.

Thanks.

You need to select the primary and secondary in the "Selected systems" field, and in the "Address records to include" field select "At most" and enter 1 in the adjacent text box.

Hi,

Following your direction, i noticed that the Status From Check tab showed the Primary Server as "IP Address Included" and the other one as "IP Address not included".

However, i simulated a failover by shutting down the Primary and the Status Check changed. The Primary showed "Excluded due to Status being down" but the Secondary Server still remains at "IP Address not included".

Did you wait a few minutes for Cloudmin to detect the downtime and update the IP address?

Yes i did... waited a longtime. It detects that the Primary host is down and changes the the status to "Excluded due to Status being down" but the secondary remains at "IP Address not included" all the while i was waiting and the IP address is not included in DNS as well.

I had a look at the code behind this, and noticed that the "IP address not included" message can be shown incorrectly when only one server of the two is up. In the case where the primary is down, which IPs are actually included in the DNS record?

Hello Jamie,

Thanks for your help. I checked as you said and realized that though Roundrobin still shows "IP address not included", the BIND DNS record has been changed to the Secondary Servers IP, which shows its actually working. But i noticed that the DNS record wasn't still resolving to the secondary servers IP when i tried to browse to it and did an nslookup until i cleared the cache of BIND DNS on the Cloudmin server. I also had to do the same thing when i brought up the primary server to have the record change back to its IP. My TTL for the A record is set to 60 secs on cloudmin and i'm wondering if i have to clear the cache every time for this record to be updated or is there something else i can tweak.

What did you do to clear the cache exactly? Restart BIND, or something else?

I ran the commands;

rndc flush

rndc reload

It was afterwards nslookup for the record was then pointing to the primary server.

I'm curious as to why you also needed to run rndc flush ?

I ran it to flush the cache.... after the Primary was down and the secondary's IP address was included/showing in the DNS, the record was still resolving to the Primary's address. It didn't change until i did the flush.

The same thing happened when the Primary came back up and it was its IP that was included in DNS.

Ok, so it was the local DNS cache. What's the TTL set to on that DNS record?

TTL on the record is set to 60 seconds.

So i simulated this again and left it for over 12 hours now..... BIND DNS shows the record pointing to another IP but opening the page or doing an nslookup of the record points to another. This doesn't change until one does a DNS flush.