Varnish and Virtualmin: Changing Website IP Addresses

Hi,

I currently have Virtualmin running on two servers, a dedicated server which acts as my "main" server and a cloud instance from DigitalOcean. I also have a second cloud instance that is just running the LDAP server and acting as a secondary BIND and Postfix slave.

I recently came across Varnish and after some testing I believe it will work well in my environment. What I'd like to do is set up another server that only runs Varnish, and then point its backends to my Virtualmin servers that are serving Web content via Apache/PHP.

So my question is, is there a way to change the DNS "A" record for only the addresses for domains that will be accessed over the Web? As an example:

  • My dedicated server, we'll call it Virtualmin #1, hosts domain.com and domain2.org. It is my first backend in Varnish.
  • My first cloud instance, we'll call it Virtualmin #2, hosts site.com and site2.org, and is the second configured backend in Varnish.
  • My Varnish machine only runs Varnish and does not run anything else. The two Virtualmin machines, besides running Apache for the sites they host, also run DNS, mail, spam/virus filtering, mailing lists, the whole works.

Now, let's say that I want to put Varnish in front of both of my Virtualmin servers, so as indicated above I set the two Virtualmin servers to be Varnish backends. However, I still want the Virtualmin servers to handle e-mail, FTP, etc. for the sites they host, but I want the IP addresses for the Web portions, like www.domain.com, www.site.com, sub.domain.com, etc. to be pointed at the Varnish instance (but leave, for example, mail.domain.com back at Virtualmin #1).

Is this possible? If so, is there a way to automate this in Virtualmin?

Thanks!

Status: 
Active

Comments

Howdy -- what it sounds like you may want is to set the DNS records for domain.com and site.com, and www.domain.com and www.site.com, to point to your Varnish server.

Then, set the DNS records for ftp.domain.com, ftp.site.com, mail.domain.com, and mail.site.com, to point to your Virtualmin server (and you can set any other names you want pointing there as well, those are just some examples).

You can see all the current DNS records in Server Configuration -> DNS Records.

Then, depending on what name is used to access the site, it would either be directed to the Varnish server, or to the Virtualmin server.

Does that sound like it will do what you want?

That sounds exactly like what I want to do. Is there a way to have Virtualmin do this for me automatically when creating new virtual servers, and a way to mass update these records for existing servers?

What you would need to do is to go into System Settings -> Server Templates -> Default -> BIND DNS Domain, and there, setup all the DNS Records you would like to setup in "BIND DNS records for new domains".

I'm reviewing the various command line DNS options for Virtualmin, and I'm not seeing a way that would automate modifying all the IP addresses you'd need to update.

You can mass-add records, or mass-delete records, but it doesn't appear that it would be simple to mass modify all the records in particular that you would need to change.

How many domains are you looking at setting up in Varnish, out of curiosity?

I am hopefully looking to set up all existing domains (about 17) and all new domains in Varnish, but as I said keep mail, FTP, and etc. pointed at the Virtualmin boxes where these domains are hosted. I looked in the server templates but am a bit confused as to how to specify which subdomains, like mail, ftp, etc., get the Virtualmin IP address and which DNS records get the IP address of my Varnish instance.

Well, what you're looking to do requires a bit of an overhaul to the DNS records that Virtualmin normally generates.

So in in the BIND DNS template, what you would need to do is manually specify all of the records you want generated, including the names and IP addresses that they'll use.

You would place all those in the "BIND DNS records for new domains" textarea, and you'd want to set the "Use only the records above" option.

All the records that you see in Server Configuration -> DNS Records -- those are what you would need to add that textarea field, but with your modified IP addresses.

The records you add there are exactly what will be put into the DNS zone file for your domains.

Ah, that makes more sense. My only question is, I see that some of the records are for DKIM signatures and other things like that. How can I add those to the DNS templates, I mean aren't they automatically generated?

Thanks!

Hi, just checking in to see if you've seen my last reply. I am still unsure as to how to automatically generate things like DKIM records and etc. that I see in the DNS zone of each domain so that I can enter all of my custom records into the server template. Or will Virtualmin still handle all that even with my custom records? Thanks!

You shouldn't need to manually add records such as those for SPF or DKIM, as they're automatically generated, and not part of that particular DNS template.

There is a separate part of Virtualmin where those records can be enabled or disabled.

For DKIM, that's available in Email Messages -> DomainKeys Identified Mail.

Hi, I was able to create a DNS records template successfully, and it works when creating new virtual servers. However, how can I update the existing virtual servers and their "www", "ftp", and other records (excluding mail) to point to my Varnish server? I tried backing up all virtual servers, deleting them, and then restoring them while hoping the new template would take effect, but they just gained the records they had before the deletion process. Is there an automated way to update these records to point to my Varnish instance?