I'm growing more and more inclined to formally write a webmin module (or virtualmin module?) that handles not just local Apache, but apache across several physical systems.

I might have time to start on this today, but I have some questions (of course!).

1. The webmin/virtualmin remote API - is it fully aware of Apache's configuration? IE, is there anything in place to push config changes to remote boxes?

2. Is this something that is already being worked on that I could lend a hand on rather than starting a new one?

My intention is to set the module up where a prerequisite is that the boxes have some form of shared storage (smb/nfs/afs/etc) that allows them to access the hosted files at the same filepath, and that they have the same apache build and config. Webmin simply updates the configs on the fly, keeps track of stop, start, and restart status.

The upshot of this is that there needs to be a load balancer in front of these boxes. I was going to write a wrapper script for ifconfig to handle this, that updates my load balancer, and maybe have webmin updating dummy0. haven't decided yet. Thoughts?

Hi,

This sounds very interesting, ive got a few questions (excuse my newbie ignorance if ive missed something entirely):

- Is the intention here for redundancy? Or are u intending to simply create a managed network?
- If redundancy have you looked into clustering? I.e. www.linux-ha.org
- Some detail on the application would be helpful.

Im very interested in developing a clustered environment similiar to this for my business, which would have multiple, concurrent instances of client accounts.

Regards,

Morgan

I didn't have time to get on this today as I'd hoped, but I'll answer your questions regardless. :)

This could be used either for fault tolerance or "clustering" (I prefer the term farm, as these aren't acting as a single cpu instance, but the term cluster gets used so much these days...). In fact, it would be able to do both.

In my situation, I have a couple of load balancers in deployment already. These are fault-tolerant, and I can assign a pool of port 80 systems to forward requests to, the only requirement is that the systems can all respond to those requests more or less identically. The module I'm looking to create allows you to do that from Webmin/Virtualmin.

My question was basically whether I could use the extant API's to handle application state (ie, is the remote apache running? stopped?) and whether there was an extant way to send over updated configs for apache (I'm thinking not).

In the case that there is no way to push configs, I'm thinking of creative rsync use. Haven't decided how to go about that exactly yet.

Hey Tony,

I think you'll be pleasantly surprised by how much of this stuff is available in the API (primarily the Webmin API), and how tightly you can integrate it into the Virtualmin interface.

Here's what I would do, were I embarking on this job:

Create a Virtualmin plugin (Cluster Web Service, or something). A Virtualmin plugin is just a special case of Webmin module...it has a couple of extra hooks, and doesn't have to have a GUI. The extra hooks allow you to add a checkbox on the Create/Edit Virtual Server page, which you could label "Add to web server cluster".

That's the easy part.

Now, to get the other stuff, you'll want to use Webmin RPC to call out to one or more remote servers to add the new domain to the Apache on those systems (every function in apache-lib.pl can be called remotely, so you could mirror the Virtualmin steps on the remote machines). This bit is easy on creation, but gets more difficult on changes (like installing scripts or whatever, where Virtualmin makes changes to the httpd.conf on the master...). Actually, it may be necessary, or at least easier, to make the changes in the core code, now that I think of it, since you'll have roughly identical activity on each machine.

It sounds like you've abstracted away some of the details that make your case a special case that might be easier, so you might use the Cluster Copy Files module to ship off the updated configuration files after any changes and RPC calls to restart Apache. If you don't have to deal with a different IP on each machine, then a simple copy will do the trick. I think, for a general solution, we'll have to address the different IP problem--so the above direct Apache configuration stuff will have to happen.

Hi Tony,

I second Joe's suggestion about doing this as a Virtualmin plugin. Basically, it could be done by writing a plugin that adds a new feature, which when enabled would copy the Apache config for the virtual server to all hosts in the cluster.

This copy could be done with scp, or with Webmin's RPC call feature to the Apache module on the remote systems. You can find documentation on writing a plugin at http://www.webmin.com/modules-virtualmin.html .. for your case, you would only need to implement the feature_name, feature_label, feature_setup, feature_delete and feature_modify functions.

Let me know if you have any questions about plugin development..

Thank you for the replies. This is likely to be simpler than I thought, thankfully. If all I have to deal with is replicating the configs using the built-in API, then it should be dead simple. The problem I foresee arising is the fact that Virtualmin pretty much insists on creating virtual interfaces locally, or that an IP address that's used in a virtual host actually exist (which usually makes sense, but here not so much).

Not sure how to get over that hurdle, unless, as I said, I create an ifconfig wrapper script that basically lies to Webmin. :P

Oh, and to deal with the "on change" situation, Joe, if I were to use rsync, I could just rsync the config files on apply, or on change, couldn't I?

There is actually an option on the Module Config page that tells Virtualmin not to check or create interfaces itself - instead, it will assume that you are going to set them up manually. It is called 'Bring up virtual interfaces?' under the 'Other server settings' section..

Ah, thanks for that. Somehow I'd overlooked it!

(Now the problem is that I need them created, just no locally...but that can be handled from the module!)

Also, we both really ought to learn to stop working. ;)

Wow, I guess it's been a while since I've done this:

manager# perl index.cgi
Undefined subroutine &main::init_config called at ./apache-cluster.pl line 5.
Compilation failed in require at index.cgi line 6.

manager# head index.cgi
#!/usr/bin/perl

# index.cgi
# Main configuration page for the Apache Clustering Module.

require './apache-cluster.pl'; # Import our module functions.

&header($text{'module_title'}, ""); # Display our title and all headings.

print "n";
manager# head apache-cluster.pl
# apache-cluster.lib
# Common functions for the Virtualmin Apache Clustering Module

do '../web-lib.pl';
&init_config();

any ideas?