Mailman configuration with multiple virtual server

27 posts / 0 new
Last post
#1 Sun, 03/23/2008 - 18:05
jjwalker

Mailman configuration with multiple virtual server

Joe,

I'm setting up a Joyent dedicated accelerator and need to understand how Mailman works with virtual servers under VM50. You posted this note in a reply to a separate query.

This will not effect your other servers. Mailman is wholly independent of almost everything else in the system.

Does this mean I should install Mailman at the top level (santafecomplex.joyent.us) rather than in an individual (or each) virtual server? can you point me to info on how to make Mailman available to the virtual servers? (I say top level because it appears the virtual servers are created within that server.)

A related qxn: Virtualmin installed via Joyent's upgrade. Mailman exists as a group & a user but won't configure because the program "/home/mailman/bin/mailman was not found on your system." There is no such folder in /home. I presume I create it through the Webmin users & groups panel in the server identified in your response to the above qxn. I then copy the Mailman tar over, expand it and run the config?

Thanks, Joe.

Don

Mon, 03/24/2008 - 11:47
Joe
Joe's picture

Good question. I don't actually know a lot about running mailman on Solaris. It actually seems a little strange that the module would default to finding it in /home/mailman...but Jamie knows solaris a lot better than I do. I've asked him to chime in on this thread.

--

Check out the forum guidelines!

Mon, 03/24/2008 - 12:56 (Reply to #2)
Joe
Joe's picture

Hi Don,

Virtualmin expects Mailman to be installed only once, is some global location like /home/mailman. Each domain with the mailman feature enabled can then have lists associated with it, each of which gets postfix virtual and alias file entries, created automatically by Virtualmin.

Assuming you Mailman is installed on your accelerator, you may need to adjust the paths that Virtualmin is using to try to find it. This can be done at Webmin -> Servers -> Virtualmin Mailman Mailing Lists -> Module Config.

--

Check out the forum guidelines!

Tue, 03/25/2008 - 08:29 (Reply to #3)
jjwalker

Thanks.

Does that rule apply to all or most Virtual min packages?

Tue, 03/25/2008 - 08:32 (Reply to #4)
Joe
Joe's picture

Yes, most of the apps Virtualmin manages like Apache, BIND, Postfix and MySQL are installed just once globally.

The exception is scripts like Wordpress and Drupal, which are separately installed into each domain.

--

Check out the forum guidelines!

Tue, 03/25/2008 - 18:36 (Reply to #5)
backspaces

<b>JamieCameron wrote:</b>
<div class='quote'>Hi Don,

Virtualmin expects Mailman to be installed only once, is some global location like /home/mailman. Each domain with the mailman feature enabled can then have lists associated with it, each of which gets postfix virtual and alias file entries, created automatically by Virtualmin.

Assuming you Mailman is installed on your accelerator, you may need to adjust the paths that Virtualmin is using to try to find it. This can be done at Webmin -&gt; Servers -&gt; Virtualmin Mailman Mailing Lists -&gt; Module Config.</div>
Hi. I'm working with Don on this issue as well. We're using the Pro upgrade on Joyent.

Sorry to be such a noobie, but:
- I thought there was a mailman install script/plugin for virtualmin pro. Is that correct? I found a reference to it while searching a while back.
- If so, is this designed to actually download and install mailman? .. or is it more of an administration interface to an already installed mailmain .. installed using the Solaris pkg system?
- Just how are we to know what directories we need to create? And how do we keep Virtualmin and the Solaris sys admin in sync in terms of where the directory is?

Basically we need instructions on how to use VMP on Joyent to make Mailman happen!

Tue, 03/25/2008 - 19:24 (Reply to #6)
Joe
Joe's picture

<div class='quote'>- I thought there was a mailman install script/plugin for virtualmin pro. Is that correct? I found a reference to it while searching a while back.</div>

No and yes. There is a Mailman plugin, but there is not a Mailman Install Script. This is a big difference, and it comes down to how the tool is used. Mailman, because it requires administrative privileges to work (you need entries in the virtual maps file), cannot be an Install Script. It has to have a plugin that has administrative privileges.

<div class='quote'>- If so, is this designed to actually download and install mailman? .. or is it more of an administration interface to an already installed mailmain .. installed using the Solaris pkg system?</div>

The latter. Mailman needs to exist on the system for Virtualmin to manage it. I kinda thought it was already on Joyent systems...but then again, they intentionally provide a minimal system, to leave lots of room for you to work. Since Mailman isn't all that commonly used, I suppose it's left out of the default install. If you don't already have mailman installed, you can get it from http://ftp.gnu.org/gnu/mailman/

Installation is documented here:

http://www.gnu.org/software/mailman/mailman-install/index.html

I kinda figured there'd be a Blastwave package of Mailman, but I don't see one. Since there isn't, we'll need to build one. Unfortunately, I have no clue how to build a Solaris package! But, I'll figure it out.

<div class='quote'>- Just how are we to know what directories we need to create? And how do we keep Virtualmin and the Solaris sys admin in sync in terms of where the directory is?</div>

Once installed, it doesn't move. There's nothing to sync. ;-)

The installation is pretty well-documented at the Mailman site, though, doing it without a package is a pain in the ass that we don't plan to ask people to deal with--so we'll build a package for Mailman. I didn't realize there wasn't one already. But it'll take at least a few days, as I don't even have a Solaris build system here (Jamie built the Solaris version of the installer, and Joyent have their own tools for their particular installation which is independent of ours).

--

Check out the forum guidelines!

Tue, 03/25/2008 - 19:32 (Reply to #7)
backspaces

Thanks, great info for us!

A couple of notes:
- Joyent must have an install script of some sort, because they use mailman on the shared accounts. Possibly they could be convinced to make it public?
- Possibly a different mail list manager would be easier, at least to start with? We use mailman only because we used it on HostGo earlier and on my shared Joyent account, so we know it fairly well, at least at the web interface level. It also has some commandline tools.

Thanks again!

Tue, 03/25/2008 - 19:41 (Reply to #8)
backspaces

Quick note: The Joyent forum has this thread on package management:
http://discuss.joyent.com/viewtopic.php?id=16230

Wed, 03/26/2008 - 05:39 (Reply to #9)
backspaces

[Sorry for all the messages, the &quot;edit&quot; function for the forum gave me errors.]

We may need to re-evaluate our strategy from the ground up. We seem to be facing several problems that we can simplify:

- Would it help a whole lot if we were not using Solaris? Even though Joyent/Solaris are wonderful, we may be getting in over our head. Our hope was that Joyent dedicated + VMP would give us the ease of their shared system, but with the power of the scalable accelerators. Linux would be OK if we could find a vendor with Joyent's sophistication. I don't think of Linux being superior in any way, but simply more people are familiar with it, thus likely it has better community support such as a mailmain install via apt-get or similar.

- Would it help to have BOTH a shared and a dedicated system? We could split our web/computation model by getting a Joyent shared for several services that are better handled by their managed scheme, primarily DNS and mail but possibly other global facilities that are non-trivial to manage by us/VMP. Then use the dedicated for things that VMP handles nicely for us. We could even have the shared by Joyent/Solaris and the dedicated be Linux if that helped.

I just wanted to think at a higher level if necessary.

Wed, 03/26/2008 - 09:42 (Reply to #10)
Joe
Joe's picture

You should be able to get everything working on Joyent systems, as they use Virtualmin extensively. You may need to contact them about solaris-specific package management issues though, as Virtualmin is much better tested on Linux (specifically CentOS, Debian and Ubuntu). And those operating systems provide mailman packages whose locations Virtualmin already knows about.

I'd be interested to know what kind of site architecture you are trying to setup there, as that would help me to make suggestions about the types of servers to use.

--

Check out the forum guidelines!

Wed, 03/26/2008 - 10:26 (Reply to #11)
backspaces

<b>JamieCameron wrote:</b>
<div class='quote'> You should be able to get everything working on Joyent systems, as they use Virtualmin extensively. You may need to contact them about solaris-specific package management issues though, as Virtualmin is much better tested on Linux (specifically CentOS, Debian and Ubuntu). And those operating systems provide mailman packages whose locations Virtualmin already knows about.</div>
Don will be getting back to you all on this with more details, but briefly: he bought a hour's system administration from Joyent for them to install mailman for us. I'd like whatever they learn to be shared with you folks too. We'll let you know what they found out, and how they configure mailman. Maybe they'll make a pkg available?
<div class='quote'>I'd be interested to know what kind of site architecture you are trying to setup there, as that would help me to make suggestions about the types of servers to use.</div>
Basically we're building a &quot;community&quot; site consisting of &quot;members&quot; and &quot;projects&quot;. The community is composed of members, &quot;Digerati&quot; living in Santa Fe, NM: scientists, urban studies experts, visual artists, ethnographers, analytic journalists and so on .. mainly tied together via ideas of cross-discipline studies, and inspired by the Santa Fe Institute .. the birth place of Complex Adaptive Systems 20 years ago. The projects are specific design efforts by the members .. i.e. projects are made up of subsets of the total set of members.

So in terms of site architecture, we will need the usual content management for the organization itself (front pages, newspaper/blog, newsletter, howtos, about, FAQ), along with sub-sites for members and projects. Our site design/sociology is not firm yet, but getting there. We may, for example, use sub-domains for the projects and just web/wiki pages for the members.

In addition, some projects need the ability to install open source software that is not in the virtualmin scripts. One example is Pootle, a translations system that lets communities translate existing sites into several languages. We have one project now which wants to translate the NetLogo simulation software, for example, as part of a project with Cuban scientists.

So our task is to create a mapping between our community structure and virtualmin facilities. And to provide help for projects needing to install and create new software. Indeed, one reason we chose Joyent was its programmer-friendly services: svn, RoR, Django and so on. We decided on dedicated rather than shared hosting due to root access for installing unusual packages and for the increased performance.

Here's a pointer to a wiki containing an example of the projects: http://sfacc.pbwiki.com/ and here's a pointer to our current, under construction web site: http://www.santafecomplex.org/

Wed, 03/26/2008 - 13:21 (Reply to #12)
backspaces

Oh, forgot to mention: One of our most important initial tasks is to make a master email list with all the members of our organization. Then make several &quot;sub-lists&quot; of that .. at least one list for each project, and around a dozen lists for the organization: admin, content management, advisory board, .. and so on.

We'll likely try to build a mail list manager front end to automate it. Maybe have a page of users and mail lists: select a user, select the lists the user wishes to be in. http://www.airset.com/ is a good example of people/groups/community management.

This is why mailman is so important. BTW: It looks like Joyent did a GREAT job solving the problem, both in building a pkg for it and integrating it into VMP. More later, after we get a chance to try it out.

I'm not sure if this would be a good way for us to start writing our own VM interfaces/forms .. maybe?

Wed, 03/26/2008 - 19:43 (Reply to #13)
Joe
Joe's picture

Glad you got the Mailman issue sorted out. It actually has a UI for managing lists, and a web interface that allows people to select which lists they subscribe to. I'd recommend making use of this as much as you can, as writing your own list management software would be a lot of work.

If you want to install Pootle into a large number of Virtual domains, you may want to look into writing your own script installer. This is documented at :
http://www.virtualmin.com/component/option,com_openwiki/Itemid,48/id,scr...

--

Check out the forum guidelines!

Thu, 03/27/2008 - 03:55 (Reply to #14)
jjwalker

Jamie,

Joyent used a package they have for the install and I am know creating mailing lists. I haven't checked them for send/receive yet but we're getting closer.

I have an immediate question for you. I couldn't access the lists through lists.sfcomplex.org, which I creted as a subdomain for Virtualmin's list manager. I modified the mailing list module to use the MailMan default wrapper and I can admin the lists through VirtualMin but not through the web. Joyent wrote this:

<div class='quote'>The lists will work on a 'lists' subdomain of each virtual server (e.g. if you create a list on 'sfcomplex.org', it will be configured to receive at LISTNAME@lists.sfcomplex.org, with the admin interface at http://lists.sfcomplex.org/mailman/admin/LISTNAME). To make a such list work, you should point the hostname of lists.sfcomplex.org to your Accelerator, and it will automatically be routed through Postfix to Mailman (using the /var/db/postfix-to-mailman.py transport script).
</div>

I assume I need to reset the module to the original custom URL for Manage button but I didn't write it down. Do you recall it off the top of your head? Also, when Joyent talks about pointing the hostname, is this a BIND step. (I'm displaying my inexperience here.) Do I add lists.sfcomplex.org to the local zone?

Thanks for all the help from you and Joe.

Don

Sun, 06/07/2009 - 07:20 (Reply to #15)
Joe
Joe's picture

Joyent's Mailman setup is actually a little different from the standard used by regular Virtualmin installs - they use that special lists. sub-domain, which you'd have to add at Services -&gt; DNS Domain in Virtualmin.

I'm not sure what their special manage URL was set to, but I guess something like http://lists.${DOM}/mailman/admin

--

Check out the forum guidelines!

Sun, 06/07/2009 - 07:20 (Reply to #16)
backspaces

<div class='quote'> Joyent's Mailman setup is actually a little different from the standard used by regular Virtualmin installs - they use that special lists. sub-domain, which you'd have to add at Services -&gt; DNS Domain in Virtualmin.

I'm not sure what their special manage URL was set to, but I guess something like http://lists.${DOM}/mailman/admin
</div>
I hope I'm not being an idiot here but I can see several way to achieve a special subdomain of this nature.

1 - Simply configure this with apache's httpd.conf or with .htaccess in our top level.
2 - Create an entirely new virtual server, with the subdomain lists.sfcomplex.org. I believe this would create a new file hierarchy within domains/lists.sfcomplex.org.
3 - Create a new dns entry which has an A record for lists.sfcomplex.org and have it resolve to a port number managed by mailman.

.. the list goes on! So basically I'm not clear what the sub-domain means in terms of Solaris/Joyent/Virtualmin interactions. Administration conventions, I guess.

One specific puzzle for me is the DNS management. If we create a true lists.sfcomplex.org subdomain, does it use Joyent's name server? Our BIND server on our Accelerator? Or are we supposed to somehow manage our own name service externally to Joyent (GoDaddy is our registrar as well as name server now)?

I just bought both webmin books you and Joe wrote, as well as downloaded their .pdfs, so I guess I'll eventually get it!

Keep up the good work, btw, we're REALLY grateful for your product and help!

Sun, 06/07/2009 - 07:20 (Reply to #17)
backspaces

<div class='quote'> Joyent's Mailman setup is actually a little different from the standard used by regular Virtualmin installs - they use that special lists. sub-domain, which you'd have to add at Services -&gt; DNS Domain in Virtualmin.

I'm not sure what their special manage URL was set to, but I guess something like http://lists.${DOM}/mailman/admin
</div>
I hope I'm not being an idiot here but I can see several way to achieve a special subdomain of this nature.

1 - Simply configure this with apache's httpd.conf or with .htaccess in our top level.
2 - Create an entirely new virtual server, with the subdomain lists.sfcomplex.org. I believe this would create a new file hierarchy within domains/lists.sfcomplex.org.
3 - Create a new dns entry which has an A record for lists.sfcomplex.org and have it resolve to a port number managed by mailman.

.. the list goes on! So basically I'm not clear what the sub-domain means in terms of Solaris/Joyent/Virtualmin interactions. Administration conventions, I guess.

One specific puzzle for me is the DNS management. If we create a true lists.sfcomplex.org subdomain, does it use Joyent's name server? Our BIND server on our Accelerator? Or are we supposed to somehow manage our own name service externally to Joyent (GoDaddy is our registrar as well as name server now)?

I just bought both webmin books you and Joe wrote, as well as downloaded their .pdfs, so I guess I'll eventually get it!

Keep up the good work, btw, we're REALLY grateful for your product and help!

Thu, 03/27/2008 - 13:38 (Reply to #18)
backspaces

<div class='quote'>Are you hosting your own DNS domain there, or is joyent or someone else managing the DNS domain sfcomplex.org for you? It is on the DNS server that you have to add the lists sub-domain entry.
</div>
Don is handling the Domain names, but I believe I understand the setup:

- GoDaddy is the Registrar. It is also the Name Server.
- Looking at whois, two name servers are registered:
NS53.DOMAINCONTROL.COM
NS54.DOMAINCONTROL.COM
- Don set up the A record for sfcomplex.org to point to the accelerator (dedicated host) IP: 8.17.171.158
- I am not clear on the rest of the DNS management. For example, when Virtualmin sets up new services like a sub-server, does it manage the name using the local BIND? I suspect GoDaddy has a wildcard for *.sfcomplex.org. pointing to the same IP address as sfcomplex.org does.

To be clear, I'm puzzled by the entire subdomain management within Virtualmin. It can be simply a .htaccess or httpd.conf configuration. Or it could be that combined with a true DNS entry as well.

And its possible that the customizations Joyent make for the Pro upgrade include use of their 4 nameservers, dns1-4.joyentdns.com.

Thu, 03/27/2008 - 14:20 (Reply to #19)
Joe
Joe's picture

So in your case, it looks like your sfcomplex.org DNS domain is being hosted by domaincontrol.com, so Virtualmin cannot manage it. I presume they have some web interface for managing DNS records - you need to add one for lists.sfcomplex.org with the IP 8.17.171.158

--

Check out the forum guidelines!

Thu, 03/27/2008 - 14:57 (Reply to #20)
backspaces

<div class='quote'>So in your case, it looks like your sfcomplex.org DNS domain is being hosted by domaincontrol.com, so Virtualmin cannot manage it. I presume they have some web interface for managing DNS records - you need to add one for lists.sfcomplex.org with the IP 8.17.171.158</div>
What if there is a wildcard *.sfcomplex.org setup? Wouldn't that be equivalent?

How is the lists.sfcomplex.org subdomain *used* in terms of mailman access?
- Is it just apache figuring out how to manage the http request? .. i.e. sending it to the mailman administration pages?
- Is it actually using a local subdomain account in some way? I.e. Is /home/sfcomplex/domains/lists.sfcomplex.org or some such set up somehow to manage the requests?

BTW: It looks like there *is* a problem in terms of the DNS .. I can ping sfcomplex.org but not lists.sfcomplex.org.
[code:1]owen|~[551]: ping sfcomplex.org
PING sfcomplex.org (8.17.171.158): 56 data bytes
64 bytes from 8.17.171.158: icmp_seq=0 ttl=244 time=74.897 ms
^C
--- sfcomplex.org ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 74.897/74.897/74.897/0.000 ms

owen|~[552]: ping lists.sfcomplex.org
ping: cannot resolve lists.sfcomplex.org: Unknown host
[/code:1]

.. so would installing a DNS wildcard A record at GoDaddy/domaincontrol.com fix everything?

You can tell I still don't &quot;get it&quot; when it comes to the interplay between Joyent, Virtualmin, Solaris and the DNS world from the local BIND on my Accelerator to how subdomains are managed.

If I had to guess, I'd say we need the wild card at GoDaddy, and from there on mailman would be managed by a &quot;virtual domain&quot; within Apache, right?

Thu, 03/27/2008 - 15:22 (Reply to #21)
backspaces

At a guess, the &quot;bug&quot; here is that Joyent should include their 4 nameservers for each dedicated host, just as they do for their shared hosting.

My shared hosting does not have this problem, and subdomains are handled in two ways: one via Apache instituting the http://wiki.foo.com -&gt; http://foo.com/wiki convention; the second by using the domains/wiki.foo.com sub virtual server configuration.

If Virtualmin is having external dependencies of this sort, Joyent should help in one of several ways. Which I won't enumerate, I've been way to verbose already!

Fri, 04/04/2008 - 15:47 (Reply to #22)
backspaces

BTW: Just a quick update about Joyent and Mailman.

Joyent has recently moved its dedicated hosting to a pkgsrc template, as they call it. (Shared apparently was already there)

This means a lot more software is now available. They make many of these packages available on the new dedicated systems, and have binaries precompiled from the pkgsrc source based installs. So now mailman is indeed available, although the configuration apparently is err.. difficult. Here's their current list:
http://pkgsrc.joyent.com/2007Q4/

As I understand it, the pkgsrc (netBSD based) template is good news from an availability standpoint, much of the Linux &amp; BSD software now is available for OpenSolaris.

Thu, 03/27/2008 - 09:35
jjwalker

The special manage url looks close enough. I'll give it a try. I don't find a Virtuamin Service-DNS Domain. Could it be the Webmin BIND or some such place?

Thu, 03/27/2008 - 13:07
Joe
Joe's picture

Are you hosting your own DNS domain there, or is joyent or someone else managing the DNS domain sfcomplex.org for you? It is on the DNS server that you have to add the lists sub-domain entry.

--

Check out the forum guidelines!

Thu, 03/27/2008 - 15:16
jjwalker

I set the wildcard at GoDaddy to our IP address. The name lists.sfcomplex.org now resolves to www.sfcomplex.org. Progress but not quite there.

Perhaps we have a misunderstanding. Should we be able to access the mailing lists through a web interface outside of Virtualmin?

Also, you say Vrtualmin cannot manage GoDaddy registrations. Can it manage other registrars?

Thu, 03/27/2008 - 19:56 (Reply to #26)
Joe
Joe's picture

I think you will have to talk to Joyent about getting the lists.sfcomplex.org domain working, as this is their special way of setting up Mailman. It should allow you to access Mailman from outside of Virtualmin though.

As for registrars, Virtualmin can currently only register domains with Register.com.

--

Check out the forum guidelines!