Apache MPM-ITK for Virtualmin

25 posts / 0 new
Last post
#1 Wed, 08/17/2011 - 02:51
wipeout

Apache MPM-ITK for Virtualmin

Hi,

I know there is one other thread on MPM-ITK on these forums but the last post was over a year ago so I thought I would start a new one..

In my current VPS that I manually configure I use MPM-ITK and I have to say I have been very impressed with it and how easy it makes things.. Sites run as the configured user and group, I can be sure that one site cant access anothers files, its great.. Performance seems very good as well.. Its much easier than the common suEXEC/suPHP combination because you just switch Apache to it and add one line to the vhost config..

I have tested MPM-ITK with php-cgi and mod_php (not php-fcgid yet because I have found mod_php to be extremely fast for the Drupal sites I run).. I have also read somewhere that it plays very nicely with PHP-FPM which is included in PHP 5.3.3 and above..

It was disappointing to see that Virtualmin, which has so much flexibility, doesn't support this option out of the box..

With the new ways of working coming down the line Virtualmin is going to have to offer the options.. MPM-ITK and PHP-FPM are absolute MUSTs for integration..

Wed, 08/17/2011 - 09:44
andreychek

Howdy,

Well, out of the box, Virtualmin offers a suexec-based setup that allows you to execute all of your PHP scripts as the Virtual Server admin. That works in the FCGID or CGI PHP Execution modes that are set in Server Configuration -> Website Options.

So I suppose the question would be, is there a benefit to using MPM-ITK over the current suexec-ased setup?

Since it's based on the Prefork MPM, I'm not sure there would be much of a performance difference. But determining that for certain may require some benchmarking.

As far as PHP-FPM goes, that is something we've been reviewing.

-Eric

Wed, 08/17/2011 - 11:25
wipeout

Hi,

From what I have read there is a performance advantage to MPM-ITK over suEXEC.. Would have tofind all the tests again but ever servers load profile is different..

An advantage that I have found to the MPM-ITK setup is the ability to use mod_php easily still with running as the virtual server owner because the user is being handled at the Apache level so everything from there runs as the user.. Another advantage is the ability to use APC as designed and have a shared cache is..

APC can be made to work under mod_fcgid apparently but I don't believe the cache is shared so if the cache was 30M it would need 30M per virtual server, hence the reason shared servers almost never run APC..

Since my sites are all Drupal, meaning heavy on PHP and not so much in the way of static content APC makes a HUGE performance difference.. Under MPM-ITK and mod-cgi (slowest way to run PHP from my testing) page generation times were around 200ms.. Under MPM-ITK with mod_php they dropped to about 100ms and with APC enabled they are down to 20ms.. So about 10 times faster.. All while dropping CPU load enormously.. Obviously thats my particular load requirement..

Granted I didn't test with a suExec/fcgid setup but if I had APC would have used up too much memory..

Guess all I am saying is that it would be nice to have all available options in virtualmin so users can choose what's best from their particular requirements.. I believe Virtualmin is the ONLY web control panel that would be able to achieve this level of flexibility..

With PHP_FPM the APC issue may be resolved so that coupled with MPM-ITK may be a very good setup..

Wed, 08/17/2011 - 15:49
staticanime

I would also be interested in seeing support in Virtualmin for "alternative" versions of Apache, would make it a much more useful tool

Thu, 08/18/2011 - 14:49
andreychek

I would also be interested in seeing support in Virtualmin for "alternative" versions of Apache, would make it a much more useful tool

That is something that's being worked on, adding support for web servers other than Apache (and in particular, those providing a small memory footprint by default).

-Eric

Thu, 08/18/2011 - 16:12
wipeout

That sounds interesting.. Is nginx on the list? Not something I have played with yet but have heard goof things.. :)

Sun, 08/21/2011 - 05:22
wipeout

Just for completeness I thought I would post results of my additional testing..

MPM-ITK and fcgid don't appear to play nicely together.. Also since the processes are shutdown by MPM-ITK after completing their task there probably wouldn't be much benefit anyway..

MPM-ITK is a simple setup to have virtual servers run as their owners and plays nicely with mod_php or php-cgi..

As MPM-ITK isn't a standard setup option for virtualmin I couldn't do much testing on the virtualmin server but keep reading..

If you want out and out performance then creating and destroying processes if not a great idea if you can help it..

The fastest solution I have now tested (and I mean significantly faster) is MPM-worker + suEXEC + fcgid.. This is very messy to setup manually as I was doing but fortunately if using Virtualmin a lot of this is done for you so its easy..

Unfortunately I wan't able to test Virtualmin with MPM-worker (default is MPM-Prefork in order to support mod_php) because when trying to switch my server wants to remove Virtualmin.. It appears that the current Virtualmin build has a dependency of mod_php which wants to be removed when installing mpm-worker.. Were this dependency no there the switch would be very easy..

So in conclusion the MPM-ITK is a great setup for a low memory system thats simple to configure so that each virtual server runs as its owner the suEXEC + fcgid setup offered by Virtualmin is very quick.. Even more so if MPM-Worker could be used (and you don't need the ability to use mod_php)..

Mon, 08/22/2011 - 23:58
helpmin

Is mpm-itk faster than MPM-worker? mpm-itk is basically the old prefork (process based), while mpm-is thread based. Linux can create new threads much faster than processes.

Tue, 08/23/2011 - 07:11 (Reply to #8)
wipeout

No, MPM-Worker is the fastest but is not recommended for use with mod_php..

MPM-prefork and MPM-ITK are about the same but with MPM-ITK there is no need for suEXEC or suPHP in order to run scripts as the script owner..

MPM-ITK is easier to setup to run as different users but since Virtualmin handles all that its not much of an issue..

Tue, 11/22/2011 - 13:25
wizonesolutions

PHP-FPM has been talked about for a while (http://www.virtualmin.com/node/11831, http://www.virtualmin.com/node/9169, http://www.virtualmin.com/node/19216, http://www.virtualmin.com/node/17258) and I know you guys are planning it and considering it. Just wondering if a date (or at least a release) has been assigned to it. Loving PHP-FPM on the sites where I'm using it but don't necessarily want to give up the ease of spinning up new sites in Virtualmin. I can reconfigure them but often it's not worth it, though being able to configure them to use PHP-FPM by default would definitely be worth it.

Tue, 11/22/2011 - 14:20 (Reply to #10)
andreychek

Well, we've been a bit reluctant to add it, due to the additional support issues it adds -- especially regarding Install Scripts, which would all need to be tested under the additional PHP Execution Mode.

But support for PHP-FPM at some point hasn't been ruled out.

I'd love to hear about how you're using it, as well as how you went about setting it up on your Virtualmin server.

Would you care to elaborate on your setup, how you got things working, and how it compares to other solutions?

Thanks!

-Eric

Fri, 12/02/2011 - 00:08 (Reply to #11)
wizonesolutions

Finally getting back to you! I understand what you're saying...but I really think that it's the way of the future, especially with the growing popularity of nginx as a Web server (just one example).

I'm using it to run some CMS-based sites (WordPress and Drupal more or less). I like it because I don't wind up with 5 php-cgi processes per site...PHP-FPM has its own server model and shares the processing load among a few "servers" (the terminology is similar to Apache). It also works well with APC.

I just set it up directly in the vhost configuration files; I don't use Virtualmin to add a server or sub-server when using PHP-FPM, for obvious reasons. Other than that, I just installed it, and it doesn't conflict with anything Virtualmin needs. So they coexist alright.

Don't have time to post a config snippet at the moment, but if you'd like one just reply and I'll example.com-ify it and pastebin it and link to it.

Fri, 12/02/2011 - 08:51
andreychek

I'd love to see what you add to the Apache config in order to use PHP-FPM. If you happen to have time for that in the future, that'd be great :-)

-Eric

Fri, 12/16/2011 - 19:05
mbakop

Hi,

@wizonesolutions could you please provide the config snippet or an how-to of modifying the apache config to use PHP-FPM? I see Eric requested it above so consider this a +1 :)

Thanks Ashish

Fri, 12/16/2011 - 19:13
wizonesolutions

I haven't forgotten. I'll try to post one by next week at least.

I don't know if mine is optimized, though, but it'll at least be a starting point.

Mon, 02/06/2012 - 06:49
nosco

@wizonesolutions POST IT.... Thanks !!!

i would like to see this dream come true ... ;)

Sat, 02/11/2012 - 11:23
LeonB

Support for PHP-FPM (and nginx?) would be awesome!

Wed, 03/21/2012 - 21:14
wizonesolutions

I'm finally posting this. Sorry for the looooong delay. You know how it is (probably).

This configuration is not pristine, and I've changed stuff from the default. I'll try to explain it a bit, and just post back questions if you don't get something.

It's been sanitized to example.com, obviously. My real one uses an Include directive for the directory options so that they're standard across virtual hosts, but that's nothing specific to PHP-FPM.

Also, this is on Ubuntu Lucid 10.04. I believe the latest versions automatically separate the pool and main configuration. Note that I've done this in mine; when you install by default, the default pool (www) is setup in php5-fpm.conf.

In addition, I've set it up to use Unix sockets. It was configured to use TCP networking by default.

Finally, this whole setup requires an Ubuntu PPA if you are on Lucid. I think that it is included in later versions of Ubuntu.

I recommend setting up your Virtualmin sites to use Apache-FCGId for now; I converted VirtualHost definitions thus configured to what I have now.

Right...so let's try to explain this from step 1...and again, this assumes Ubuntu 10.04.4.

Setting this up won't mess up your sites that run in Apache-FCGId mode, by the way. I recommend against this since you'll be running more processes that way and that sort of defeats the purpose of this setup...but to you.

Another thing I noticed is that each PHP-FPM pool seems to use a bit more memory than having Apache-FCGId sites running under the same amount of users. Not sure why this is, though it probably has to do with the php5-fpm process size. The thing I like is that the PHP processes are managed better, APC WORKS (which can further reduce process size if the entire codebase can be cached), and thus memory usage is more predictable - it will spike and then come back down rather than staying up.

Paths in here refer to paths on my system.

Step 1: Install PHP-FPM

sudo apt-add-repository ppa:brianmercer/php
sudo apt-get update

Step 2: Install stuff

  1. Make sure the multiverse repository is enabled your /etc/apt/sources.list!
  2. Run: sudo apt-get install php5-fpm libapache2-mod-fastcgi
    sudo a2enmod fastcgi
    sudo service apache2 reload

Step 3: Change Apache and VirtualHost configurations! This is where you start looking at my attached files. Brief rundown:

  • /etc/php5/fpm/php5-fpm.conf: I deleted the [www] pool configuration. I turned on the include directory for pool configurations (a pool is basically an instance of PHP-FPM running as a particular user and configured to run in a certain way. I use several of these to segregate access and to allow PHP to run as the virtual server user. Virtualmin would ideally create a pool per Virtual Server (and reuse them for sub-servers)).
  • /etc/php5/fpm/example-pool.conf: I turned on Unix socket use instead of TCP networking (about 10% faster, they say) and I changed the variables related to the listening and running user and group (this is recommend around the net) and reduced the amount of servers running since I was on a VPS. I found that the defaults work well though on a machine with 2 GB RAM or a bit more...depending on what you're doing. This machine was a Web + database server (though it wasn't the one this configuration is from).
  • /etc/apache2/apache2.conf: I added the FastCgiExternalServer directive.

Step 4: Restart stuff

sudo service apache2 restart
sudo php5-fpm restart

That's about it. I've tried to be as accurate as possible, but might've made a couple typos. So let me know! Glad I finally got this posted!

Thu, 03/22/2012 - 14:57
wipeout

@wizonesolutions - Can you post your apache conf directives to get mod_fastcgi to talk to the php-fpm pool?

I have been running nginx with php-fpm for a few months and its a great way to run php.. However there are some things to be aware of..

1) If you want to run scripts as the site owner, as wizonesolutions mentioned, you will need to create a new pool per site.. With the static and dynamic governors this will keep at least one php-fpm process running for each site all the time.. Good news for performance because you don't have to spawn one but bad news for memory consumption.. The solution is the new "ondemand" governor that will close down all the processes and start a new ones when needed for that particular site.. This means that php-fpm will now be usable in a shared hosting environment..

2) Yes, APC does work with PHP-FPM.. Just remember it creates a new cache per pool so if you have a 32MB cache its 32MB per pool of memory you will need.. If using the ondemand governor the cache would be destroyed when the last of the processes was closed so for sites with sporadic visits its probably not worth using it..

Thu, 03/22/2012 - 18:25 (Reply to #19)
wizonesolutions

Re. directives, they're in the apache2.conf I posted. FastCgiExternalServer is all I needed. I used the default mod_fastcgi.load if you meant module configuration.

1) Ooh, "ondemand"? Never heard of it, though it sounds interesting. Do you have a link?

2) Yeah, although if using that governor then the user probably can't install APC anyway or at least wouldn't think of it. My conf is on a self-managed VPS that hosts my own sites, so I don't have to worry about unpredictable scaling too much...I know what I'm dealing with. Good notes on APC - that's probably why my memory usage is a bit higher...I set my cache 128 MB or something since I'm running Drupal.

Thanks for the feedback.

Thu, 03/22/2012 - 21:32 (Reply to #20)
wipeout

The ondemand governor may already be in your php-fpm build..

Here is the best information I have found.. https://bugs.php.net/bug.php?id=52569

Tue, 08/07/2012 - 19:17
wizonesolutions

Hey Virtualmin folks - if you need an example of setting up PHP-FPM programmatically, check out TuxLite. It uses PHP-FPM exclusively. Maybe it will give you some inspiration.

(And if Virtualmin started supporting PHP-FPM and I missed it, let me know. That'd rock.)

Fri, 10/03/2014 - 17:44
aedifex Pro Licensee

Any update on PHP-FPM support in Virtualmin, Eric? I don't want to sound harsh but it's been a long time since you were reviewing it, does this mean you're not going to add it?

Patrick

Fri, 10/03/2014 - 23:54
andreychek

Unfortunately, for the time being, that's something that would need to be manually configured. Jamie doesn't have plans to add Virtualmin support for that in the near future.

-Eric

Mon, 10/06/2014 - 18:08
aedifex Pro Licensee

Ok, thanks for letting me know!

Patrick