Here I thought I'd outline just some of the problems I am having with Webmin at the moment, in the hope that they will actually be addressed by somebody, because I can't find anything online about them except for dead-end solutions for people who actually had the problem(s) I do!
1) Domain setup (which might affect quotas, bandwidth etc.) Basically, I wanted to host multiple virtual servers using Virtualmin. BUT I wanted to give each user a subdomain, attached to the main site address, e.g. user1.mainsite.tld. I didn't like the idea of using sub-servers in Virtualmin, because that would mean having a domains folder with separate directories. So instead I decided to make a NEW VIRTUAL SERVER for each subdomain. There is also a separate virtual server for the mainsite.tld. I have bought the mainsite.tld from GoDaddy.com, and I use CNAMEs to point each subdomain to my dedicated server's ip-address. Apache then does the rest with Named Virtual Hosts, and knows which user's home directory each subdomain points to.
2) Bandwidth Bandwidth for each domain is being incorrectly calculated, for some unknown reason. Webalizer is also way off the mark. The most interesting part of this bizarre fact, is that when I create the domain on a certain date, Virtualmin tells me (in the domain settings area) when I created it - which is normal, but the bandwidth says 'from <a date>' that is nearly a month before i even setup the virtual server!! For example, I recently built a server on 18 Feb 2008. But the bandwidth usage is reported 'since 23 Jan 2008'! I don't know if this is intentional, but it would be nice if it reported it from when i created the virtual server, or from a date that i specify. I know that it is possible to choose 'by date' etc, but that is no use really, because the bandwidth recorded was 146GB in one day (on the first day after I created the virtual server), way off the mark since I knew that this was impossible; the files transferred don't match this apart from the fact that when I checked the Apache logs, a download manager was used and split parts were made. I did try adding up the data, and it did seem to pan out to 196GB, but if so, this means that bandwidth is calculated according to the logs, so even if i start a 4GB file download, and stop it after, say, 1 second, the bandwidth usage will be 4GB?
3) Quotas / Ownership calculations This is a bizarre one, and I really don't know how this was overlooked (if it was). If a user (by user, I mean a subdomain user with their own home directory) accesses their site via FTP, and create a new folder or file, that folder/file will by default have the ownership and permissions of that user, correct? But let's say the user wants to transfer a bulk load of files remotely to their server, say a large zipped backup of a database, they will need to remotely transfer it using either PHP or a Perl script. In this case, Apache takes over, and when the file is transferred, it has the ownership and group of 'apache' (or whatever Apache is set to run as). The effect of this, it seems, is that Webmin only calculates the user's disk quota only by taking files with the user's group ownership into account. If the files are not owned by that user, then they are not taken into account. In light of this problem, I am thinking that perhaps I got a 146GB bandwidth reading because ownerships were being calculated from apache's user group and added onto the current domain users account - because I had to mount a directory to their home directory from outside...not sure.
Finally, I have to mention that some of these problems were most likely fixed, but I upgraded Webmin to the latest version (as opposed to a fresh install). Please help me if you can, and do tell me if I am the one making the mistakes (e.g the domain setup).
1) in VM under server templates, choose a template and go to Virtual server creation. I used this to create temporary subdomains if a user was waiting for his domain to propagate. (i use subservers now instead as they are more powerful)
2) bandwidth monitoring based on logfiles can never be accurate.
<div class='quote'>this means that bandwidth is calculated according to the logs, so even if i start a 4GB file download, and stop it after, say, 1 second, the bandwidth usage will be 4GB?</div> Yes
You can try stuff like vnstat http://humdi.net/vnstat/ or ntop http://www.ntop.org/
3) i have no good idea really but per haps, also in server templates, under Apache website, you can change the user there. Not sure if that is what you meant though....
1) I'd be pleased to know why you think sub-servers are more powerful? If I want to offer separate clients their own sub-domain of a site, e.g. user1.mainsite.tld, then it would get confusing for calculating their quota separately, wouldn't it? The other reason I didn't want to use sub-servers for subdomains is because I thought bandwidth would be included for the whole virtual server including sub-domains, and so I went with creating a separate virtual server for each client with a subdomain. However I found that the bandwidth isn't even working correctly - and I suspected it might be due to how i have decided to set up subdomains (e.g. as separate virtual servers). Is this the case?
2) Thanks, I will look at those, but i have since installed mod_cband on my server and it appears to be doing its job correctly. It would be nice if Webmin had an integrated section with mod_cband, so it would show bandwidth usage according to how mod_cband records it.
3) well the quota is only calculated according to file ownership. so if user1 uploads files via their ftp to their folder, they will have ownership of those files. when quota is calculated, those files will be taken into account.
But if apache or another process like a php script creates any files in/under their home directory, those files will have the ownership of the process that ran them; in my case, the files have the ownership of 'apache', and group 'apache', and when quota is calculated, it ignored these files, simply because they are not owned by the user in question. Of course, I can use SuExec with CGI scripts to correct this, but not PHP. And I am running PHP as an apache module, and i don't plan on running it as an executable.
Finally I have to mention that when i installed webmin, i had just requested to have Plesk removed from my server, so I thought maybe the incorrect quota/bandwidth calculations had something to do with incorrect configuration of Webmin, but I can't figure what.
Don't use sub-domains OR sub-servers for separate users! What you want is simply separate accounts. Virtualmin doesn't care what domains are named, it only cares about ownership.
This confusion is why the sub-domain account type is hidden by default. It convinces people that they need to use it for things that are named with a subdomain of any existing virtual server. Sub-servers (and sub-domains, if you choose to enable that particular abomination) are owned by the parent virtual server account. If you've got separate users, you don't want that at all. Not even a little bit.
Create a new virtual server for each of your users. Name it whatever you like. Be happy.
I'll try to parse out the rest of your questions next. But I wanted to get that out of the way really fast.
--
Check out the forum guidelines!
Thanks for the reply Joe!
I see you make the point about using separate accounts for each user, regarldess of their domain being sub-/domain. If you look at my post 1) above, that's what I have done - just in case you didn't notice. I'll assume you did notice it already, and just wanted to make this point absolutely clear!
In relation to 2), Bandwidth monitoring, I can only presume that your bandwidth monitoring has a bug based on what one of my clients told me - and it was pretty strange how the bandwidth was shooting up in gigabytes. Funny thing is I started adding the apache access logs for their domain and it seemed it would work out exactly as Virtualmin calculated (I didn't add them all though). So perhaps my client is telling me a white lie!
3) SuExec as far as I have checked does not work with PHP. It wasn't made entirely clear whether it only doesn't work with the modular setup of PHP, but might be ok with the binary setup of PHP.
SuExec is pre-installed on my Fedora server, so are you suggesting I re-setup SuExec with the correct docroot? Or do I have to set PHP to run as an executable?
Thanks for the help so far!
<div class='quote'>1) above, that's what I have done - just in case you didn't notice. I'll assume you did notice it already, and just wanted to make this point absolutely clear!</div>
Yes, I noted. Just wanted to make sure anyone reading this thread didn't find themselves thinking along those lines, since sub-domain and sub-server accounts were mentioned quite a bit. It's a common source of confusion, and I've yet to figure out how to make the Right Thing more obvious in this particular circumstance (a bunch of users with subdomain named accounts).
<div class='quote'>3) SuExec as far as I have checked does not work with PHP. It wasn't made entirely clear whether it only doesn't work with the modular setup of PHP, but might be ok with the binary setup of PHP.
SuExec is pre-installed on my Fedora server, so are you suggesting I re-setup SuExec with the correct docroot? Or do I have to set PHP to run as an executable?</div>
Of course suexec works with PHP. You can't use mod_php, but that's alright...mod_fcgid makes running it as CGI just as fast for most workloads. If you want reasonable security, you have to run things under suexec...and if you want quotas to work, and file ownership to be sane, and all sorts of things.
You have to rebuild Apache to have suexec_docroot set to /home, and you also have to create wrapper scripts for PHP that have the ownership of the virtual server user. As I mentioned, all of this is automated in Virtualmin Professional (and you don't have to rebuild Apache if you're on CentOS or Debian, since we provide packages already built with the appropriate options in our GPL repository). But for GPL on Fedora, you're completely on your own.
I've covered the steps for rebuilding Apache here in the forums in the past--it's not hard if you're familiar with rebuilding RPMs (and even if you aren't, if you're comfortable with the command line, yum, and a text editor, if you follow the steps I provided).
As for the php wrapper, I've posted discussion of that as well, but since I suspect it'll be harder to dig up than the Apache build instructions, here's an example of a wrapper that you'd put in /home/domain/fcgi-bin (you could put it in your skel if you want it to end up in all virtual server homes):
[code:1]
#!/bin/sh
PHPRC=$PWD/../etc/php5
export PHPRC
umask 022
PHP_FCGI_CHILDREN=4
export PHP_FCGI_CHILDREN
exec /usr/bin/php-cgi
[/code:1]
And in your Server Templates Apache section, you'd add something like this:
[code:1]
AddHandler fcgid-script .php
FCGIWrapper ${HOME}/fcgi-bin/php5.fcgi .php
[/code:1]
You also want to make sure the suexec option is turned on in Virtualmin (I think there's an option for it in GPL, anyway).
And, of course, mod_fcgid needs to be enabled.
See, it's quite complex, which is why none of this stuff is enabled or dealt with in a stock GPL installation--unless we are also able to provide a bunch of packages (as we do in Professional, and to some degree for CentOS and Debian for GPL, if you use the install.sh script) it'd just lead to a bunch of stuff that doesn't work. It takes a lot of components working together to get these things spinning smoothly.
--
Check out the forum guidelines!
<div class='quote'>1) above, that's what I have done - just in case you didn't notice. I'll assume you did notice it already, and just wanted to make this point absolutely clear!</div>
Yes, I noted. Just wanted to make sure anyone reading this thread didn't find themselves thinking along those lines, since sub-domain and sub-server accounts were mentioned quite a bit. It's a common source of confusion, and I've yet to figure out how to make the Right Thing more obvious in this particular circumstance (a bunch of users with subdomain named accounts).
<div class='quote'>3) SuExec as far as I have checked does not work with PHP. It wasn't made entirely clear whether it only doesn't work with the modular setup of PHP, but might be ok with the binary setup of PHP.
SuExec is pre-installed on my Fedora server, so are you suggesting I re-setup SuExec with the correct docroot? Or do I have to set PHP to run as an executable?</div>
Of course suexec works with PHP. You can't use mod_php, but that's alright...mod_fcgid makes running it as CGI just as fast for most workloads. If you want reasonable security, you have to run things under suexec...and if you want quotas to work, and file ownership to be sane, and all sorts of things.
You have to rebuild Apache to have suexec_docroot set to /home, and you also have to create wrapper scripts for PHP that have the ownership of the virtual server user. As I mentioned, all of this is automated in Virtualmin Professional (and you don't have to rebuild Apache if you're on CentOS or Debian, since we provide packages already built with the appropriate options in our GPL repository). But for GPL on Fedora, you're completely on your own.
I've covered the steps for rebuilding Apache here in the forums in the past--it's not hard if you're familiar with rebuilding RPMs (and even if you aren't, if you're comfortable with the command line, yum, and a text editor, if you follow the steps I provided).
As for the php wrapper, I've posted discussion of that as well, but since I suspect it'll be harder to dig up than the Apache build instructions, here's an example of a wrapper that you'd put in /home/domain/fcgi-bin (you could put it in your skel if you want it to end up in all virtual server homes):
[code:1]
#!/bin/sh
PHPRC=$PWD/../etc/php5
export PHPRC
umask 022
PHP_FCGI_CHILDREN=4
export PHP_FCGI_CHILDREN
exec /usr/bin/php-cgi
[/code:1]
And in your Server Templates Apache section, you'd add something like this:
[code:1]
AddHandler fcgid-script .php
FCGIWrapper ${HOME}/fcgi-bin/php5.fcgi .php
[/code:1]
You also want to make sure the suexec option is turned on in Virtualmin (I think there's an option for it in GPL, anyway).
And, of course, mod_fcgid needs to be enabled.
See, it's quite complex, which is why none of this stuff is enabled or dealt with in a stock GPL installation--unless we are also able to provide a bunch of packages (as we do in Professional, and to some degree for CentOS and Debian for GPL, if you use the install.sh script) it'd just lead to a bunch of stuff that doesn't work. It takes a lot of components working together to get these things spinning smoothly.
--
Check out the forum guidelines!
<div class='quote'>3) Quotas / Ownership calculations</div>
Yup, suexec is for sure way to go. On FreeBSD you may also be interested in mount(8) option:
suiddir
A directory on the mounted file system will respond to
the SUID bit being set, by setting the owner of any new
files to be the same as the owner of the directory. New
directories will inherit the bit from their parents.
Execute bits are removed from the file, and it will not
be given to root.
This feature is designed for use on fileservers serving
PC users via ftp, SAMBA, or netatalk. It provides secu-
rity holes for shell users and as such should not be used
on shell machines, especially on home directories. This
option requires the SUIDDIR option in the kernel to work.
Only UFS file systems support this option. See chmod(2)
for more information.
Regards.
Thanks expro!
Correct me if I'm wrong, but seeing as it's even hard to find simplified docs on how to setup FastCGI + SuExec (as opposed to fcgid), can I ask if FastCGI has essentially the same setup as mod_fcgid (namely a fcgi script wrapper)?
Reason I ask is I'm having a lot of difficulty getting FastCGI tutorials I found online to actually work - so I thought maybe setting it up using the basic tutorials for mod_fcgid here, with the obvious FastCGI config replacements from other sites..?
<div class='quote'>2) Bandwidth
Bandwidth for each domain is being incorrectly calculated, for some unknown reason. Webalizer is also way off the mark.</div>
I'm willing to accept that we might have a bug in our bandwidth monitoring...but when you see two completely unrelated programs give you similarly "wrong" results, I'd start looking for answers other than "bug". Sounds to me like you have traffic you don't know about.
<div class='quote'>3) Quotas / Ownership calculations
This is a bizarre one, and I really don't know how this was overlooked (if it was). If a user (by user, I mean a subdomain user with their own home directory) accesses their site via FTP, and create a new folder or file, that folder/file will by default have the ownership and permissions of that user, correct?</div>
You need to switch to using suexec for execution of scripts. This is handled automatically in Virtualmin Professional (because we have some control over the available components and can insure that suexec is available and built with the correct docroot), but you'll have to set it up manually in Virtualmin GPL. Otherwise the files are owned by the Apache user, and there's nothing we can do about it.
--
Check out the forum guidelines!
Can I make mod_fcgid flush its buffer implicitly?
<div class='quote'>Can I make mod_fcgid flush its buffer implicitly?</div>
I have no idea what you're talking about. ;-)
--
Check out the forum guidelines!
See here:
https://www.apachelounge.com/viewtopic.php?t=2382&start=0&postda...
If you have any ideas at all, please tell me! Thanks..
What's that have to do with mod_fcgid? That's a PHP option. So, I reckon you'd set it in whatever php.ini is being used for the domain in question.
--
Check out the forum guidelines!
Also, if you want to discuss a completely new topic, it's better to start a new thread. This obviously has nothing to do with quota or bandwidth.
--
Check out the forum guidelines!