NFS errors on Virtual Server Creation

16 posts / 0 new
Last post
#1 Sun, 11/18/2012 - 03:50
nibb

NFS errors on Virtual Server Creation

I mounted /home on a NFS shared storage but when I create a virtual server in Virtuamin I get some errors, I see the domain home folder is created but not anything else under it as opposed to using the local disk:

Adding new virtual website .. .. done

Adding webserver user apache to server's group .. .. done

Performing other Apache configuration .. .. configuration failed : Failed to copy /etc/php.ini to /home/example/etc/php5/php.ini : cp: accessing `/home/example/etc/php5/php.ini': Permission denied

Setting up scheduled Webalizer reporting .. .. Webalizer reporting failed! : Failed to open /home/example/public_html/stats/.htaccess.webmintmp.2719 : No such file or directory at ../web-lib-funcs.pl line 1360, line 1.

This seems to be permissions denied errors, but from where or what? I checked the NFS storage, and full/write privileges are set.

Why can´t the Virtualmin create the data on the NFS storage?

Sun, 11/18/2012 - 04:05
Locutus

One possible reason that comes to mind here is:

Virtualmin does all its things as the root user, while NFS's default behavior is to forbids operations as root. If you want your NFS export to allow operations as root, you need the option "no_root_squash" in your export definition.

Sun, 11/18/2012 - 04:12 (Reply to #2)
nibb

I'm pretty sure the problem is the NFS export some how, but I don´t know what exactly.

no_root_squash is on the export, and I get this errors.

Also, Jamie said here that option should not be on the export, and then someone else says it does. So im confused: https://www.virtualmin.com/node/21752

Reply 1.

So which one is it?

Also, this is concerning: http://www.centos.org/docs/4/4.5/Security_Guide/s2-server-nfs-noroot.html

Sun, 11/18/2012 - 07:09
Locutus

Jamie said the option "root_squash" must not be on the export. The options are "root_squash" (which turns the "reject root" option ON), and "no_root_squash" (which turns it OFF). If you specify neither nor, the default is "ON". So you need to have "no_root_squash" on the export.

The security implications of this topic do apply, so it is not really advisable to have /home on an NFS import if it can be avoided.

If you still get access denied errors, there might be further permission problems on the server which exports the home directory. Can you manipulate directories manually in the mounted /home from a shell login as root?

Sun, 11/18/2012 - 19:14 (Reply to #4)
nibb

Yes I understand this options and the errors where always with the option "no_root_squash" on.

I assume, that if you do not have this option on, then Virtuamin server will not be able to write/read the share, because it does everything as root. And im correct on this.

If you turn the option "no_root_squash" off, then you cannot even create a folder or anything on the share, as root access is denied.

When the option is turned on, which is what I always had from start in my setup, I can just write/read just fine, I create delete, files, folders, etc. So the option "no_root_squash" is correct here.

The Virtualmin creates a virtual server. Lets say "example.com" the folder itself it created. I see on the share the folder "example.com" but its empty.

And it shows the errors I posted above.

On the share I manually created the /home folder"

I exported this and this is where Virtualmin is supposed to create the data when its creating the virtual apache servers.

I would avoid using the share if I could, sadly right now this is sadly what I have and I need just to create one virtual server on the shared. Which permissions should I exactly check on the NFS server?

Sun, 11/18/2012 - 23:37
nibb

I honestly give up after almost 2 days. The problem is clearly with Virtualmin, the errors fails to copy files because the folder does not exists, example:

Performing other Apache configuration .. .. configuration failed : Failed to copy /etc/php.ini to /home/example/etc/php5/php.ini : cp: accessing `/home/example/etc/php5/php.ini': Permission denied

The folders does not exist "etc" or "php5", so it fails here to copy the php.ini to that location. And every possible file.

Before that not a single error exists on Virtualmin, and of course some thing fails because Virtualmin is lying about the creation of the files on:

Adding administration user to groups .. .. done

Creating home directory .. .. done

Not true. It creates the "home" directory in this case "example" under /home

But nothing under it. I tried every possible configuration on the NFS share, and everything is not working. There is absolutely no problem in creating/erasing, editing files from the server in the share but Virtualmin cannot create the configurations files under it.

As there is no error either why its not creating folders and files im complete dark.

Mon, 11/19/2012 - 06:52
Locutus

So when your attempt to create a VM domain fails with the error message you listed, you can manually create the required directories "etc" and "php5" from a shell with root account?

If so, you might want to turn on Webmin's debug logging, which will hopefully give some more insight into which commands it tries to execute and if any of those fail.

I myself never tried having the home directory on an NFS export, but I'll do some tests when I get around to it.

Mon, 11/19/2012 - 07:11
Locutus

Addendum: I just did some tests regarding the problems here.

On my experimental Virtualmin installation, I created an NFS export under "/srv/export/home", allowing mount by localhost and no_root_squash. I copied the existing home contents to there and mounted it as "/home". Re-checked Virtualmin config, no problems (except for quotas being turned off, which is irrelevant).

I created a test domain on the NFS export with the usual methods. "etc" and "php5" get created just fine, I don't get any problems here.

It is possible that you're seeing the problems because your NFS export is actually on a different machine with different users. The problems might stem from attempts to set the directory ownership or something.

I can't do much more on my end but suggest to turn on Webmin's debug logging, under "Webmin / Webmin / Webmin Configuration : Debugging Log File". Options should be self-explanatory.

I can also offer to log in to your system myself and take a look for myself, if I can spot any problems. I'd need root access to your Virtualmin, and the shell on both systems (Virtualmin and NFS export) for that.

Mon, 11/19/2012 - 07:22 (Reply to #8)
nibb

Thanks for the reply. Yes, I can create folders manually just fine from the Virtualmin server from the command line and even with my SFTP software, it seems its Virtualmin itself that just can´t.

I was looking how or where to enable a log at all, as the error was not very helpful. So thank you on that. I will turn it on and try again as I was looking how to enable some logs for debugging.

Yes, the NFS share is on another server. I don´t see the point otherwise I would just use the local drives. How did you exactly created the NFS share? In the same server?

I was so frustrated with this that I actually created a full new NFS server just to discard any problems from my current NFS share, and I had exactly the same problem with the same errors, and this new NFS server had only 2 things on it. CentOS 6 minimal, Virtualmin. Absolutely nothing else. Got the same errors.

I have not looked into the user/group problem yet, because I did not even came that far. I know Apache with the correct ID and usergroup also exists in the NFS server, they both have ID 48 and root of course ID 0. Even when I would assumed I would have this problem along the road when starting services. Im not even that far, since I can´t even create the files for a single virtual server, and I assume Virtualmin users "root" for this, which can access the NFS share fine.

No email, no extra services, I just want to setup a website in the share.

Let me turn on logging and I will reply back with further info to see what is going on. Which version of Virtualmin did you used for your tests by the way? And which Linux OS where you using? Both in the Virtualmin server, and in the NFS export?

I only mounted /home in the Virtualmin server to the NFS share, as this is where its creating the files. I know I will have to mount some other folders in the future for Apache to work, but I was not even that far yet.

Mon, 11/19/2012 - 07:52
nibb

I don´t see anything strange in the webmin.debug log

The actions there appear as if its creating data, files/folder which is not.

Like: 3252 [19/Nov/2012 08:27:07.895486] root 192.168.0.1. virtual-server WRITE "/home/example/.bash_profile"

The only think I see from day one, with both NFS servers shares I tested is that there is a folder created in the /home share called: virtualmin-autoreply

It has permissions 1777

When I delete the virtual server example.com I use on my tests, Virtualmin also deletes the folder it created "example" under /home/example

Just fine.

I cannot see anything in the log where Virtualmin has an error creating or setting a perm on a file. I understand quotas would not work in the share, which is fine, but I think the reason why it worked in your case is because your NFS share was probably in the same server as your Virtualmin virtual server.

I think this must be some type of file locking of something else that is escaping my mind which has not sleep for 2 days now.

I just did not imagined it would be so complicated to accomplish something like this. In particular because the server has no problems writing/reading the mount share. Its virtualmin which has or just does not like the share for some reason. If I unmount /home, then Virtualmin creates everything fine with allot of folders and files under /home/example/

Like it should. The minute you mount the share, it only and only creates one empty folder "example". Nothing under it.

Mon, 11/19/2012 - 10:32
Locutus

I put the NFS export on the same server in my test only because I didn't have a second server available to do the test. :) Generally it doesn't make sense of course to export a folder via NFS and mount it on the same server.

I'm using Ubuntu 12.04. Installed the "nfs-kernel-server", edited the exports using Webmin's NFS module. Virtualmin Version is 3.95.

I'm afraid I'm quite out of ideas, short of taking a look at your server(s) myself. Maybe the Virtualmin guys have some more insight aside from that, hopefully Eric will notice this thread. :) It is of course possible that it's a Virtualmin bug which doesn't occur on Ubuntu and/or with NFS mounts on the same server.

Tue, 11/20/2012 - 10:36 (Reply to #11)
nibb

Indeed, this seems a problem with Virtualmin on files/configuration creation. There is nothing I can do, just create the Apache files manually.

Its sad as I imagined I could use Virtualmin for this, and to automated some tasks, at least for account creation, suspend, etc. It would have being nice to start replacing cPanel servers with something like this. But based on this and the absolutely lack of commercial support I cannot depend on this for more serious issues. With cPanel at least I get a reply in 15 minutes to any problem I have, and even if I go with Virtualmin Pro there does not seem to be any paid support or requests anyway. I guess its a problem with open source products, once you have a problem you are on your own. Gladly this started in my initial testing states, and not later with services running on it.

I guess its back to the old manual file creation. I spend to much time lost on this and there is nothing I can do if the bug and problem is with how Virtualmin creates this files.

Tue, 11/20/2012 - 11:10
Locutus

Uuhm, I think you're getting something wrong there. :)

If you get Virtualmin Pro, you DO get free "commercial" support from the developers, via the Issue Tracker. During U.S. business hours, they usually reply within a few minutes to hours.

And if you think you spotted a bug, even as a Virtualmin GPL user, you're free to open a bug report in the Issue Tracker. The developers will certainly respond to it.

Sat, 12/15/2012 - 01:33
RichardCheung

I have the same error too.

I mount the /home at a NFSv4 server with the following exports settings

(ro,nohide,insecure,no-subtree-check,async,no-root-squash)

then the domain is created with root:root owner but not with the domain name.

and there are nothing inside the domain folder

e.g.

drwxr-x--- 2 root root 4096 Dec 15 15:13 .

drwxrwxrwt 8 root root 4096 Dec 15 15:13 ..

Richard

Sat, 12/15/2012 - 04:19 (Reply to #14)
RichardCheung

I just found that it is NFSv4 problem.

When I tried to do the following

chown example.com:example.com /home/example.com

I got

chown: changing ownership of `/home/example.com': Invalid argument

Please check this bug

https://bugs.launchpad.net/ubuntu/+source/nfs-utils/+bug/662711

Sun, 12/16/2012 - 07:51 (Reply to #15)
RichardCheung

I have tested on ubuntu 12.04 and 12.10

12.10 works fine on chown

but

12.04 will have the above error