VirtualMin inaccessible after server reboot.

17 posts / 0 new
Last post
#1 Sun, 04/26/2015 - 13:30
jxburns

VirtualMin inaccessible after server reboot.

Firstly I am not a Linux expert so please be kind - my expertise is in Windows servers although I have run my own Linux server for past 15 years or so in various disguises/operating systems as a basic unit.

I have leased a new dedicated server with Ubuntu 14.04/64bit installed and did nothing else it it after being passed over to me by the provider. Thought I would use VirtualMin/Webmin as the GUI and do everything from there as have had good reviews from friends. Anyway I used the install.sh to install and everything apparently went well. I was able to login to the GUI and check all was OK using the configuraton checker. All I did was then disable mailman. I did nothing else after this other that running through the configuration check which confirmed everything OK.

I then used the GUI to reboot the server (just to check all was OK) and the server restarted normally. However I could no longer get to the VirtualMin GUI - it eventually times out without any error. I checked various logs and there did not seem anything untoward as far as I could tell including the ones in /var/webmin. I checked Apache2 and WebMin are they both show as running. As does MYSQL.

I have had a knowledgeable friend of mine spend some time looking over the system and even he cannot find out why not. Even just typing the IP address of the server fails to bring up the Apache Ubuntu holding page which it did immediately after installing VirtualMin.

If I then uninstall VirtualMin using (sh install.sh --uninstall) and then reinstall I can once again gain access including displaying the server holding page. Reboot the server (either through GUI or command line) and same thing happens. I cannot obviously see any difference in services running or not.

I have seen some vague references to this behaviour elsewhere but none of the "fixes" appear to solve this. It would appear to be an VirtualMin problem with Apache but have no idea.

So asking to see if anyone has any ideas?

Thanks - John

Sun, 04/26/2015 - 17:27
lostandfound

I am no expert all but how are you accessing Virtualmin? I think by default it is https://xx.xxx.xx.xx:10000 . Do you get a response if you run "virtualmin info" on the command line? If yes then what does "virtualmin check-config" give you?

If you inspect the headers the page which had shown the Apache holding page does it give you an error, not found or forbidden?

Mon, 04/27/2015 - 10:03 (Reply to #2)
jxburns

I am using the IP:10000. This is not the first time I have used WebMin/VirtualMin but 1st time I have installed from the all-in-one installer. However the last time I installed WebMin then VirtualMin and rebooted I had exactly the same result at the time putting it down to something I had done wrong.

As for virtualmin info this is what I get

cpu: * 1 * 0 * 100 * 0 * 0 disk_free: 929582524416 disk_total: 982214553600 fcount: dir: 0 dns: 0 doms: 0 ftp: 0 logrotate: 0 mail: 0 mysql: 0 postgres: 0 spam: 0 ssl: 0 unix: 0 virus: 0 web: 0 webalizer: 0 webmin: 0 ftypes: * doms * dns * web * ssl * mail * dbs * users * aliases host: hostname: serverxxxxxxxxxxxxxxxxxxxxx.live-servers.net [hidden] module root: /usr/share/webmin/virtual-server os: Ubuntu Linux 14.04.2 root: /usr/share/webmin theme version: 9.0 virtualmin version: 4.16.gpl webmin version: 1.740 io: * 0 * 1 kernel: arch: x86_64 os: Linux version: 3.13.0-49-generic load: * 0.01 * 0.03 * 0.05 * 1600 * Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz * GenuineIntel * 6291456 * 4 maxquota: 0 mem: * 12172516 * 11200100 * 4000176 * 4000176 * 1019920 * procs: 145 progs: * * Operating system * Ubuntu Linux 14.04.2 * * Perl version * 5.018002 * * Path to Perl * /usr/bin/perl * * BIND version * 9.9.5 * * Postfix version * 2.11.0 * * Mail injection command * /usr/lib/sendmail -t * * Apache version * 2.4.7 * * PHP versions * 5.5.9 * * Webalizer version * 2.23-08 * * Logrotate version * 3.8.7 * * MySQL version * 5.5.43 * * ProFTPd version * 1.35 * * SpamAssassin version * 3.4.0 * * ClamAV version * 0.98.6 reboot: 0 status: * feature: web name: Apache Webserver status: 1 * feature: dns name: BIND DNS Server status: 1 * feature: mail name: Postfix Mail Server status: 1 * feature: dovecot name: Dovecot IMAP / POP3 Server status: 1 * feature: ftp name: ProFTPd FTP Server status: 0 * feature: sshd name: SSH Server status: 1 * feature: mysql name: MySQL Database Server status: 1

virtualmin check-config gives the following:

Your system has 11.61 GB of memory, which is at or above the Virtualmin recommended minimum of 256 MB.

BIND DNS server is installed, and the system is configured to use it.

Mail server Postfix is installed and configured.

Postfix can support per-domain outgoing IP addresses, but is not currently configured to do so. This can be setup in the Postfix Mailserver module.

Apache is installed.

The following PHP versions are available : 5.5.9 (/usr/bin/php5-cgi)

Webalizer is installed.

Apache is configured to host SSL websites.

MySQL is installed and running.

ProFTPd is installed.

Logrotate is installed.

SpamAssassin and Procmail are installed and configured for use.

ClamAV is installed and assumed to be running.

Plugin AWstats reporting is installed OK.

Plugin Protected web directories is installed OK.

Using network interface eth0 for virtual IPs.

Default IPv4 address for virtual servers is xxx.xxx.xxx.xxx. [hidden]

Default IP address is set to xxx.xxx.xxx.xxx, which matches the detected external address.

Both user and group quotas are enabled for home and email directories.

All commands needed to create and restore backups are installed.

The selected package management and update systems are installed OK.

OK

And the page reports taking too long to respond. The header however shows as 200 OK.

Thanks - John

Wed, 04/29/2015 - 09:01 (Reply to #3)
jxburns

I thought I would replicate the steps on a brand new VirtualBox session by installing Ububtu 14.04LTS first (to simulate the server build) and then the VirtualMin script. Needless to say that after rebooting everything works perfectly.

So perhaps there is something on the Ubuntu setup that is overwritten by the VirtualMin setup that, upon reboot, the server objects to. Does anyone else have any ideas of what I should be looking for.

This is on a FastHosts dedicated server if that helps.

Thanks - John

Wed, 04/29/2015 - 10:12
andreychek

Howdy,

Hmm -- does it correct that problem to manually restart Webmin?

You can do that with this command:

service webmin restart

If so, that may mean that Webmin isn't listed in the services to start at launch time, for some reason.

-Eric

Wed, 04/29/2015 - 10:21 (Reply to #5)
jxburns

No. Already tried that. Webmin does restart but does cure the problem.

In fact, after a server reboot

service webmin status

returns

Webmin (pid 2018) is running

(similary so does apache2)

So appears to be normal.

Rgds John

Wed, 04/29/2015 - 11:03
andreychek

Is there perhaps a firewall blocking access to your server?

You can determine that with this command:

iptables -L -n

Wed, 04/29/2015 - 16:12 (Reply to #7)
jxburns

As I mentioned the problem occurs after rebooting after installing VirtualMin. Before installing VirtualMin and immediately afterwards I have no problem accessing any of the system. I can't see it being a firewall problem unless VirtualMin is setting something unexpected.

Currently (with VirtualMin/holding page unobtainable) iptables -L -n reports:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:22
DROP       all  --  0.0.0.0/0            0.0.0.0/0

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0

Thanks - John

Wed, 04/29/2015 - 17:41
andreychek

Howdy,

Ah, it looks like you have a firewall that is dropping everything except for SSH traffic on port 22.

My recommendation would be to disable the firewall on your server, that should resolve the issue you're experiencing.

-Eric

Wed, 04/29/2015 - 18:35 (Reply to #9)
jxburns

That was after a server reboot after installing VirtualMin. Before rebooting (and with VirtualMin running OK) it was reporting this which I now see has lots of relevant ports open.

Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0            udp dpt:53
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:20000
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:10000
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:443
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:80
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:993
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:143
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:995
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:110
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:20
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:21
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:53
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:587
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:25
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:22


Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

UFW was not running as you can see.

After rebooting the server, and proving VirtualMin no longer working, I then ran

iptables -P INPUT ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -F

to clear the firewall rules and voila, the system working albeit without any firewall.

I then rebooted. Immediately after booting I regained access to the terminal. Logged in and checked the firewall and saw what I expected:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

But no access to VirtualMin. Then checked the firewall again and this time it reported:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:22
DROP       all  --  0.0.0.0/0            0.0.0.0/0

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0

Somehow the server boot process starts off with a firewall that is as it was just before the reboot only to be overrideen with one just allowing access using SSH. I am now wondering if the hosting company have done something to ensure the server is firewalled by default (whereas an Ubuntu install normally leaves everything open) and their iptables is being installed by default from some location I have yet to find in a startup script.

Thanks for the suggestions. This is leading me somewhere as they are not telling me a lot yet about how they set up Ubuntu so ammunition to go back to them with.

Rgds John

Wed, 04/29/2015 - 20:58
Diabolico
Diabolico's picture

Why just dont rebuild your server with fresh Ubuntu? Should not take more than few minutes.

- I often come to the conclusion that my brain has too many tabs open. -
Failing at desktop publishing & graphic design since 1994.

Wed, 04/29/2015 - 22:18
andreychek

Howdy,

Are you using any software that could be adding a firewall?

The process of installing Virtualmin does open holes in a firewall.

However, if there's other software that had been installed that's re-configuring the firewall afterwards, that could be causing the problem you're seeing.

You could always show us the process list from your server, you can get that by running "ps auxw".

-Eric

Thu, 04/30/2015 - 02:02 (Reply to #12)
jxburns
Thu, 04/30/2015 - 09:55
andreychek

Hmm, I actually don't see anything there that would be configuring a firewall.

You could always talk to your provider and see if they know what might be adding that (if you didn't already, you may have).

However, while this is a bit "hack-ish", just to help you for the time being, you could always add those commands you used earlier to clear the firewall to your startup scripts.

That is, in /etc/rc.local, you could add these lines:

iptables -P INPUT ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -F

That rc.local file runs after everything else launches during the bootup process, so whatever it is that's adding the firewall, those lines to clear it should run later.

And then you could start tinkering with the rest of your system in the meantime, and can deal with the firewall issue some other time :-)

Thu, 04/30/2015 - 09:58
andreychek

Also, once you get that working -- you might want to double-check in Webmin -> Network Configuration -> Linux Firewall, that you don't see a firewall configured in there.

If you do, you may want to disable it.

-Eric

Thu, 04/30/2015 - 10:07 (Reply to #15)
jxburns

Great. Thanks for the advice.

I am talking to the hosting company as they have told me that if I place rules in /etc/network/firewall it will use them. However there is nothing there at the moment but still they are being overwritten at server boot up. Obviously I don't want it to be open except for the bare minimum but now you have told me where to look will see if I can do something temporary.

And noted re WebMin firewall.

Will get back to you once I have managed to progress things.

Rgds John

Fri, 05/01/2015 - 03:15 (Reply to #16)
jxburns

The comment regarding rc.local gave me the clue to what is going on.

The only line in there was a call to another script (/etc/sysconfig/firewall) which (surprise surprise) had all the offending firewall rules in it overwriting the VirtualMin versions.

When I contacted the hosting company (FastHosts) they revealed this is something they do to protect the server after a rebuild until the customer can alter to their own requirements. Unfortunately they don't tell anyone. I have requested they add to their KB articles or at least to the email they send out telling the customer the server is ready.

So thanks to all for your help. I am rebuilding the server again ready to install VirtualMin so I know it is clean of all the half hearted attempts I made at changing things.

John