apache issue??

20 posts / 0 new
Last post
#1 Tue, 10/19/2010 - 19:42
midol

apache issue??

When I log in to my server as root on port 10000 the default system status shows a red X for apache web server. When I push the green arrow to its right I get:

Failed to start service :

Starting httpd: (98)Address already in use: make_sock: could not bind to address [::]:80 (98)Address already in use: make_sock: could not bind to address 0.0.0.0:80 no listening sockets available, shutting down Unable to open logs [FAILED]

Now in fact apache IS running and serving pages just fine. What can I do to get the status report back on track? VM Pro3.81

Dave

Tue, 10/19/2010 - 19:50
andreychek

It's possible the Apache pid files are out of sync with what Virtualmin is expecting. Using the command line, you may want to restart both Apache and Virtualmin.

If you aren't sure of what commands to use, I can help, just let me know what distro you're using.

-Eric

Tue, 10/19/2010 - 20:17 (Reply to #2)
midol

using centos 5.5

Tue, 10/19/2010 - 21:04
andreychek

Okay, so you can try running these two commands as root on your server:

/etc/init.d/httpd restart
/etc/init.d/webmin restart

After doing that, do the services show up correctly in the status area of Virtualmin?

-Eric

Wed, 10/20/2010 - 11:30 (Reply to #4)
midol

[root@cserver admin]# /etc/init.d/httpd restart Stopping httpd: [FAILED] Starting httpd: (98)Address already in use: make_sock: could not bind to address [::]:80 (98)Address already in use: make_sock: could not bind to address 0.0.0.0:80 no listening sockets available, shutting down Unable to open logs

Fri, 12/17/2010 - 13:33 (Reply to #5)
midol

But. If instead of running those commands I go to Webmin | Servers | Apache Webserver and then click on Stop Apache (in the top right corner) then wait for the screen to refresh and click on Start Apache that DOES work. I find this a) handy because it works b) disconcerting because it seems to me to imply that the actions are not equivalent, which I would expect them to be. So what is the difference in what the scripts do in the two cases? I found this exact behaviour in an instance of VM GPl on another machine, too.

Bug?

Dave

Wed, 10/20/2010 - 14:12
Locutus

That looks to me like it is trying to stop the Apache, but fails, and the subsequent start attempt fails because the port is already in use (Apache still running?)

Try ps aux | grep apache to see if there are any Apache processes currently running.

Also check the file /var/run/apache2.pid which should contain the process ID if any Apache is running currently.

Wed, 10/20/2010 - 18:28 (Reply to #7)
midol

[root@cserver admin]# cat /var/run/apache2.pid cat: /var/run/apache2.pid: No such file or directory [root@cserver admin]# cd /var/run [root@cserver run]# ls acpid.socket gdm.pid mysqld saslauthd atd.pid gpm.pid named sdp audispd_events haldaemon.pid named.pid setrans auditd.pid hpiod.pid netreport setroubleshoot autofs.fifo-misc hpiod.port NetworkManager snmpd autofs.fifo-net hpssd.pid news spamassassin avahi-daemon hpssd.port nscd sshd.pid console klogd.pid ntpd.pid sudo crond.pid lvm pcscd.comm syslogd.pid cups mailman pcscd.pid utmp cupsd.pid mdadm pcscd.pub winbindd dbus mdmpd pm wpa_supplicant ddclient messagebus.pid ppp xfs.pid dhclient-eth0.pid mod_fcgid proftpd xinetd.pid dovecot munin rpc.statd.pid

Wed, 10/20/2010 - 19:04
Locutus

Okay, so there's no Apache PID file. That would explain why stopping the service fails. What about the ps aux output I mentioned?

Next, you can check what is listening on port 80, if it's not Apache, using this:

netstat -ptln | grep :80

And a hint: When posting output of commands, please embed the text in code-tags, like so:   and put your text between those tags. That will prevent line-breaks from being ignored and will use a monospaced font, making text output MUCH easier to read.

Wed, 10/20/2010 - 19:08 (Reply to #9)
midol
[root@cserver run]# ps -aux | grep apache Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.7/FAQ apache 586 0.0 1.0 45248 22512 ? S Oct17 0:00 /usr/sbin/httpd apache 587 0.0 2.2 90368 45792 ? S Oct17 1:57 /usr/sbin/httpd apache 588 0.0 2.1 90160 45636 ? S Oct17 1:51 /usr/sbin/httpd apache 589 0.0 2.2 90436 45752 ? S Oct17 1:55 /usr/sbin/httpd apache 590 0.0 2.2 90448 45772 ? S Oct17 2:00 /usr/sbin/httpd apache 591 0.0 2.2 90348 46368 ? S Oct17 1:58 /usr/sbin/httpd apache 592 0.0 2.2 90400 45824 ? S Oct17 1:47 /usr/sbin/httpd apache 593 0.0 2.2 90332 46392 ? S Oct17 1:49 /usr/sbin/httpd apache 594 0.0 2.2 90416 45848 ? S Oct17 1:57 /usr/sbin/httpd root 8065 0.0 0.0 4876 692 pts/2 S+ 16:27 0:00 grep apache   [root@cserver run]# netstat -ptln | grep :80 tcp 0 0 :::80 :::* LISTEN 586/httpd

and thanks for the  tip. I knew such a thing existed but didn't know exactly what it was.

Wed, 10/20/2010 - 19:07
Locutus

Next hint: If netstat reveals that it is indeed Apache who is listening on :80, the PID file might have gotten lost. You can then kill all active Apache processes, and subsequently restart it, by doing this:

killall apache2 /etc/init.d/apache2 start
Wed, 10/20/2010 - 19:11
Locutus

Question: what distro and what version of Apache are you running? In mine, the executable is named "apache2" and not "httpd". /usr/sbin/httpd -v should reveal that info.

Fri, 10/22/2010 - 10:33 (Reply to #12)
midol

CentOS 5.5, apache 2.2.3. I think Debian and derivatives use apacheX as process name

Fri, 10/22/2010 - 19:46 (Reply to #13)
midol

killall httpd /etc/init.d/httpd start

and the virtualmin system status page still shows apache as not running. Of course apache is serving up the page I'm looking at....

Dave

Sat, 10/23/2010 - 18:11
Locutus

Maybe VMin uses an incorrect method to determine if Apache is running... Try "Refresh System Information" or System Settings / Re-Check Configuration. Might have to do with "httpd vs. apache2" naming, or with PID file.

Also check the items in the Apache Webmin module, Module Config / System configuration, like "Command to start/stop Apache" and the "Path to Apache PID file".

Tue, 11/02/2010 - 18:09
midol

the /etc/httpd/conf/httpd.conf file points to /etc/httpd/run as the folder for the .pid file and there is no such file there. So if that's where VM is looking for the file I can see why it doesn't find it. BUT - how has this taken place, what can be done and how can I prevent it from happening again?

Dave

Wed, 11/03/2010 - 04:57
Locutus

The place where Webmin looks for the PID file is configured in Webmin -> Servers -> Apache Webserver -> Module Config -> System Configuration.

The place where Apache stores its PID file is configured in Webmin -> Servers -> Apache Webserver -> Global Configuration -> Miscellaneous.

The corresponding config file entry for me reads:

# # PidFile: The file in which the server should record its process # identification number when it starts. # This needs to be set in /etc/apache2/envvars # PidFile ${APACHE_PID_FILE}

Might be different for you of course. And how they got out of sync for you, if they did, I can't tell right off the bat. :)

Fri, 01/07/2011 - 21:48 (Reply to #17)
midol

I see from the httpd.conf file that the .pid file is supposed to be in the /etc/httpd/run folder. On one of my VM instances this file is present and correct. On the other it isn't (it's absent). That's the one that gives the erroneous system information display. The server is running and serving pages but the inaccuracy in the display is troubling. Also there is some (related?) issue that periodically results in all but one instance of the daemon being killed off. This gives extremely slow response and makes for grumpy users. Ideas?

Dave

Fri, 03/25/2011 - 08:27
sschueller

I am also having this issue on 3 VMPro and 1 VMGPL CentOS 5.5 installs . Three of them have Coldfusion running along side and one is running railo with tomcat which I though may have something to do with it. I have changed the PIDFILE path in httpd.conf to match the init script path but it didn't fix the problem.

At some point after a few days the PID file disappears and the red X appears in the panel. Although apache is still working it looks like the main process which runs as root is missing but all the children are still there. I have to kill the processes to start apache again with the init script. Pid file will appear in the right place.

I am not sure if it happens at this time but I did find this in the logs every week at the exact same time:

/var/log/cron.1:Mar 20 04:02:01 myhostname crond[26715]: (root) CMD (run-parts /etc/cron.daily)
/var/log/cron.1:Mar 20 04:02:01 myhostname crond[26716]: (root) CMD (/etc/webmin/virtual-server/collectinfo.pl)
/var/log/httpd/error_log:[Sun Mar 20 04:02:01 2011] [notice] Digest: generating secret for digest authentication ...
/var/log/httpd/error_log:[Sun Mar 20 04:02:01 2011] [notice] Digest: done
/var/log/httpd/error_log:[Sun Mar 20 04:02:01 2011] [notice] mod_python: Creating 4 session mutexes based on 256 max processes and 0 max threads.
/var/log/httpd/error_log:[Sun Mar 20 04:02:01 2011] [warn] No JkShmFile defined in httpd.conf. Using default /etc/httpd/logs/jk-runtime-status
/var/log/httpd/error_log:[Sun Mar 20 04:02:01 2011] [notice] Apache/2.2.3 (CentOS) configured -- resuming normal operations
/var/log/httpd/mod_jk.log:[Sun Mar 20 04:02:01.922 2011] [21311:654213888] [info] init_jk::mod_jk.c (3183): mod_jk/1.2.28 initialized
/var/log/httpd/error_log.1:[Sun Mar 20 04:02:01 2011] [notice] SIGHUP received.  Attempting to restart
/var/log/messages:Mar 20 04:02:01 myhostname syslogd 1.4.1: restart.

/var/log/httpd/error_log.2:[Thu Mar 17 04:02:02 2011] [notice] SIGHUP received.  Attempting to restart
/var/log/httpd/error_log.1:[Sun Mar 20 04:02:01 2011] [notice] SIGHUP received.  Attempting to restart
/var/log/httpd/error_log.4:[Sun Feb 20 04:02:01 2011] [notice] SIGHUP received.  Attempting to restart

I just changed Webmin -> Servers -> Apache Webserver -> Module Config -> System Configuration. to point directly to the file instead of auto detect which was set by default. Lets hope this will fix it.

Any suggestions on how I can track down what kills the main httpd process if this doesn't solve the problem?

Thanks,

Stefan

Sun, 03/27/2011 - 08:25 (Reply to #19)
sschueller

The changes didn't fix the problem. It happened again at 4am, here are the logs from today:

[root@myhostname:~] $ grep -r "Mar 27 04:02:0" /var/log/
/var/log/cron.1:Mar 27 04:02:01 myhostname crond[9153]: (root) CMD (run-parts /etc/cron.daily)
/var/log/cron.1:Mar 27 04:02:01 myhostname crond[9156]: (root) CMD (/etc/webmin/virtual-server/collectinfo.pl)
/var/log/httpd/error_log.1:[Sun Mar 27 04:02:01 2011] [notice] SIGHUP received.  Attempting to restart
/var/log/httpd/error_log:[Sun Mar 27 04:02:01 2011] [notice] Digest: generating secret for digest authentication ...
/var/log/httpd/error_log:[Sun Mar 27 04:02:01 2011] [notice] Digest: done
/var/log/httpd/error_log:[Sun Mar 27 04:02:01 2011] [notice] mod_python: Creating 4 session mutexes based on 256 max processes and 0 max threads.
/var/log/httpd/error_log:[Sun Mar 27 04:02:01 2011] [warn] No JkShmFile defined in httpd.conf. Using default /etc/httpd/logs/jk-runtime-status
/var/log/httpd/error_log:[Sun Mar 27 04:02:01 2011] [notice] Apache/2.2.3 (CentOS) configured -- resuming normal operations
/var/log/httpd/mod_jk.log:[Sun Mar 27 04:02:01.932 2011] [15766:143188736] [info] init_jk::mod_jk.c (3183): mod_jk/1.2.28 initialized
/var/log/messages:Mar 27 04:02:01 myhostname syslogd 1.4.1: restart.
Virtualmin version 3.83 Pro
Webmin version 1.530
Kernel and CPU Linux 2.6.18-194.26.1.el5xen on x86_64
Operating system CentOS Linux 5.5
Virtual memory 511.99 MB total, 384 kB used
Real memory 1024 MB total, 555.35 MB used
Local disk space 34.46 GB total, 4.34 GB used

Any suggestions where to look next?

Stefan