Apache stops randomly and replace with

Hi,

My Apache randomly stops and I cannot start it again. I said the address already in use. https://goo.gl/Osz905 I have to follow this post (first answer) to kill the process: http://askubuntu.com/questions/277162/apache-fails-to-start-address-alre... And then restart Apache by: service apache2 start

Then I follow the second answer to disable the port in ports.conf but the problem still exists. It show other error when I try to start via webmin: https://goo.gl/cKt31z

Please help me. It is very urgent for me.

Thanks, Giang Anh

Status: 
Active

Comments

Howdy -- it sounds like the process may have gotten stuck.

You should be able to correct that by running these two commands:

killall -9 apache2
service apache2 start

Hi,

Is this the permanent solution? Or I have to use these command every I have this issue?

Thanks, Giang Anh

Hmm, are you seeing that issue a lot?

That issue shouldn't be occurring frequently... if it is, we may need to try and troubleshoot what is causing the processes to lock up like that.

Yes. I have this issue about 3-4 day/time. I have just happened as well.

Ah, that sounds very bad, that should definitely not happen. Are you perhaps on a VPS? If so, what is the output of "free -m"?

Also, do you see any errors or interesting messages in /var/log/apache2/error_log around the time that Apache is stopping?

Hi,

# free -m total used free shared buffers cached Mem: 56360 4984 51376 1016 259 2791 -/+ buffers/cache: 1933 54427 Swap: 0 0 0

Do you see any errors or interesting messages in /var/log/apache2/error_log around the time that Apache is stopping?

And if this is a VPS, what kind of VPS is it? For example, KVM, Xen, or OpenVZ?

Hi,

The log here.

[Wed Apr 13 18:03:53.727260 2016] [core:notice] [pid 1857] AH00051: child pid 24041 exit signal Aborted (6), possible coredump in /etc/apache2 *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** [Wed Apr 13 18:05:09.810731 2016] [core:notice] [pid 1857] AH00051: child pid 23172 exit signal Aborted (6), possible coredump in /etc/apache2 *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** [Wed Apr 13 18:05:19.825293 2016] [core:notice] [pid 1857] AH00051: child pid 24039 exit signal Aborted (6), possible coredump in /etc/apache2 *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** [Wed Apr 13 18:05:38.846757 2016] [core:notice] [pid 1857] AH00051: child pid 20311 exit signal Aborted (6), possible coredump in /etc/apache2 *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** [Wed Apr 13 18:05:40.849122 2016] [core:notice] [pid 1857] AH00051: child pid 22663 exit signal Aborted (6), possible coredump in /etc/apache2 *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** [Wed Apr 13 18:13:03.324194 2016] [core:notice] [pid 1857] AH00051: child pid 24531 exit signal Aborted (6), possible coredump in /etc/apache2 *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** [Wed Apr 13 18:13:04.325337 2016] [core:notice] [pid 1857] AH00051: child pid 24530 exit signal Aborted (6), possible coredump in /etc/apache2 *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** [Wed Apr 13 18:14:05.393549 2016] [core:notice] [pid 1857] AH00051: child pid 24125 exit signal Aborted (6), possible coredump in /etc/apache2 *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** [Wed Apr 13 18:14:06.394841 2016] [core:notice] [pid 1857] AH00051: child pid 22666 exit signal Aborted (6), possible coredump in /etc/apache2 *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** [Wed Apr 13 18:14:30.425816 2016] [core:notice] [pid 1857] AH00051: child pid 24124 exit signal Aborted (6), possible coredump in /etc/apache2 *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** [Wed Apr 13 18:31:11.365460 2016] [core:notice] [pid 1857] AH00051: child pid 20943 exit signal Aborted (6), possible coredump in /etc/apache2 *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** [Wed Apr 13 18:31:15.370020 2016] [core:notice] [pid 1857] AH00051: child pid 24625 exit signal Aborted (6), possible coredump in /etc/apache2 *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** [Wed Apr 13 18:38:15.777484 2016] [core:notice] [pid 1857] AH00051: child pid 25897 exit signal Aborted (6), possible coredump in /etc/apache2 *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** [Wed Apr 13 18:38:16.778719 2016] [core:notice] [pid 1857] AH00051: child pid 25554 exit signal Aborted (6), possible coredump in /etc/apache2 *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** [Wed Apr 13 18:38:17.780041 2016] [core:notice] [pid 1857] AH00051: child pid 23177 exit signal Aborted (6), possible coredump in /etc/apache2 *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** [Wed Apr 13 18:41:17.977523 2016] [core:notice] [pid 1857] AH00051: child pid 24636 exit signal Aborted (6), possible coredump in /etc/apache2 *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** [Wed Apr 13 18:41:47.004606 2016] [core:notice] [pid 1857] AH00051: child pid 24624 exit signal Aborted (6), possible coredump in /etc/apache2 *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** [Wed Apr 13 18:42:01.017409 2016] [core:notice] [pid 1857] AH00051: child pid 23636 exit signal Aborted (6), possible coredump in /etc/apache2 *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** [Wed Apr 13 18:51:10.598754 2016] [core:notice] [pid 1857] AH00051: child pid 23178 exit signal Aborted (6), possible coredump in /etc/apache2 [Wed Apr 13 18:51:15.260912 2016] [mpm_prefork:notice] [pid 1857] AH00171: Graceful restart requested, doing restart *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** [Wed Apr 13 18:51:15.265141 2016] [core:notice] [pid 1857] AH00060: seg fault or similar nasty error detected in the parent process *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 *** *** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f2efc15eb30 ***

It's MS Azure Virtual Machine, not VPS.

Yikes, it looks like you're seeing some unusual Apache crashes.

What is the output of these commands:

dpkg -l 'apache*'
ls /etc/apache2/mods-enabled

Hi,

~# dpkg -l 'apache*' Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-==============-============-============-================================= ii apache2 2.4.7-1ubunt amd64 Apache HTTP Server un apache2-api-20 <none> <none> (no description available) ii apache2-bin 2.4.7-1ubunt amd64 Apache HTTP Server (binary files ii apache2-data 2.4.7-1ubunt all Apache HTTP Server (common files) ii apache2-doc 2.4.7-1ubunt all Apache HTTP Server (on-site docum un apache2-mpm-ev <none> <none> (no description available) un apache2-mpm-it <none> <none> (no description available) un apache2-mpm-pr <none> <none> (no description available) un apache2-mpm-wo <none> <none> (no description available) un apache2-suexec <none> <none> (no description available) ii apache2-suexec 2.4.7-1ubunt amd64 Apache HTTP Server configurable s un apache2-suexec <none> <none> (no description available) un apache2-utils <none> <none> (no description available) un apache2.2-bin <none> <none> (no description available) un apache2.2-comm <none> <none> (no description available) root@JWeb:~# ls /etc/apache2/mods-enabled access_compat.load autoindex.conf fcgid.load proxy_connect.load actions.conf autoindex.load filter.load proxy_http.load actions.load dav_fs.conf mime.conf proxy.load alias.conf dav_fs.load mime.load rewrite.load alias.load dav.load mpm_prefork.conf setenvif.conf auth_basic.load dav_svn.conf mpm_prefork.load setenvif.load auth_digest.load dav_svn.load negotiation.conf slotmem_shm.load authn_core.load deflate.conf negotiation.load socache_shmcb.load authn_file.load deflate.load php5.conf ssl.conf authz_core.load dir.conf php5.load ssl.load authz_host.load dir.load proxy_balancer.conf status.conf authz_svn.load env.load proxy_balancer.load status.load authz_user.load fcgid.conf proxy.conf suexec.load

Hmm, that all looks fairly normal.

When did this problem start occurring?

I've been doing some research on your problem, and it sounds like some people who were seeing that issue were using a PHP cache, such as Zend. Are you using any PHP caching modules for any of your domains?

Hi,

It happened from around 2 months ago. I'm using PHP caching for my Joomla and Wordpress websites. Please let me know if you need root account of the server.

Thanks, Giang Anh

You appear to be seeing either an Apache bug, or a PHP bug.

What you might want to do is disable the PHP caching temporarily, to see if that resolves the problem that you're experiencing.

If it does resolve the issue, you may need to look into a different caching type.

Which type of PHP cache are you using, out of curiosity?

I have just disable OPcache in my php.ini file. Will keep you updated.

It still happens. 5 times today.

And just to verify, you removed those caching modules from all domains that are using them?

I disable it in php.ini file in my server. And yes, it will be removed on all sites.

Just to verify -- are you having any problems with any other services running on your server?

If not, then it's either an Apache bug, or a PHP bug, that you're experiencing.

There isn't an easy way to resolve that.

You could always post a note in the Ubuntu bug tracker regarding that, but they're going to ask what modules you're using, as it's very rare to see Apache crash like that.

So what you could always try, is to review both your Apache modules, as well as your PHP modules, and remove any that you don't need.

For starters, what is the output of this command:

dpkg -l 'php*'

Also, how many domains are running on your server there?

Hi,

I don't have any other issue and this problem happens randomly. Also I'm not familiar with Ubuntu nor Ubuntu bug tracker since from the first time I began with Linux, I use Virtualmin under your support :)

# dpkg -l 'php*' Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-==============-============-============-================================= un php-apc <none> <none> (no description available) un php-crypt-gpg <none> <none> (no description available) ii php-pear 5.5.9+dfsg-1 all PEAR - PHP Extension and Applicat un php-xml-util <none> <none> (no description available) un php4-pear <none> <none> (no description available) ii php5 5.5.9+dfsg-1 all server-side, HTML-embedded script ii php5-cgi 5.5.9+dfsg-1 amd64 server-side, HTML-embedded script ii php5-cli 5.5.9+dfsg-1 amd64 command-line interpreter for the ii php5-common 5.5.9+dfsg-1 amd64 Common files for packages built f ii php5-curl 5.5.9+dfsg-1 amd64 CURL module for php5 un php5-dev <none> <none> (no description available) un php5-fpm <none> <none> (no description available) ii php5-gd 5.5.9+dfsg-1 amd64 GD module for php5 ii php5-json 1.3.2-2build amd64 JSON module for php5 un php5-mhash <none> <none> (no description available) ii php5-mysql 5.5.9+dfsg-1 amd64 MySQL module for php5 un php5-mysqli <none> <none> (no description available) un php5-mysqlnd <none> <none> (no description available) ii php5-readline 5.5.9+dfsg-1 amd64 Readline module for php5 un php5-suhosin <none> <none> (no description available) un php5-user-cach <none> <none> (no description available) un php5-xcache <none> <none> (no description available) un php5-xdebug <none> <none> (no description available) un phpapi-2012121 <none> <none> (no description available)

I'm running around 6 websites under this server. However, this is a powerful server with 16 core strong CPU and 64GB RAM

What you're seeing there is very odd!

Ubuntu 14.04 is one of the more popular distributions for running Virtualmin, and we've never seen crashes like you're experiencing.

I was actually wondering if maybe it were a hardware issue, but if you aren't seeing problems with other services, that is unlikely.

I double-checked the Apache modules that are loaded, and noticed one that stood out -- the socache_shmcb module.

You may want to try disabling that to see if that helps... that can be disabled with this command:

a2dismod socache_shmcb

After running that, restart Apache.

Hi,

I tried to disable that module but it requires ssl module to be disabled. After disable those 2 modules. I cannot restart apache:

# a2dismod socache_shmcb ERROR: The following modules depend on socache_shmcb and need to be disabled first: ssl root@JWeb:~# a2dismod ssl Module ssl disabled. To activate the new configuration, you need to run: service apache2 restart root@JWeb:~# a2dismod socache_shmcb Module socache_shmcb disabled. To activate the new configuration, you need to run: service apache2 restart root@JWeb:~# service apache2 restart * Restarting web server apache2 [fail] * The apache2 configtest failed. Output of config test was: AH00526: Syntax error on line 91 of /etc/apache2/sites-enabled/adanhgia.com.conf: Invalid command 'SSLEngine', perhaps misspelled or defined by a module not included in the server configuration Action 'configtest' failed. The Apache error log may have more information.

Okay, I didn't realize the SSL module depended on it. It sounds like you'll need to re-enable both socache_shmcb and ssl.

I'll do some digging to see if I can get any other ideas.

This Solve it for me: For Debian based systems :

Replace ' apache2 reload ' with ' apache2 restart ' in /etc/logrotate.d/apache2 file.

For RHEL based systems

Replace ' httpd reload ' with ' httpd restart ' in /etc/logrotate.d/httpd file.

If the above steps do not help on Debian/Ubuntu OSes, edit log rotation settings of apache in /etc/logrotate.d/apache2 file. Change " /etc/init.d/apache2 reload " to " /etc/init.d/apache2 restart ":

cat /etc/logrotate.d/apache2

.... /etc/init.d/apache2 restart > /dev/null;