iNTERAL 500 Error after Restoring VM from snapshot

Hey I made a mistake on my server and then restored a snapshot of my server, but since then I have been getting internal error 500 on all my sites for e.g., http://olexor.com, http://madific.com, etc... also, I performed a system update. i need this solved ASAP and was wondering if buying a support incident would be the fastest way to do it I already have Virtualmin License and tried to send a ticket for the Virtualmin interface but came with an HTTP1.4 ERROR

below is the apache logs [Mon May 16 16:20:05.490260 2016] [fcgid:warn] [pid 9918] (104)Connection reset by peer: [client 37.211.87.249:56819] mod_fcgid: error reading data from FastCGI server, referer: http://olexor.com/ [Mon May 16 16:20:05.490293 2016] [core:error] [pid 9918] [client 37.211.87.249:56819] End of script output before headers: index.php, referer: http://olexor.com/ [Mon May 16 16:20:05.492606 2016] [fcgid:warn] [pid 9918] (104)Connection reset by peer: [client 37.211.87.249:56819] mod_fcgid: error reading data from FastCGI server, referer: http://olexor.com/ [Mon May 16 16:20:05.492621 2016] [core:error] [pid 9918] [client 37.211.87.249:56819] End of script output before headers: index.php, referer: http://olexor.com/ [Mon May 16 16:20:23.755936 2016] [fcgid:warn] [pid 9919] (104)Connection reset by peer: [client 130.193.51.25:55732] mod_fcgid: error reading data from FastCGI server [Mon May 16 16:20:23.755973 2016] [core:error] [pid 9919] [client 130.193.51.25:55732] End of script output before headers: index.php [Mon May 16 16:20:23.790701 2016] [fcgid:warn] [pid 9919] (104)Connection reset by peer: [client 130.193.51.25:55732] mod_fcgid: error reading data from FastCGI server [Mon May 16 16:20:23.790722 2016] [core:error] [pid 9919] [client 130.193.51.25:55732] End of script output before headers: index.php [Mon May 16 16:20:46.627875 2016] [fcgid:warn] [pid 9920] (104)Connection reset by peer: [client 66.249.75.236:49689] mod_fcgid: error reading data from FastCGI server [Mon May 16 16:20:46.627910 2016] [core:error] [pid 9920] [client 66.249.75.236:49689] End of script output before headers: index.php [Mon May 16 16:20:46.640507 2016] [fcgid:warn] [pid 9920] (104)Connection reset by peer: [client 66.249.75.236:49689] mod_fcgid: error reading data from FastCGI server [Mon May 16 16:20:46.640521 2016] [core:error] [pid 9920] [client 66.249.75.236:49689] End of script output before headers: index.php [Mon May 16 16:21:05.563476 2016] [fcgid:warn] [pid 9917] (104)Connection reset by peer: [client 37.211.87.249:56847] mod_fcgid: error reading data from FastCGI server

Status: 
Active

Comments

Howdy -- unfortunately, due to the switch to our new website, the Virtualmin Support module isn't able to submit support queries at the moment.

You certainly don't need to purchase a new support incident if you have a Virtualmin Pro license though. Premium support comes with your license, and we're happy to help out.

It looks like you're seeing a generic FCGID error there for that domain.

Could you try going into Server Configuration -> Website Options for that domain, and set the PHP Execution Mode to "CGI"?

Doing that will improve the error messages you're receiving, and should help us figure out the cause of that problem.

Ok will do , but here is an update when i changed the virtualserver options \ users and groups to Run CGI programs as user from global , then the port 80 starts working , but as soon as i change it back to #578 and #515 i get the same error again

How ever the above trick does not work for SSL sites

Ok i just did the PHP execution mode to CGI and then i still get the same error

Anto

ok actually the 443 is working once I change it to as user from global but my WHMCS LOGIN AT https://olexor.com/support is showing an error about permission, could the snapshot restore have caused this But now I am scared if this would be a security loophole leaving the user from global on

Anto

Could you share the exact error you're receiving in the Apache error log when switching the "olexor.com" domain to CGI?

For the moment, you'd just want to change one website, as changing it to CGI will produce better errors in the logs.

Also, can you clarify what the PHP Execution Mode was previously set to in olexor.com? The errors you shared above seem to indicate that it was set to FCGID previously, but I just wanted to make sure that's what you're seeing in the GUI.

Lastly, what is the output of this command:

rpm -qa | grep php

I changed the PHP Script execution from FCGID to CGI, and I still had the 500 error, however leaving the FCGID on and changing the setting on the Apache server for the website to User from the global configuration on the Run CGI program as, I could get to the website.I am now reverting the change in the Run CGI to its original to reproduce the error.

the log output is below [Mon May 16 18:03:18.324099 2016] [cgi:error] [pid 14469] [client 148.251.246.209:50331] AH01215: Unable to open /home/arappai/public_html/wp-content/wflogs/ips.php for reading and writing. [Mon May 16 18:04:26.907095 2016] [cgi:error] [pid 14470] [client 46.235.103.30:26686] AH01215: Unable to open /home/arappai/public_html/wp-content/wflogs/ips.php for reading and writing., referer: http://olexor.com/wp-login.php [Mon May 16 18:14:14.331096 2016] [cgi:error] [pid 15854] [client 37.211.87.249:52918] End of script output before headers: php5.cgi [Mon May 16 18:14:14.344804 2016] [cgi:error] [pid 15854] [client 37.211.87.249:52918] End of script output before headers: php5.cgi [Mon May 16 18:14:15.618503 2016] [cgi:error] [pid 15855] [client 37.211.87.249:60249] End of script output before headers: php5.cgi, referer: http://olexor.com/ [Mon May 16 18:14:15.622625 2016] [cgi:error] [pid 15855] [client 37.211.87.249:60249] End of script output before headers: php5.cgi, referer: http://olexor.com/ [Mon May 16 18:14:28.324432 2016] [cgi:error] [pid 15855] [client 108.162.236.6:60124] End of script output before headers: php5.cgi [Mon May 16 18:14:28.360057 2016] [cgi:error] [pid 15855] [client 108.162.236.6:60124] End of script output before headers: php5.cgi [Mon May 16 18:14:51.073258 2016] [cgi:error] [pid 15853] [client 37.211.87.249:52936] End of script output before headers: php5.cgi [Mon May 16 18:14:51.077529 2016] [cgi:error] [pid 15853] [client 37.211.87.249:52936] End of script output before headers: php5.cgi [Mon May 16 18:14:52.063521 2016] [cgi:error] [pid 15856] [client 37.211.87.249:60271] End of script output before headers: php5.cgi, referer:

Output of the rpm -qa | grep php

rpm -qa | grep php php-imap-5.4.16-3.el7.x86_64 php-pdo-5.4.16-36.1.el7_2.1.x86_64 php-cli-5.4.16-36.1.el7_2.1.x86_64 php-pgsql-5.4.16-36.1.el7_2.1.x86_64 php-process-5.4.16-36.1.el7_2.1.x86_64 php-pear-1.9.4-21.el7.noarch php-5.4.16-36.1.el7_2.1.x86_64 php-odbc-5.4.16-36.1.el7_2.1.x86_64 php-xmlrpc-5.4.16-36.1.el7_2.1.x86_64 wbm-php-pear-1.5-1.noarch php-common-5.4.16-36.1.el7_2.1.x86_64 php-mysql-5.4.16-36.1.el7_2.1.x86_64 php-mbstring-5.4.16-36.1.el7_2.1.x86_64 php-gd-5.4.16-36.1.el7_2.1.x86_64 php-snmp-5.4.16-36.1.el7_2.1.x86_64 php-xml-5.4.16-36.1.el7_2.1.x86_64

Aha, that log message is very helpful!

It looks like you're seeing a file or directory permission issue. While I'm not yet sure why that is suddenly coming up now, we can certainly troubleshoot and fix it.

What is the output of these commands:

ls -ld /home/arappai
ls -ld /home/arappai/public_html
ls -ld /home/arappai/public_html/wp-content/
ls -ld /home/arappai/public_html/wp-content/wflogs/ips.php
ls -l /home/arappai/public_html/wp-content/wflogs/ips.php

ls -ld /home/arappai drwxr-x--- 32 arappai arappai 4096 May 13 12:13 /home/arappai ls -ld /home/arappai/public_html drwxr-x--- 15 arappai arappai 4096 May 16 18:18 /home/arappai/public_html ls -ld /home/arappai/public_html/wp-content/ drwxr-xr-x 10 arappai arappai 4096 May 15 21:38 /home/arappai/public_html/wp-content/ ls -ld /home/arappai/public_html/wp-content/wflogs/ips.php -rw-r--r-- 1 arappai arappai 51 May 4 09:10 /home/arappai/public_html/wp-content/wflogs/ips.php ls -l /home/arappai/public_html/wp-content/wflogs/ips.php -rw-r--r-- 1 arappai arappai 51 May 4 09:10 /home/arappai/public_html/wp-content/wflogs/ips.php

additonally do you think this has to do with the system updates and /or Snapshot restore ? that is the is the only thig i can think off .

and also like i mentioned chaecking the global user setting in the Apache seems at least get me into the site but still not my WHMCS under Https://olexor.com/support/cms

Hmm, those permissions all look pretty normal.

Out of curiosity though, does it help to do this:

chmod +x /home/arappai/
chmod +x /home/arappai/public_html

so I had another guy look at it and this is how he fixed it • Site worked with mod_php but not CGI of FCGI • Noticed the following errors in /var/log/secure. This told me that FCGI was not able to set the correct runuser, resulting in the 500 error- • More research showed that the suexec binary (/usr/bin/suexec) had 710 permissions. Changing permissions to 4510 resolved the problems

Any idea how this could have happened ?

Anto

I think you had a permission problem in the extended attributes that aren't saved and restore correctly by backups. I had the same problem in the past (CentOS 7.0). And every time I move or clone a VM (openVZ).

https://www.virtualmin.com/node/35153

Regards, P.

Looking at our CentOS 7 system here, it looks like those are actually the correct permissions for suexec. That is, suexec is 710 on this server as well, which is using FCGID.

That is a bit unusual that you're seeing that issue there, though I'm happy to hear things are working properly for you now. It's not clear to me why this issue would start occurring for you all the sudden after a restore, unless it is as PaulVM mentioned above... some form of extended attribute had been setup on the filesystem that wasn't included in the backup.

I'm curious, what is the output of this command -- this will show what your Apache version is (just to make sure you have the version Virtualmin expects):

rpm -qa | grep httpd

rpm -qa | grep httpd httpd-2.4.6-31.el7.centos.1.vm.x86_64 httpd-tools-2.4.6-31.el7.centos.1.vm.x86_64

Is ssh for guys a better way to look into this ? i am actually worried if the current permission on the suexc is less secure ?

Perfect, your server has the correct Apache version.

Those permissions are okay though, I wouldn't be concerned about that. Although that's not what CentOS 7 uses by default, other distro's do. That's what my Ubuntu systems using, for example.

We may want to test to make sure Virtualmin isn't seeing anything else unusual though... if you log into Virtualmin, and then click System Settings -> Re-Check Config, can you paste the output you receive into the support ticket here? Thanks!

The status of your system is being checked to ensure that all enabled features are available, that the mail server is properly configured, and that quotas are active .. Your system has 988.17 MB of memory, which is at or above the Virtualmin recommended minimum of 256 MB. BIND DNS server is installed, and the system is configured to use it.

Mail server Postfix is installed and configured.

Postfix can support per-domain outgoing IP addresses, but is not currently configured to do so. This can be setup in the Postfix Mailserver module.

Apache is installed.

The following PHP versions are available : 5.4.16 (/bin/php-cgi), 5.6.5 (/opt/rh/rh-php56/root/usr/bin/php-cgi)

Webalizer is installed.

Apache is configured to host SSL websites.

MySQL is installed and running.

ProFTPD is installed.

Logrotate is installed.

SpamAssassin and Procmail are installed and configured for use.

ClamAV is installed and assumed to be running.

Plugin AWstats reporting is installed OK.

Plugin DAV Login is installed OK.

Plugin Protected web directories is installed OK.

Plugin Additional content styles is installed OK.

Plugin Additional content styles from OpenWebDesign.org is installed OK.

Plugin Virtualmin Support Links is installed OK.

Using network interface eth0 for virtual IPs.

IPv6 addresses are available, using interface eth0.

Default IPv4 address for virtual servers is 109.104.118.173.

Default IPv6 address for virtual servers is 2001:9d8:1000:1a2e:216:3eff:fe1d:59f6.

Default IP address is set to 109.104.118.173, which matches the detected external address.

Both user and group quotas are enabled for home and email directories.

All commands needed to create and restore backups are installed.

Resource limits are supported and configured correctly.

The selected package management and update systems are installed OK.

.. your system is ready for use by Virtualmin.

Hey, guys, i am still having some weird issues with my Virtualmin sites

One of my WP sites have some weird permission issue, it asks for an FTP cred while adding and deleting plugins, but not for my other sites on the same server, these are the things that I have done so far with my server inorder to fix the initial 500 error. And there is some weird permission issue with WHMCS as well 1.More research showed that the suexec binary (/usr/bin/suexec) had 710 permissions. Changing permissions to 4510 resolved the problems 2.Changed /home/username permissions to 711 from 750 to prevent cross-account file access.

some digging has made two points stand out from the WordPress blog "WordPress asks for your FTP credentials when it can't access the files directly. This is usually caused by PHP running as the apache user (mod_php or CGI) rather than the user that owns your WordPress files."

Response from WHMCS Support For folder permissions, "it is dependent upon the server configuration and what it needs for a folder to be writable by the user handling the incoming HTTP request. For example: on a standard cPanel server where Apache runs as the unprivileged "nobody" user, the folder needs to be 777 to ensure anyone (not just the owner, the cPanel account holder) can write to it. If you've configured Apache to handle requests as the same user as owns the folder (using something like SuEXEC or SuPHP), the permissions can be lower (usually 755). Again, the required permissions depend on your server - not WHMCS - and not something we can answer for you."

I have been working with a guy to help me out, but i have reached a dead end and i am in desperate need of some help, I am launching a new product and has been behind schedule for a month, the emailing back and forth and coordinating with multiple people is getting quite frustrating

What does it take to get someone from your team to SSH into my server and resolve this for once? i am ready to buy a support incident for one hour for a 1:1 session

please let me know

thanks Anto

Would it be possible for us to login to your system to see what's going wrong?

yes please

cam i email you the cred ?

What if my business is losing money and my business is dependent on your product, and I need this fixed now ? is there some kind of support service you provide where I can just have you remote into this within a specified time.

How would I get that kind of support?

for, e.g., when I email WHMCS they will shh or FTP into the server and fix it no less than 6 hours, I mean something like that?

Since there's quite a bit of history on this bug, can you summarize which domains / URLs are currently broken?

Ok, my domain olexor.com username arappai has the issue.It keeps asking me for an FTP cred when installing or deleting a plugin, also i installed a plugin called wordfence and it says it has no permission to write to the wflogs under the wp-contents folder.

but there is no problem with any of my other sites, i double checked the permissions and it is owned by arappai, it is not broken, i just can't add any plugins and activate the WordPress wordfence firewall.

chcek post #17

also, check out post 1 and 11 in this thread

Ok, I had a look and it seems like your PHP scripts are being run as the apache user, not the domain owner.

Looking into why ...

I'm mystified actually - have you made any special Apache configuration or modules on this system?

Jamie is traveling at the moment, and hasn't been able to provide me with the login credentials.

Would it be possible for me to log in and take a look? If so, could you send credentials to eric@virtualmin.com?

After reading Jamie's account of the issue, I got an idea as to what might be causing that problem.

not that I know off , I tried messing with Clam AV but that was long before I had the problem, the only thing I can think of is the VM restore , and I tried permission change on the /home/arappai to 711 and back to 750 and every other change is mentioned in the doc I send to Jamie.

Then the updates, the first symptom was none of the sites would load, it would just show 500 error then I had a guy look at it and he did this

-More research showed that the suexec binary (/usr/bin/suexec) had 710 permissions. Changing permissions to 4510 resolved the problems

I would like it reverted to its original state if we find the cause. I am just too paranoid about security.

I installed WordPress plugin called wordfence , but i really never had any issues until the restore.

Can you test if the problem is still occurring with the WordPress installation for the arappai user? For example, is WordFence still giving you that same error?

yes i still have the error on the Wordfence and still asking for the FTP to delete the plugins

Anto

Jamie and I have both been looking into this.

It's unfortunately going to take some time -- we've never seen anything like this before, and neither of us have been able to determine the cause as of yet.

But I did want you to know that we're reviewing the issue.

I think we made some progress.

Can you take another look now, and see if that makes a difference?

hey i can not get into two of my sites my main site olexor.com and madific.com , it shows an internal 500 error

Hmm, that was all working when I posted my update earlier today. Do you know anything that might have changed since then?

nothing at all, i just changed the ClamAV script to run at midnight 12:00 am and send me a report at the same time

nothing else

Okay, it's unfortunately about 1:30am in my timezone, so I'll be heading to bed here shortly. I explained what is going on to Joe in case he has some ideas, as he's in a different timezone -- but so far the three of us have been really puzzled as to what's going on.

You're experiencing an issue none of us have seen before.

It unfortunately may end up needing to wait until tomorrow sometime.

Okay, one last attempt before I need to head to bed -- is that working for you now?

I "pressed a few buttons", as they say, and it seems as if it's working now.

ok the site is showing up now , let me check if the permission issue is sorted out now and i understand you have to head to bed now , thanks for looking into this.

hope you guys figure it out soon

thanks Anto

Got damn it is working now. Thank the dear lord baby Jesus and thank you guys.

If you could after you wake up take the time to give a report as to how this happened and what was done to fix it I can keep a record of it.

Agreed, both sites appear to be working for me still, which is great news.

I'm not entirely certain what is going on still, we'll talk it over more in the morning.

However, so long as it's working that's a good start :-)

Do let us know if the plugin permissions are working as expected. We made a number of changes today that should affect that, we're hoping those will work properly for you.

/home/madific/public_html/test.php: {HEX}php.exe.globals.403.UNOFFICIAL FOUND /home/arappai/public_html/test.php: {HEX}php.exe.globals.403.UNOFFICIAL FOUND

----------- SCAN SUMMARY ----------- Known viruses: 4396462 Engine version: 0.99.2 Scanned directories: 18805 Scanned files: 119330 Infected files: 2 Total errors: 2 Data scanned: 2531.70 MB Data read: 8289.07 MB (ratio 0.31:1) Time: 344.089 sec (5 m 44 s)

ok i found these in my clam scan results , you think this has something to do with the cause ?

yes i meant the plugin is working too and no more FTP permission issues

Joe's picture
Submitted by Joe on Tue, 05/24/2016 - 01:03 Pro Licensee

So, Eric gave me a bit of a briefing on what he changed to make it spin again before he hit the sack (we've all been kinda stumped by this issue). He changed the suexec binary to be setuid. Now, this is ordinarily not required on a CentOS 7 system with the stock kernel and capabilities configuration; so it shouldn't have fixed it, but it did. So, that made me ask whether you have a different kernel than stock. Turns out, you do!

Your system has a kernel from CentOS Plus, which has a variety of differences from the stock RHEL/CentOS kernel. We don't test against that kernel, and unless there's a specific feature you need in that kernel, I recommend running the stock CentOS kernel. I don't know that this kernel is the source of this trouble, but when Eric said changing suexec to be setuid made it work again, I immediately said, "I bet he's got a different kernel."

We're gonna do a bit more research on this, as I don't know anything about the CentOS Plus kernel. I see it has TOMOYO enabled, which is a mandatory access control system (comparable to SELinux), among other things. I wouldn't be surprised if one of those patches or additional enabled features is preventing the capabilities from working right for suexec.

In short: I want to blame your kernel or capabilities configuration for these problems. If you can use the stock kernel, I'd recommend it. If you can't, we'll need to figure out what CentOS Plus is doing to capabilities. There are some security considerations to running it setuid (it's the way it was run for many years before capabilities support in the kernel was widespread, so it isn't the craziest thing to do, but if we don't have to, we might as well not).

There may be other stuff going on; a scan of this ticket makes me think there are multiple issues. But, at the very least, setuid making suexec work makes me think capabilities (or at least the one suexec needs) is not working right on this kernel.

oh my, thanks that was some good info, so couple more questions. Is post number 42 an issue? It does not look like to me

Secondly, my knowledge of Linux is just weak, the most I have learned is in the past few days. What does it take to switch to the stock kernel ? is that something you can assist me with ? or I need to find a third party , I want to go as per your recommendations.

Thirdly i noticed the public_html on arappai and madific has permission set to 751, is that secure ?

I haven't done any Kernel switching, I wonder if it was the default image provided by my hosting provider ? it probably is

thanks

Joe's picture
Submitted by Joe on Tue, 05/24/2016 - 02:00 Pro Licensee

With regard to #42: That may indeed indicate a security problem. Because it is in public_html, if you don't know where those test.php files came from, and don't recognize their contents, I would assume your web application (WordPress maybe?) has been compromised. Making sure you're running the latest version would be a start on preventing future problems. Getting rid of those files (after making a copy somewhere safe for later study to figure out what they do), would also be recommended. I think Eric mentioned you'd migrated these domains from a cPanel system; those files may have existed before the migration.

With regard to changing the kernel, you'd want to check with your provider to be sure they haven't chosen the CentOS Plus kernel for a reason; they may have seen the "Plus" and thought, "That means it's better!" But, really it just means it is vastly less well-tested and has a bunch of extra stuff that is very rarely used (like ReiserFS and TOMOYO). I wouldn't want to rely on it for a production system without a really good reason; but your host may have a really good reason (maybe hardware drivers for network or similar). Kernels are out of scope of our support; it's not an area we work in enough to have confidence with, so we don't poke at them. I only make the recommendation to switch to stock in this case because I strongly suspect we are seeing bad behavior from the kernel you currently have. Though my general advice with regard to all packages is, "Stick with the stock OS package until you know you need something else, and you know exactly why you need something else and understand the implications of switching." The CentOS Plus repo docs say this: "Using the CentOSPlus repository is more dangerous than using other CentOS repositories, as it is designed to have several updated packages and it is not really designed to be completely enabled."

Actually, looking deeper, I see you won't be able to change the kernel within your VM, as it has been booted from a kernel on the host machine. You'll need to talk to your hosting provider about it, if you wish to change it (unless they provide a method for selecting boot kernel). If the configuration you have now doesn't cause any troubles going forward, you may want to leave it (as I mentioned, suexec ran the way Eric has configured it for years...it is quite new that it can run without being setuid, and many distros still run it that way).

And, 751 permissions on public_html probably came over from cPanel, which has a slightly different security model from Virtualmin. We default to 750. It should be safe to switch to 750. I'm surprised our migration doesn't do that automatically, but there may be some implications to doing that I don't know. I'll talk to Jamie about it at our next meeting.

i could have sworn that it was 750 till yeaterday, and the contents of the test .php are below and i was pretty sure it was not there either

<?php
system("id -a");
phpinfo();
?>

what would be the out put of that command ?

one more question is 711 for the /home/username better than 750 ?

If things are working now, you can delete the contents of that test.php file. That is something Jamie and I used for testing in order to determine what userid the script thought it was running as..

As far as permissions go -- if you wish to try to tighten them up, you could always try making /home/USERNAME "750", and then see what happens.

After making that change, be sure to restart Apache. Otherwise, the cached FCGID processes may make it appear to be working when it isn't.

Ok , figured the test.php was done by you guys

But just wanted to double check if you guys had changed the permission on the /home/arappai to 751 ?

For now I have changed the permission back to 750 .

just FYI, I think the reason the provider used the other kernel was for some OnApp and Hostbill integration , still yet to confirm it though .

I will restart the Apache and check it out.

Thanks guys I am back in business .

One more thing regarding the the setuid, is that something you change for each virtual server site user ?and just a brief outline of the steps you did for changing this , so if something goes wrong I can be prepared

Because what puzzles me is that I had absolutely no problem with the other sites

Maybe the host name of the server and the website being same ? That is Olexor.com ?

Maybe using cloudflare DNS ?

Yeah we had changed those directory permissions in our troubleshooting.

Are things working properly with them set to 750?

The setuid on suexec was set once, on the file located in /ust/sbin.

That's something you had set previously, and was one of the first things we unset since it, in theory, shouldn't be necessary.

We actually got things working with that unset.

However, for reasons I don't currently understand, that's no longer working, and appears to be requiring that suexec be set to suid.

While we do recommend setting the hostname to something in the format of host.domain.tld, that wouldn't cause the problem you're seeing. Those issues shouldn't have anything to do with your hostname.

Another possibility for why the other sites worked, is that the other sites could be configured to use a different PHP Execution Mode. Do you happen to know what they're set to?

ok, I restarted the Apache and set the permission to 750 and all still works - in fact, the best working condition since the migration, the site that was working while olexor.com was not was techshebang.com and the PHP execution was set to FCGid same as the others.

So all you had to do was change the permission on the suexec to 6755 ?

Again I wonder how this could have happened ? do you want me to do a snap shot and restore to see if this problem happens again on another server |?

ok one more thing , i just wanted to confirm that one you guys renamed the .htaccess files under arappai ? because today when i checked i realized the permalinks was not working on my site olexor.com

but it is working now because i changed it back

We actually spent quite a bit of time combing through the Apache config and making a number of changes there. I'm unfortunately at this point not sure which changes it was that resolved the issue.

The change to suexec was after all the above. Once we got things working after modifying the Apache config, I tried removing the suid bit from suexec to see if that would work.

It did appear to be working at that point. Though I suppose it's possible it had gotten cached somewhere instead.

When you indicated there was a problem, we put the suid bit back on suexec, and things have been working the way they are since then.

My apologies for the .htaccess file change, we temporarily renamed that during our testing to rule out any strangeness with it (we actually get quite a few problems with htaccess files).

hey Just FYI on why the CentOS plus kernel was used

CentOS 7.x templates use centosplus kernel following issues with missing initrd lines when upgrading with yum.

We've decided to switch into the CentOS Plus's kernel. Corresponded repositories was enabled/added to install/upgrade (include) the kernel-plus* packages only. Native RHEL/CentOS 7.x kernel* packages were removed. The grubby was downgraded and excluded from the repositories to disable its upgrade until the initrd issue is fixed. RHEL/CentOS 7.x templates were updated to the recent 3.10.0-229.14.1.el7.centos.plus.x86_64 kernel

One would have to disable the kernel-plus packages and re-enable the native kernel packages as per CentOS / rhel docs to use the default kernels.

Hey so i just reverted my snapshot to another VM with a different IP, and I changed the IP address of the Virtualmin site as well, but guess what I have the same issue of 500 error, I had this problem on another server and you helped me with it, however I am unclear on the exact steps to resolve this , can you look from post no:35 and tell me how to set the user permissions

At least we can confirm that this is caused by the restoring the VM from snapshot and also

Thanks Anto

There were a few different issues going on, but one of them was that the username and group number in the "SuexecUserGroup" line of Apache was incorrect for that domain.

A good place to start would be to take a look at that line for the domain that's failing, and ensure that the userid and group id are correct for the Virtual Server owner.

so the owner is correct of that domain , now what ?

if you look at post 44, that may be what i need to do , and how do i do that ? this is the exact same scenario as the previous

Are you asking how to change kernels?

Is this a VPS, or a dedicated server?

If it's a VPS, you should be able to use the control panel provided by your provider to change kernels.

no I mean this part "He changed the suexec binary to be setuid" that is what you did to fix right ?

Anto

I had to make quite a few changes to your system to get things working.

That included directory permission changes, Apache config changes, and changes to the suexec binary.

We can certainly double-check the suexec binary permissions though -- what is the output of this command:

ls -l /usr/sbin/suexec

-r-x--x--- 1 root apache 15352 Jun 2 20:22 /usr/sbin/suexec

thats the out put

Ah yes it does appear that was reset.

You can put that back to the way it was by running this command:

chmod 4755 /usr/sbin/suexec

Let us know if that helps!

Hey, it worked, wait, so that was the culprit of the previous Virtualmin server error too? When I restored

Thanks Anto

We had to change quite a few things to get things working when we logged in previously.

We did notice at that time though, that your system would not work unless suexec was set to use those permissions.

Ok thanks man

Back in business

Anto