I had a major outage today of our main sites, resulting in an error 500 on them (FCGID mode).
I don't know when exactly this happened, but it looks that it most probably happened when i did a partial backup and restore at last minute before switching servers to synchronize quickly databases and email (and it didn't synchronize email folders!) of:
- mysql data
- Email users
for just 2 servers out of 15...
Most of the virtual servers apache suexec config lost relation with userids AND groups:
E.g. this part:
< VirtualHost 22.214.171.124:80 > SuexecUserGroup "#1036" "#1015"
should have been:
< VirtualHost 126.96.36.199:80 > SuexecUserGroup "#1010" "#1003"
So suddenly most of the 15 (actually 13, and actually not the 2 that got imported, but all others !!!) stopped working !
Also this discrepancy didn't get catched by the "Check config"s.