FYI: no I don't have backups to restore from, so shoot me.
One of my sub-servers was running Nextcloud and it was pressuring me to be upgrading to PHP 7.3 but because I'm a hack admin, I just assumed I knew what I was doing. I knew I was in serious trouble this afternoon when I rebooted the server and BIND, ProFTPD, and SpamAssasin weren't running despite me not touching them.
I don't know if this time travel review of the latest changes will help anything, but this is what I think may be relevant to my failure.
I installed the 7.3 version from
http://ppa.launchpad.net/ondrej/php/ubuntu and because of his documentation, I added
http://ppa.launchpad.net/ondrej/apache2/ubuntu to my list of
apt sources file. I went to that virtual sub-server and altered its default PHP version. Things were going well.
Not long after that, I saw that my phpMyAdmin on an internal server managed with Webmin (not Virtualmin), was complaining about me upgrading to the new version. That also went well without trouble. I have been connecting to that internal servers database via a pass-through type sub-server on my Virtualmin managed server and I noticed the PHP version was listed as 7.0, so I thought (mistakenly) that changing it in the sub-servers PHP version settings it would alter it. When it didn't I realized it was just delivering what was processed on the internal server. About 10pm last night, I did the same thing to upgrade that internal servers PHP version as I did to my externally facing server. It was still saying that I was running 7.0 so I looked in Apache's module manager with Webmin. I saw the new version was off, so I enabled it. While I was at it I thought it would be good to do the same on my externally facing server. It turned out to be a bad idea, but I didn't know that till about midnight last night when I tried to access my Friendica server using the normal web interface. It let me see the login page but then never loaded the site. I tried deactivating the module in Apache, and that did nothing so I went to bed.
Today I tried more things including clearing browser cache and so forth. I was plagued by a variety of issues with all my sites, like PHP pages downloading instead of rendering. While trying to "Google the error message" I was led to things that made my problems worse, like adding some mime types like
application/x-httpd-php .php. I became convinced that running two versions of PHP was unnecessary, and decided to uninstall php7.2 from my outward-facing server. After uninstalling it with the
.com:10000/software/ module (with settings to purge and remove unused dependencies) I thought I was all set.
Well Apache wouldn't start, and I had to manually remove a bunch of things that got thrown into
*.conf files, like rows starting with
FCGIWrapper RemoveHandler .php AddType application/x-httpd-php .php AddHandler fcgid-script .php FcgidMaxRequestLen IPCCommTimeout
After Apache was running again, the sites were still acting wrong. I rebooted. I saw the aforementioned services not running. I became shocked to find they weren't just not running, they were uninstalled. Apparently the user-specific files were gone too, not just the software. As of right now, my PHP pages display as plaintext in the browser. BIND has no zones for all my virtual servers. I'm hoping there's a way to regenerate the BIND entries.
I don't know what else I fouled up but things were a bit wonky in Postfix too.