After 10/18 webmin/virtualmin module update, multiple things seem broken with virtualhost creation

82 posts / 0 new
Last post
Wed, 10/23/2019 - 15:17 (Reply to #51)
Dibs

I meant do it thru Virtualmin (with NewDomain2 in the drop down) - Server configuration - SSL Certs - Lets' encrypt: check there 2 domains associated with it (NewDomain2.com & www.NewDomain2.com) & hit request.

Wed, 10/23/2019 - 15:36 (Reply to #52)
hoogs

Hmmm, well that's peculiar.

For some reason I was blind to the "Server Config - SSL Certs" menu entry

https://Server_IP_Addr:10000/virtual-server/cert_form.cgi?dom=157171039118361

so I went to "Edit Virtual Server - Enabled Features" and checked " Apache SSL website enabled"

https://Server_IP_Addr:10000/virtual-server/edit_domain.cgi?dom=157171039118361

It chugged away, and Let's Encrypt issued certs for NewDomain2, www.NewDomain2, and mail.NewDomain2. (NB -- if you should happen to use Cloudflare some time, be sure when you do the SSL request that CF proxy is disabled or it will fail.)

Then ... [drumroll] ... https://NewDomain2/ worked (??) without the ERR_TOO_MANY_REDIRECTS message from the last few days.

This doesn't make sense. I've not been fiddling with anything, so why / how would the server "magically" fix itself?

I looked in httpd.conf, and the Rewrite rules were NOT included in the :443 block. (They remain commented out in the :80 block.)

Wed, 10/23/2019 - 15:58 (Reply to #53)
Dibs

You could try a few things if you are after more info:

  • Disable the SSL config in Enabled features, confirm the SSL block is NOT in httpd.conf, uncomment the 4 lines (rewrite rules) in the non-SSL block, restart Apache, and then enable the SSL config in Enabled features and check the httpd.conf file to see if the rewrite rules are there in the SSL VirtualHosts block. If they are, that would suggest that when you enable SSL , Virtualmin basically copies the non-SSL block and adds a bit to it or changes a bit.

  • Disable SSL config, confirm SSL block isn't in httpd.conf (rewrite rules are commented out in the non SSL block), enable your CF proxy, enable the SSL config, request the SSL cert using Virtualmin and then see if you get the TOO_MANY_DIRECTS? If you do then it confirms that the SSL cert needs to be requested 1st, then CF proxy enabled. EDIT: Ignore this one, you did say with the CF proxy enabled, the cert request fails.

Hope it makes sense?

Wed, 10/23/2019 - 14:57 (Reply to #54)
hoogs

All I know about the 10/18 "update" was that it said there were 2 modules -- one for Webmin, the other for Virtualmin. There were no other packages noted.

Hence, I'm flummoxed why the devs won't tell me how to back out those 2 module updates so I can prove / disprove that they are the source of the issues.

Wed, 10/23/2019 - 15:52
Dibs

One last thing to check:

  • Disable your CF proxy, uncomment the rewrite rules in both SSL block and non-SSL block, request a LE cert for the 3 hosts for NewDomain2.com, check if both .php & .html files will display?
  • then enable your CF proxy and check if both files will display?

I wonder if and I could be wrong - it's to do with the CF proxy being enabled before obtaining the certs for SSL. Hopefully the few tests I've mentioned might shed some light on what is going on. EDIT: You did mention the cert will fail to issue if CF proxy is on.

Wed, 10/23/2019 - 16:02
Dibs

You said you hadn't noticed the "Virtualmin - Server Config - SSL certs" option, so when you originally created NewDomain2.com and enabled (by default) the SSL option, what cert were you using?

If you can't remember - create a new domain (for test purposes), enabling Website & SSL - then go to Virtualmin - Server Config - SSL Certs: and check the Current Cert tab. I'm wondering if it's a self signed cert, or no cert.

Wed, 10/23/2019 - 16:15
hoogs

This makes no sense at all... NewDomain1 is failing completely now.

I first tried backing out the SSL block, and it failed ... then I gave up, deleted the whole domain and rebuilt it ... failed.

From Opera and Firefox browsers I get "Forbidden, you don't have permission to access / on this server"

From Chrome I get ERR_TOO_MANY_REDIRECTS

uhhh.. wut?

I'll try these next:

  1. delete NewDomain1

  2. remove all CloudFlare proxies. (DNS stays but that resolves correctly.)

  3. recreate NewDomain1 (http)

  4. check to see if browser connect works to reach http://NewDomain1

4a1. If not, check httpd.conf, and comment out those ReWrite rules again (assuming they're there)

4a2. Check again to see if http://NewDomain1 works

4a3. If not ... give up?

4a4. If so ... then onto 5

  1. Assuming connection to http://NewDomain1 works, then "Enable Feature SSL"

5a. test to see if browser connect works

5a1. If not, check httpd.conf, and comment out those ReWrite rules again (assuming they're there)

5a2. Check again to see if http://NewDomain1 works

5a3. If not ... give up?

Wed, 10/23/2019 - 16:21 (Reply to #58)
Dibs

for 5. Assuming connection to http://NewDomain1 works, then "Enable Feature SSL" - I would request the LE cert too., then test.

Wed, 10/23/2019 - 16:17
Dibs

For Chrome - I tend to use Incognito mode when testing stuff like this. Does't keep cache etc.

Wed, 10/23/2019 - 16:19 (Reply to #60)
hoogs

Yeah, I generally use Incognito mode all the time with all browsers, except when I connect to Chrome webstore (as it won't let you install plugins in Incongito mode)

Wed, 10/23/2019 - 16:18
Dibs

Did you have a LE cert for NewDomain1?

Wed, 10/23/2019 - 16:19 (Reply to #62)
hoogs

Not now that the domain is deleted. ;)

Wed, 10/23/2019 - 16:20 (Reply to #63)
Dibs

ROFL.. I meant before you deleted it & it played up. Might be hard to answer now that's its gone. LOL

Wed, 10/23/2019 - 16:26 (Reply to #64)
hoogs

[HeadDesk] [Scream]

[From Opera] Your connection is not private This server could not prove that it is NewDomain1; its security certificate is from DefaultDomain. This may be caused by a misconfiguration or an attacker intercepting your connection.

NET::ERR_CERT_COMMON_NAME_INVALID

[From Chrome] Forbidden You don't have permission to access / on this server.

EDIT1: I had commented out those ReWrite rules ... no effect

Wed, 10/23/2019 - 16:28 (Reply to #65)
Dibs

Is that with a LE SSL cert or nothing, i.e. the self signed cert?

Wed, 10/23/2019 - 16:30 (Reply to #66)
hoogs

Nothing.

(No cert at all. Only :80 virtualhost block created.)

Wed, 10/23/2019 - 16:33 (Reply to #67)
Dibs

Any files in there? I suspect not. If not - drop the test .php & .html files in the root.

Wed, 10/23/2019 - 16:48 (Reply to #68)
hoogs

OK, both index.html and myinfo.php are now in NewDomain1/public_html , and work properly.

httpd.conf v-block for NewDomain1 does not have the ReWrite rules commented out. (Nothing commented out.)

I'll see about adding SSL site now via "Edit Virtual Server - Enabled Features":

https://Server_IP_Addr:10000/virtual-server/edit_domain.cgi?dom=157186493720342&xnavigation=1

Wed, 10/23/2019 - 16:51 (Reply to #69)
hoogs

SIGH.

Error signing certificate: 429 Error creating new cert :: too many certificates already issued for exact set of domains: [domainlist] : see https://letsencrypt.org/docs/rate-limits/
Wed, 10/23/2019 - 16:58 (Reply to #70)
Dibs

Have you tried changing the "exact set of domains" - so use - NewDomain1.com - www.NewDomain1.com - admin.NewDomain1.com

basically stick anything in there alongside the www? So it's not exactly the same as last time or the last few times.

Wed, 10/23/2019 - 17:13 (Reply to #71)
hoogs

We're in deep weeds now... guess I'll have to wait a few days. :'/

testcert.NewDomain1.com challenge did not pass: Invalid response from DefaultDomain.com/.well-known/acme-challenge/xg7oI4vHWnuA8-Y1crQkuRcf4TVrojOiZkG1xfDvG0c : "\n\n\n\n<meta http-equiv=\"Content-Type\" c"

Wed, 10/23/2019 - 17:17 (Reply to #72)
Dibs

As a last ditch attempt - maybe testcert.NewDomain1.com needs to exist. Try creating a subdomain called testcert for NewDomain1.com. Basically same as create New Virtual Server except at the top select it to be a subdomain (subserver) of NewDomain1.com.

Whether it works or not isn't relevant I think - when LE checks it should be there.

EDIT - or maybe just request it for the single domain for now, i.e. www.NewDomain1.com on it's own? Previously you've done root domain, www & mail, so www on it's own isn't a duplicate. And might allow you to carry on with the "testing".

Wed, 10/23/2019 - 17:53 (Reply to #73)
hoogs

Requested for NewDomain1.com, and www.NewDomain1.com. (Dropped mail, as I don't use mail on this server anyway...)

LE approved that.

Wed, 10/23/2019 - 16:21 (Reply to #74)
hoogs

One concern I have is that after a couple tries, LetsEncrypt stops issuing certs. ("Too many requests from this domain.")

I tripped over that problem once, due to a -very- slow server host. After a few days, tried again; then LE issued it.

Wed, 10/23/2019 - 16:25 (Reply to #75)
Dibs

Just saw a post on a LE forum about some guy getting that "too many requests" message but he did 5 identical requests in a week. Maybe leave the "mail" host off - so it isn't an identical request. Or if you get a cert - make a copy of it, you can then copy it back?

Wed, 10/23/2019 - 17:12
Dibs

Saw the following on a LE forum post

… or you have to use “loophole” in rate limits. Certificate is counted as a duplicate, if it has exact same set of hostnames. Hence, if you request a new certificate for heidelberg.yaroscloud.com and (for example) www.heidelberg.yaroscloud.com, it won’t be treated as duplicate one.

Please note that there is also a weekly limit of 20 certificates per domain.

Where the guy was trying to request for heidelberg.yaroscloud.com (for the upmteenth time)

So put www.NewDomain1.com & MadeUp.NewDomain1.com in the hosts fields for the LE Cert and see what happens?

EDIT - put the domains in the cert that you require and the MadeUp one as well. That way you will be covered for the required domains & a non-existent one, which should be no headache. Then in 7 days time request a new one without the MadeUp one. Otherwise you could hit the weekly limit of new certs.

Wed, 10/23/2019 - 18:28
Dibs

So if LE supplied the SSL - does that mean that the .html & .php files now display correctly on NewDomain1.com (both HTTP & HTTPS)? Assuming the CF proxy is still turned off. It files display correctly with the CF proxy off - what happens if you turn the proxy on?

[Make a copy of the cert just in case. Don't put it in a directory that belongs to that VirtualHost ; ) ]

Wed, 10/23/2019 - 19:59 (Reply to #78)
hoogs

Somebody shoot me, I'm in (virtualmin) Hell.

I tried fiddling manually with the main public_html dir (minor stuff) and suddenly the website started giving me ERR_TOO_MANY_REDIRECT errors again.

So... after futzing with multiple variations of editing httpd.conf and then adding/removing website options via Virtualmin -- and consistently failing -- I saved the SSL cert file and blew away NewDomain1 and remade it, and put the SSL certs back in.

FAIL again.

This led me down the well-trod path of deleting / recreating NewDomain1 and consistently #FAIL'ing again.

I have NO idea what is causing these problems. I just know that ever since 10/18 nothing works. At least, not for long.

SCREAM
Wed, 10/23/2019 - 22:58
Dibs

I can sympathise with your plight. What I would say is that it would be best to go back to your 5 point plan (sort of).

  • Turn off your CF proxy & leave it off until you get a successful result. [?]
  • You need to get a basic webpage to display in html & php. Just HTTP is fine, so (re)create a VirtualServer for NewDomain1.com with just Website Enabled. [?]
  • Place your .php & .html files in the public folder & test that they display. [?]

Without this - no point going further. Assuming you are successful at this stage: [?]

  • enable the SSL for the website. [?] Then in SSL serts (Server Conf) look at the current cert - it should be a self signed one. [?] Note it's location and check (and note) the persmissions. [?]
  • Place your backed up cert there and ensure it's permissions are the same. [?] Even make the file name the same for now if need be. [?]
  • Go back to Server Configs - SSL certs & check that Current Cert's details are as expected, i.e. issued by LE & the time\date should marry up to when you requested it prior to saving it. [?]
  • At this point test via HTTPS for the .html file & .php files - that they display.[?]
  • If all is well, switch on your CF proxy and test it works. [?]

You may find the following useful - https://stackoverflow.com/questions/41583088/http-to-https-nginx-too-man... - I know it isn't about Apache, but the CF proxies and SSL certs and how they interplay might be useful.

Give it a go and use the above tasks in the bullet points above and reply back adding some commentary to the bullet points, i.e. use them as a bit of a check list? Feel free to cut & paste it back into your reply. I've left [?] place holders for a Yes or a No, or a fuller answer if you feel that adds to things.

HIH

p.s. As the saying goes "The morning is wiser than the evening." - might be an idea to leave it for a day and then come back to it. ;)

Thu, 10/24/2019 - 03:36
Jfro

Take care of caches everywhere. Sometimes with ssl / redirects config files paths those are giving me ... and not all are gone after cleaning browser cache.

( how this exposed me, things seems to work , after a while but much longer then a changed config file with pats or redirects a got failures , before some caching was responsable to have it look like it works well. )

Also if error it happened to be there while caching false config to long, even after getteing it done right.

I try alway's also on other computer after that experience to be more sure, .

Cachings can take place at a lot of places ( also proxies / dns and so on.)

When installing / configuring server / websites / cms it is best praxis to switch as much of them off.

( only pointing out , not saying ...)

Firewall and such rules somewhere.

IN software as cms itself no strict enough same paths/redirs ( http<>https , docroot and so on)

So cache not the source for error, but source for hard to find such errors after changing settings / configs if changes had take place in between cachings.

Another maybe important thing to take care of is LETSENCRYPT V1 api is for new creates EOL / brown out time... Don't know how far the v2 api in virtualmin is see topics. https://www.virtualmin.com/comment/818604#comment-818604 https://www.virtualmin.com/comment/818433#comment-818433

And ofcourse TTL times in DNS / bind / registrar > patience.

And don't use cloudflare if testing your server / cms config tot start with at all, first things first while. (CF:Provisioning typically takes around 15 minutes for paid plans and up to 24 hours for Free) so every change..............

Wrong www<> non www redirects...

You can always try in between with self signed certs to test.

So i can't help you out but pointing above some .... things to keep n mind to. ;)

Please post here cause and solution?

Fri, 10/25/2019 - 15:13
adamjedgar

how many domains on this server? If it's a vps, personally I think you are better off blowing it away and creating a new one...all this time stuffing around you might have had new one up and running and domains restored.

AJECreative is the home of $5 webhosting, $15/month VPS servers (1cpu,1gb RAM, 25GB storage)
Centos7, Debian9, or Ubuntu18LTS
Available Control Panels = Centos-Webpanel, Cyberpanel, or Virtualmin

https://ajecreative.com.au

Pages