Virtualmin/Webmin Continually Inaccessible

  • KitchM
  • 02/25/11
  • Offline
Posted: Sun, 2011-11-13 13:20

I can accept it if the access is not available once in a great while, but I am having the problem every day now.

I am forced to reboot my container, open a secure terminal into my container root, and enter "/etc/init.d/webmin restart" to get things working again. Even some of my e-mail accounts respond that there is a "Temporary authentication failure", until this is done.

What in the world might be causing such a thing? I'm beginng to get desperate. Anyone have any ideas?

Thanks.


Howdy, Well, it's not normal

  • andreychek
  • 01/04/09
  • Offline
  • Mon, 2011-11-14 08:43

Howdy,

Well, it's not normal to have to restart a process each night. If that's what you're seeing, it sounds like a resource problem of some sort... that your system may be running out of resources, causing Webmin to die or malfunction.

What does "free -m" show on your system?

-Eric


How do, Eric. Sorry it took

  • KitchM
  • 02/25/11
  • Offline
  • Mon, 2011-11-14 15:05

How do, Eric. Sorry it took so long to reply, but here is the stuff:

mem 512 total 307 used 204 free

no shared, buffers, or swap

Should have 512 burst.

I noticed on the statistics graph that there is a time when the memory goes to zero within the last 12 hours. The average is 228.8 M. and the maximum is 524.4 M..

Thanks.


Ah, that sounds like you're

  • andreychek
  • 01/04/09
  • Offline
  • Mon, 2011-11-14 16:03

Ah, that sounds like you're using OpenVZ.

There's a file on your system called /proc/user_beancounters -- can you post the contents of that file?

Thanks!

-Eric


Attached please find the file

  • KitchM
  • 02/25/11
  • Offline
  • Mon, 2011-11-14 16:46

Attached please find the file .txt.

Thanks.


Howdy, Okay, check out the

  • andreychek
  • 01/04/09
  • Offline
  • Mon, 2011-11-14 17:34

Howdy,

Okay, check out the "failcnt" field at the far right of that output -- that field lists anytime something failed due to OpenVZ's resources limits.

It looks like your numproc and privvmpages resources have significant fail counts -- and those are likely the source of the problems you're seeing.

The "numproc" parameter describes the maximum number of processes you're allowed to run at a given time, and privvmpages describes a type of available memory.

That means some processes are being denied resources because you already have too many running processes, and in other cases because you don't have enough available guaranteed RAM.

These are all issues specific to using OpenVZ.

However, you may want to try and reduce your memory usage -- you can read about doing that here:

http://www.virtualmin.com/documentation/system/low-memory

If you continue to have problems, you may need to look into obtaining more resources for your VPS, or using a different VPS-type altogether.

-Eric


Thank you very much indeed.

  • KitchM
  • 02/25/11
  • Offline
  • Mon, 2011-11-14 19:28

Thank you very much indeed. I appreciate your clear explanation of the likely issue. I will follow the steps you suggest.

One thing I wonder about is why there is a time when there is no memory usage on my graph. That seems odd. I would think that there should always be some memory used.

Thanks again.


Followup

  • KitchM
  • 02/25/11
  • Offline
  • Mon, 2011-11-28 17:30

Here's a followup to the issue. I checked out the memory usage just now when I found I had lost access again. It states that I have 367.24 MB out of 512 MB used, with a burst of 512 MB. The suggestions given are fucused on those with less than 512. Therefore, I should have plenty.

Anyway, in trying to follow the directions regarding low memory at the supplied link, I found they are incorrect, so of little help.

Any other information would be appreciated.


The suggestions given are

  • andreychek
  • 01/04/09
  • Offline
  • Mon, 2011-11-28 20:48

The suggestions given are fucused on those with less than 512. Therefore, I should have plenty.

Jumping to that conclusion seems to not work in your case, since we've seen evidence that services are dieing on your system due to lack of memory :-)

You're dealing with low-memory situations, and need to find ways to increase your available RAM.

That document was written before OpenVZ became popular. It was written with the assumption that less than 512MB of RAM is tiny -- but it assumed you have swap available on top of your real memory.

With OpenVZ in the picture, that implies that no swap available. So not only do you have a small amount of RAM to start (512MB really isn't that much), but you also have no swap to help out.

If you search the forums here for "openvz", you'll see oodles of similar cases where folks had services mysteriously dieing, due to memory issues. Having just 512MB of RAM on an OpenVZ system often isn't going to be enough, you'll need to put a lot of effort into reducing memory usage to have a chance at getting your system to work reliably.

Anyway, in trying to follow the directions regarding low memory at the supplied link, I found they are incorrect, so of little help.

There's quite a few suggestions at that link, which ones aren't working for you? What problems are you seeing when you try?

I do see one suggestion that appears to be distro-specific, but in general, most everything listed in there is a good way to reduce memory usage.

-Eric


Sorry, I was not being clear.

  • KitchM
  • 02/25/11
  • Offline
  • Mon, 2011-11-28 23:31

Sorry, I was not being clear. As I read the article, it seemed to indicate that 512 was enough, so that should have been a question. I do not doubt that there is a memory problem because the diagnostics by reading the file info clearly showed that.

I did wonder about the swap, but if the assumption is as you pointed out, then I don't have enough memory. (I just wish the article had been clear on that point.) However, I do have swap according to my SolusVM control panel, as I mentioned above.

However, when I went to the article, it says steps at the beginning which are not what one finds in Virtualmin. Therefore, I could not follow them. I did stumble around to the best of my ability (not very good ;) ), but I didn't get very far. Obviously, I did make enough changes.

So, starting at the beginning, I quote: "To disable preloading of Webmin libraries, follow these steps :

  1. Login to Virtualmin as root.
  2. Open the System Settings category on the left menu, then click on Module Config.
  3. Change Preload Virtualmin libraries at startup? to No
  4. Click Save. You will then be prompted to re-check the Virtualmin configuration.
  5. Click the Re-Check button. Once this process is complete, the Webmin server process will reduce RAM use to about 10M."

My Virtualmin does not look like that. I can't find Module Config. So I used the manual method. Next I went to "System Settings - Spam and Virus Scanning", but again, no such thing.

I don't use clamav, and I don't believe I use LDAP. I use dovecot and mysql.

I have not tried the rest of the items as yet. But if I've already reduced from ~90 MB to ~10 MB, I must have a hell of a problem to still not be good enough.

The other issue I am confused about is that the Swap versus OpenVZ issue? How did you know there was no swap? I had assumed there was! I am very supprised to check with free -m and find none.


I went through that document,

  • andreychek
  • 01/04/09
  • Offline
  • Tue, 2011-11-29 09:07

I went through that document, and tweaked the language a little bit to help out. It now reads that folks with 512MB of memory or less, or those with a VPS without swap, may benefit from the information in there.

I also added some information on reducing ClamAV and SpamAssassin usage.

Next I went to "System Settings - Spam and Virus Scanning", but again, no such thing.

Ah, are the "Virus Scanning" and "Spam Scanning" features disabled in System Settings -> Features and Plugins? If so, that may be why "Spam and Virus Scanning" doesn't show up.

My Virtualmin does not look like that. I can't find Module Config

Aha! The term "Module Config" was changed to "Virtualmin Config" a few months back, and there's still a handful of references to the old name. I've corrected that.

I don't use clamav, and I don't believe I use LDAP.

Okay, in your case then, you can just ignore any suggestions regarding those two services.

The other issue I am confused about is that the Swap versus OpenVZ issue? How did you know there was no swap?

That's an OpenVZ limitation -- it doesn't support swap space, unfortunately. Which can really limit your available memory :-/

I have not tried the rest of the items as yet. But if I've already reduced from ~90 MB to ~10 MB, I must have a hell of a problem to still not be good enough.

Well, the trouble is that it may work fine for awhile, but then, if a couple of new processes begin running at the same time that each use up a bit of memory -- that could quickly chew through your available RAM, and push you into the unreliable burst RAM.

The other issue is that it sounds like you also have a limitation setup on your VPS regarding how many processes you're allowed to run. If you hit that number, things could go awry then. It looks like your process limit is 150 processes, and you appear to be bumping into that quite a bit too -- so you may want to determine how to further reduce the number of processes you're running.

Just for the record though, most VPS types don't carry such strict restrictions regarding resource usage. The Xen-based VPS I'm using now has swap, and has no process limits. Trying to work within such restrictions as you've been setup with can make it difficult to run a reliable server :-) But in addition to all the software changes we've talked about -- increasing your guaranteed RAM as well as your process limits may help with the problems you're seeing.

-Eric


Thanks

  • KitchM
  • 02/25/11
  • Offline
  • Tue, 2011-11-29 12:29

How do, Eric. Thanks for the good info. As I am still learning, and will be soon forced to find another host, these are important considerations. They become game-changers as far as I'm concerned. I will be particularly cognisant of the process restrictions.

Thanks again.