Memory and Virtual Memory running at max capacity

13 posts / 0 new
Last post
#1 Sun, 08/16/2009 - 22:09
agatohuang

Memory and Virtual Memory running at max capacity

Hi

I just rolled out my Ubuntu 8.04 server with Virtualmin installed live. The server memory (756mb) and virtual mem shot up when everyone started to log in in the morning.

Now the server basically hang. It was ok when I tested the server with 1 or 2 user.

But now we have about 50 active user running at the same time. Can somebody please help!

Thank You!

Sun, 08/16/2009 - 22:19
andreychek

Howdy,

Well, first off, it's possible that your usage requires more than 750MB of RAM.

That said, there's some things I might look into:

  1. Verify that you have swap enabled; not having swap with that amount of RAM could cause some trouble

  2. Disable any services you don't need -- that also includes PHP modules that you aren't using

  3. If you're using FCGID, you may want to switch to CGI to save RAM

  4. If you're using SpamAssassin and ClamAV as daemons, you might want to look into moving them to the standalone versions. That will require more CPU usage, so I'd look into the other options first if you can.

Those are some tips to get you started, let us know how that goes!

-Eric

Sun, 08/16/2009 - 22:51
agatohuang

Hi Eric

Thank you for the reply.

Do you think it is the memory not enough issue ? Is Virtual Memory same as swap file? Because I have 1 GB virtual memory enabled.

Or is it email setting issue. I have the following error in Email.log - Server reached -- max-children setting - do not list domain example.com in both mydestination and virtual_alias_domain.

I'm now in LIVE situation and very desperate. Hope you can stay with me a while.

Thank You!

Sun, 08/16/2009 - 22:48
andreychek

Well, I'm not sure what the problem is, I'm just making some suggestions after hearing you mention memory usage :-)

756MB isn't all that much these days compared to what some daemons use.

Regardless of what the problem is, I'd certainly do items #1 and #2 in what I mentioned above. Those would help in any setting.

But, the exact issue could be a range of problems, it depends on what resource is being consumed, and by who/what.

You may want to log in via SSH and run "top" to get an idea of what processes are using up a lot of CPU.

While you'e logged into your box there, what is the output of these two commands:

  1. uptime

  2. free

Sun, 08/16/2009 - 23:16
agatohuang

Hi Eric

I reboot the server and I disconnected the lan cable. Still the memory and virtual memory status is running very high.

1) Yes I have 2 GB of Virtual Memory if this is the same as swap file.

2) I have disabled spamassassin and I will try to disable other unused daemon.

The 2 most frequent process is collectinfo.pl running at 2-3% of cpu.

And Monitor.pl running at 2-3% of cpu.

1) Running uptime gives - 11:54:22 up 2:07, 1 user, load average 22.28, 24.40, 25.64 .

2) Running free gives - Mem Total=774372 Used=766280 Free=8094.

Swap Total=2265124 Used=1162604 Free=1102520

Sun, 08/16/2009 - 23:27
andreychek

Yeah, it looks like you're out of RAM; that probably accounts for at least some of the high CPU usage (as swap goes up).

I'd definitely suggest disabling anything else you don't need.

A few other things I'd do:

  1. Go into System Settings -> Features and Plugins, and disable "Status monitoring" (temporarily, at least, to keep monitor.pl from running)

  2. In System Settings -> Module Config -> Status Collection, and set "Interval between status collection job runs" to something higher -- perhaps 60 minutes, 120 minutes, or some other number. It's not important, it's just for the stats in Virtualmin.

Sun, 08/16/2009 - 23:50
agatohuang

But the mem usage was just between 200 to 300 MB before I go live today.

I have been testing the server for 1 month now.

The problem only started this morning.

So it is the user loggin in to access the email that causes the problem?

Mon, 08/17/2009 - 00:02
andreychek

So it is the user loggin in to access the email that causes the problem?

I'm just throwing out suggestions for where to look, and thoughts on how to resolve it. There's too many variables for me to make any conclusions as to whether it's related to email, or something else :-)

Yeah, RAM usage will go up whenever the server is being used more heavily.

It takes more resources to support 50 users than 1-2.

If it's related to email, it's probably due to either spam or virus processing of incoming messages, rather than just users checking their messages. Checking email isn't very resource intensive.

So one option is that you're getting so much email, the system is struggling to keep up.

For that, you might consider enabling Greylisting -- you can do that in Email Messages -> Email Greylisting (if it's not already enabled). That will really cut down on unnecessary emails.

Another option is that your web server is getting a lot of traffic. You might look in System Settings -> Bandwidth Monitoring -> Show Usage Graph to see if any given Virtual Server is using a particularly large amount of bandwidth.

However, in general, what you can do is review your process list -- which you can do by running "ps auxw" and "top" -- and do two things with that information:

  1. Remove unnecessary processes

  2. Review what's taking up a lot of CPU, and look into ways to reduce the usage

Mon, 08/17/2009 - 02:32
agatohuang

Hi Eric or anyone can help me?

How to disable Clamav?

Thanks!

Mon, 08/17/2009 - 08:24
andreychek

To disable them entirely, you'd go into System Settings -> Features and Plugins, and disable the Spam and Virus features.

To switch between the daemon and individual client mode, you can look in Email Messages -> Spam and Virus Scanning.

-Eric

Mon, 08/17/2009 - 15:23
ronald
ronald's picture

11:54:22 up 2:07, 1 user, load average 22.28, 24.40, 25.64

A load average of 23 with 1 user logged in is way too much. What kind of server is that, a P1 ? Or is there some script hanging?

Also in the documentation you can find ways to reduce memory usage.
go to virtualmin - System Settings - Module Config - Preload Virtualmin libraries at startup? and set it to no

In webmin - system - running processes you can easily identify what is hogging your server.

Also you didn't say what version you are running of virtualmin. should be 3.72

Fri, 08/21/2009 - 10:51
agatohuang

Hi Everyone

Thank You so much for your support.

I was running Virtualmin GPL. But I have upgraded to Virtualmin Pro.

After that, I disabled Spam and Virus features.

Everything is stable now.

Really appreciate your help. Ronald and eric.

The server is running great now and I have even added two more virtual domains to it.

Fri, 08/21/2009 - 12:02
ronald
ronald's picture

good for you!

Topic locked