Email Not being delivered

14 posts / 0 new
Last post
#1 Sun, 01/04/2009 - 13:59
jsigned

Email Not being delivered

OK I think IĆ¢

Sun, 01/04/2009 - 15:11
andreychek

Yeah, something definitely sounds awry :-)

So, coming back to the ClamAV issue later (it sounds like you still have issues even when that's disabled) -- what sort of messages are you seeing in /var/log/procmail.log? According to the above, procmail is receiving messages, so perhaps it'll have more info on what's happening to them.

Regarding the CPU usage, if you run "top" from the command line, do you see a major offender for CPU usage? Feel free to post the output of "ps auxww" here, perhaps we can help diagnose the issue.

A few other questions:

* What distro are you using?

* What Virtualmin version do you have?

* How did you go about installing Virtualmin?

Thanks,
-Eric

Sun, 01/04/2009 - 17:53
jsigned

<div class='quote'>So, coming back to the ClamAV issue later (it sounds like you still have issues even when that's disabled) -- what sort of messages are you seeing in /var/log/procmail.log? According to the above, procmail is receiving messages, so perhaps it'll have more info on what's happening to them.</div>
Here something interesting from the procmail.log
[code:1]Time:1231124915 From:testemailer@gmail.com To:info@virtualdomain1.com User:info.virtualdomain1 Size:2227 Dest:/dev/null Mode:Virus[/code:1]and here is the output from /var/log/mail.log
[code:1]Jan 4 20:13:52 server3 postfix/smtpd[3237]: connect from rv-out-0708.google.com[209.85.198.251]
Jan 4 20:13:52 server3 postfix/smtpd[3237]: 340101D77FB: client=rv-out-0708.google.com[209.85.198.251]
Jan 4 20:13:52 server3 postfix/cleanup[3241]: 340101D77FB: message-id=&lt;83d66cc50901041914w10db01c5m5cf9cb3364f42ad9@mail.gmail.com&gt;
Jan 4 20:13:52 server3 postfix/qmgr[2524]: 340101D77FB: from=&lt;testemailer@gmail.com&gt;, size=2716, nrcpt=1 (queue active)
Jan 4 20:14:22 server3 postfix/smtpd[3237]: disconnect from rv-out-0708.google.com[209.85.198.251][/code:1]

<div class='quote'>Regarding the CPU usage, if you run &quot;top&quot; from the command line, do you see a major offender for CPU usage? Feel free to post the output of &quot;ps auxww&quot; here, perhaps we can help diagnose the issue.</div>Here are the results from the top command. (btw cool command) The server was so messed up I had to literally power it off, it wouldn't even respond to the terminal.
[code:1]
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2670 1005 25 0 37460 7264 1072 R 13.3 1.9 4:49.63 clamscan
3117 1005 25 0 35484 29m 1072 R 13.3 7.9 2:05.62 clamscan
2700 clamav 25 0 37628 6648 456 R 13.0 1.7 4:50.02 clamd
2679 info.sha 25 0 37464 7268 1072 R 10.3 1.9 4:49.62 clamscan
2599 art.shar 25 0 37464 7264 1072 R 10.0 1.9 4:49.05 clamscan
2675 1005 25 0 37464 7264 1072 R 10.0 1.9 4:49.22 clamscan
2988 art.shar 25 0 36144 29m 1072 R 10.0 7.9 2:58.51 clamscan
2999 sales.bl 25 0 36144 24m 1072 R 10.0 6.5 2:52.21 clamscan
3252 1005 25 0 34820 29m 1072 R 10.0 7.7 1:29.60 clamscan
3542 root 15 0 2228 1144 856 R 0.3 0.3 0:00.57 top
1 root 15 0 1944 560 540 S 0.0 0.1 0:01.35 init
2 root 34 19 0 0 0 S 0.0 0.0 0:00.00 [/code:1]

<div class='quote'>A few other questions:
* What distro are you using?</div>
Debian Etch with as best I can tell all of the current patches

<div class='quote'>* What Virtualmin version do you have?</div>
Virtualmin 3.64 GPL

<div class='quote'>* How did you go about installing Virtualmin?</div>
I created a Debian Etch Netinstall CD and more or less followed the instructions on a clean hard drive. To the question of &quot;Software selection&quot; I choose &quot;Standard system&quot; and nothing else.

To install Virtualmin I downloaded and ran the script located on virtualmin.com/download.html. The script ran without an error.

Sun, 01/04/2009 - 18:18 (Reply to #3)
andreychek

Alrighty, it looks like ClamAV is causing some problems.

Here's where I would start --

Log in as root over SSH, and type:

/etc/init.d/clamav-daemon restart

Then, log into Virtualmin, and go back to that Spam and Virus Scanning you were in before, and set the virus scanning program to &quot;Server scanner (clamdscan)&quot; (then hit Save).

That should take a huge load off your server.

There may be a few &quot;clamscan&quot; processes running, you can either way for them to finish, or just type &quot;killall clamscan&quot; to get rid of them.

Then, use either top or &quot;ps aux | grep clam&quot; to see if ClamAV is using a saner amount of resources.

If it doesn't, as a temporary bandaid, you can always disable the Virus Scanner just so it doesn't bog down your system while we're working this out (you can do that in System Settings -&gt; Features and Plugins).

Let's see where that gets us :-)

Oh, and what version of Clam do you have? You can tell by typing:

dpkg -l 'clam*'

Thanks,
-Eric

Mon, 01/05/2009 - 12:49
jsigned

<div class='quote'>Log in as root over SSH, and type:

/etc/init.d/clamav-daemon restart

Then, log into Virtualmin, and go back to that Spam and Virus Scanning you were in before, and set the virus scanning program to &quot;Server scanner (clamdscan)&quot; (then hit Save).

That should take a huge load off your server.</div>

OK that's done and this is what happened when I set the virus scanning program to &quot;Server scanner (clamdscan)&quot;

Error The selected virus scanning command does not work :

connect(): No such file or directory

WARNING: Can't connect to clamd.

----------- SCAN SUMMARY -----------

Infected files: 0

Time: 0.031 sec (0 m 0 s)

<div class='quote'>There may be a few &quot;clamscan&quot; processes running, you can either way for them to finish, or just type &quot;killall clamscan&quot; to get rid of them.</div>

clamscan is starting to bug me, so I was glad to kill it ;-)

<div class='quote'>Then, use either top or &quot;ps aux | grep clam&quot; to see if ClamAV is using a saner amount of resources.</div>

According to the top command clamd is now using 99.9% Definitely not sane

<div class='quote'>If it doesn't, as a temporary bandaid, you can always disable the Virus Scanner just so it doesn't bog down your system while we're working this out (you can do that in System Settings -&gt; Features and Plugins).</div>

So I did that and clamd was still hogging the system, so I killed it too. After that the server was much more responsive. But when I start messing around I often reboot just to have a clean start on things. And danged if that stupid clamd didn't restart and start hogging everything again. (btw I killed it again)

<div class='quote'>Oh, and what version of Clam do you have? </div>

server3:~# dpkg -l 'clam*'
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Installed/Config-files/Unpacked/Failed-config/Half-installed
|/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
||/ Name Version Description
+++-========================-========================-================================================================
ii clamav 0.90.1dfsg-4etch16 antivirus scanner for Unix
ii clamav-base 0.90.1dfsg-4etch16 base package for clamav, an anti-virus utility for Unix
ii clamav-daemon 0.90.1dfsg-4etch16 antivirus scanner daemon
un clamav-data &lt;none&gt; (no description available)
ii clamav-docs 0.90.1dfsg-4etch16 documentation package for clamav, an anti-virus utility for Unix
ii clamav-freshclam 0.90.1dfsg-4etch16 downloads clamav virus databases from the Internet
ii clamav-testfiles 0.90.1dfsg-4etch16 use these files to test that your Antivirus program works

Thanks J

Mon, 01/05/2009 - 14:45 (Reply to #5)
andreychek

Alright, well, ClamAV definitely is not behaving; while it's never been the most lightweight daemon on the system, it certainly shouldn't be taking up 99% of your CPU.

Clam is probably setup to launch on bootup.

You can disable that by logging into Virtualmin, and clicking Webmin -&gt; System -&gt; Bootup and Shutdown, and disable ClamAV from in there.

That doesn't resolve the problem, as something isn't working right with that app -- it should work!

But this at least prevents it from bringing down your system :-)

I'll look into some potential causes for this and get back to you.
-Eric

Mon, 01/05/2009 - 15:09 (Reply to #6)
Joe
Joe's picture

I haven't been following this thread closely, but I suspect this is just the slowness of the Debian clam package.

I've seen it take nearly 30 minutes to start clamd on a VPS system. Once started, it runs OK...but, it eats all of the CPU for those 20-30 minutes while starting up.

The solution I ended up using for that particular case was to install the volatile repo version of clamav, which is dramatically faster on startup (still slow, because clamav is just really slow to start up since the .9x version), but only took ~5 minutes or so to fire up on the system in question.

Might be worth a try in this case, as well.

--

Check out the forum guidelines!

Mon, 01/05/2009 - 19:29
jsigned

For what it is worth I got this message back on my Hotmail account used for testing:
<div class='quote'>This is the mail system at host server3.mydomain_one.com.

I'm sorry to have to inform you that your message could not
be delivered to one or more recipients. It's attached below.

For further assistance, please send mail to postmaster.

If you do so, please include this problem report. You can
delete your own text from the attached returned message.

The mail system

&lt;info.virtual_domain_two@server3.mydomain_one.com&gt; (expanded from
&lt;info@virtual_domain_two.com&gt;): Command time limit exceeded:
&quot;/usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME&quot;</div>

On a working server how long does it take between when the mail is received and when it can be picked up by the client, ie Thunderbird?

I just ran an experiment and the var/log/mail.log showed the receipt of the email immediately after it was sent, but Thunderbird was not able to pick it up from the server for several minutes, even though I repeatedly clicked on the &quot;get mail&quot; for that address. A few minutes delay is not the issue, just that when I'm testing I need to have my expectations set appropriately.

Again guys, Thank you
J

Mon, 01/05/2009 - 19:34 (Reply to #8)
andreychek

Well, in your case, the problem exists between when your system receives the email, and attempts to deliver it to the mail spool on the file system.

The attempt to deliver it to the filesystem is what's timing out -- it's never at a point where Thunderbird can pick it up.

Does the above occur with ClamAV enabled or disabled in Features and Settings?

I'd make sure it works with Clam disabled, then work from there (I suspect it's the one causing the timeouts).

For example, I'd try Joe's suggestion of working with a non-standard Clam package, that may help.
-Eric

Tue, 01/06/2009 - 11:54
jsigned

Well there is a reason I post in the newb section :-) How to install the volatile clamscan wasn't something I knew how to do, and I really do try and RTM before I post. Google is my friend and I found this link: <a href='http://www.prakti.org/~myself/2007/12/29/howto-backport-clamav-daemon-in... target='_blank'>bug in clamav and debian etch</a>I got to step 5 and everything went south, with a bunch of error messages. Could someone maybe point me to a better set of instructions?

Tue, 01/06/2009 - 12:11 (Reply to #10)
andreychek

Sorry, all the forum names start to blur together after awhile :-)

It looks like ClamAV is available in Debian Etch's Volatile repository.

There's some info on that here:

http://packages.debian.org/etch-volatile/clamav-daemon

What I would do is run:

dpkg -l '*clam*'

And for every package that shows up, I'd replace it with a version from the volatile repository.

You can either manually download the binary packages from the above URL, or add an entry for the volatile repository into /etc/apt/sources.list, and then use apt to pull it down.

For example, something like:

deb http://volatile.debian.net/debian-volatile etch/volatile main

Just make sure not to pull in more than just the ClamAV packages, you won't want your entire system to be running volatile :-)
-Eric

Tue, 01/06/2009 - 12:18 (Reply to #11)
Joe
Joe's picture

<div class='quote'>Just make sure not to pull in more than just the ClamAV packages, you won't want your entire system to be running volatile</div>

It's probably not <i>too</i> scary. volatile is a pretty specific set of packages: those that move fast, and need constant updates to stay ahead of trouble. SpamAssassin, ClamAV, etc. It certainly wouldn't hurt to be running new versions of those. It is more risky, as the volatile packages are dramatically less well-tested, but probably not too bad. A lot of users do have volatile enabled, without any exclusions and such. Jamie mentioned last night that he's been running one of his boxes with volatile enabled for a while and hasn't had any problems.

We're actually considering enabling the repo, by default, during installation, because the stock clamav in Debian etch is so darned slow, particularly on VPS systems.

--

Check out the forum guidelines!

Tue, 01/06/2009 - 17:11
jsigned

It works! Email is working, Yea haw!

I installed the volatile clamav and put all the changes made in Virtualmin back to where they should be and email still works. Now instead of clamd claiming 99% of the cpu, init claims the top spot for the heaviest use at .1% cpu. Then I sent one email addressed to all of the email addresses on the server and clam show up on the top for just a few seconds.

I couldn't have done it without your help, Thank you
J

Tue, 01/06/2009 - 17:15 (Reply to #13)
andreychek

That's great news! Thanks for the update.

Have a good one,
-Eric

Topic locked