Mail delivery and spamassassin

24 posts / 0 new
Last post
#1 Tue, 02/07/2006 - 11:12
BenjaminVanWagner

Mail delivery and spamassassin

when I originally sert this server up i chose postfix for email.. then changed it to sendmail in roder to use email address as login..

in this process my sendmail config became completely FUBAR.

i have sendmail working. and if mail is set to forward to someone it does.

but it fails on local delivery.

if i emailk user 'jlw' it will say

Feb 7 11:49:00 shareweb sendmail[[10173]]: k17HmvYl010172: to=jlw@salinecounty.net, delay=00:00:03, xdelay=00:00:02, mailer=local, pri=32498, dsn=2.0.0, stat=Sent

-rw------- 1 jlw@salinecounty.net mail 0 Feb 2 16:25 jlw@salinecounty.net

but here is his mail file.

it goes into lala land.. never to be seen again.

also.. it never goes through spamassassin at all..

whats up with this??

Tue, 02/07/2006 - 11:47
BenjaminVanWagner

more info..

even weirder

if i erase the mail file.. jlw@...

it remakes the file when i send him mail... it is just empty

i removed sendmail and procmail with --nodeps.. and reinstalled them both..

still no email

Wed, 02/08/2006 - 08:57
BenjaminVanWagner

anyone ??

Wed, 02/08/2006 - 13:59
Joe
Joe's picture

Hey Benjamin,

I'll first point out that we now support a bit of magic to allow Postfix to be used with "user@domain.tld" usernames. It's a workaround (because Wietse believes strongly enough that @ in the username is a bad idea that it has been stripped from Postfix and will not be re-added), but it works.

Next up...I'm not sure what to make of what you're seeing. The log looks like it is working. However, procmail is not being used as the delivery agent, as far as I can tell (but maybe it's been too long since I looked at a sendmail log--maybe it doesn't tell you that it is handing off the mail to a delivery agent the way Postfix does). But nonetheless, it looks like there is a successful delivery to somewhere.

Maybe you should post your Sendmail config and /etc/procmailrc. We might be able to spot where things are going astray.

--

Check out the forum guidelines!

Thu, 02/23/2006 - 06:42 (Reply to #4)
BenjaminVanWagner

does anyone have an idea how this part works

is anyone going to try to help

I have a customer who has been SCREAMING at me for a week while I have been trying to find answers here with no help

Fri, 02/10/2006 - 15:16
BenjaminVanWagner

Well i did remove and readd sendmail and procmail

and used a default installation with the exception of removing the addr=127.0.0.1 line from sendmail.mc and regenerating..

is there any way to reinit virtualmins mail settings ?

Mon, 02/13/2006 - 11:47
BenjaminVanWagner

okay.....

here is sendmail config

divert(-1)dnl
dnl #
dnl # This is the sendmail macro config file for m4. If you make changes to
dnl # /etc/mail/sendmail.mc, you will need to regenerate the
dnl # /etc/mail/sendmail.cf file by confirming that the sendmail-cf package is
dnl # installed and then performing a
dnl #
dnl # make -C /etc/mail
dnl #
include(`/usr/share/sendmail-cf/m4/cf.m4')dnl
VERSIONID(`setup for Red Hat Linux')dnl
OSTYPE(`linux')dnl
dnl #
dnl # default logging level is 9, you might want to set it higher to
dnl # debug the configuration
dnl #
dnl define(`confLOG_LEVEL', `9')dnl
dnl #
dnl # Uncomment and edit the following line if your outgoing mail needs to
dnl # be sent out through an external mail server:
dnl #
dnl define(`SMART_HOST',`smtp.your.provider')
dnl #
define(`confDEF_USER_ID',``8:12'')dnl
dnl define(`confAUTO_REBUILD')dnl
define(`confTO_CONNECT', `1m')dnl
define(`confTRY_NULL_MX_LIST',true)dnl
define(`confDONT_PROBE_INTERFACES',true)dnl
define(`PROCMAIL_MAILER_PATH',`/usr/bin/procmail')dnl
define(`ALIAS_FILE', `/etc/aliases')dnl
define(`STATUS_FILE', `/var/log/mail/statistics')dnl
define(`UUCP_MAILER_MAX', `2000000')dnl
define(`confUSERDB_SPEC', `/etc/mail/userdb.db')dnl
define(`confPRIVACY_FLAGS', `authwarnings,novrfy,noexpn,restrictqrun')dnl
define(`confAUTH_OPTIONS', `A')dnl
dnl #
dnl # The following allows relaying if the user authenticates, and disallows
dnl # plaintext authentication (PLAIN/LOGIN) on non-TLS links
dnl #
dnl define(`confAUTH_OPTIONS', `A p')dnl
dnl #
dnl # PLAIN is the preferred plaintext authentication method and used by
dnl # Mozilla Mail and Evolution, though Outlook Express and other MUAs do
dnl # use LOGIN. Other mechanisms should be used if the connection is not
dnl # guaranteed secure.
dnl # Please remember that saslauthd needs to be running for AUTH.
dnl #
dnl TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
dnl define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
dnl #
dnl # Rudimentary information on creating certificates for sendmail TLS:
dnl # cd /usr/share/ssl/certs; make sendmail.pem
dnl # Complete usage:
dnl # make -C /usr/share/ssl/certs usage
dnl #
dnl define(`confCACERT_PATH',`/etc/pki/tls/certs')
dnl define(`confCACERT',`/etc/pki/tls/certs/ca-bundle.crt')
dnl define(`confSERVER_CERT',`/etc/pki/tls/certs/sendmail.pem')
dnl define(`confSERVER_KEY',`/etc/pki/tls/certs/sendmail.pem')
dnl #
dnl # This allows sendmail to use a keyfile that is shared with OpenLDAP's
dnl # slapd, which requires the file to be readble by group ldap
dnl #
dnl define(`confDONT_BLAME_SENDMAIL',`groupreadablekeyfile')dnl
dnl #
dnl define(`confTO_QUEUEWARN', `4h')dnl
dnl define(`confTO_QUEUERETURN', `5d')dnl
dnl define(`confQUEUE_LA', `12')dnl
dnl define(`confREFUSE_LA', `18')dnl
define(`confTO_IDENT', `0')dnl
dnl FEATURE(delay_checks)dnl
FEATURE(`no_default_msa',`dnl')dnl
FEATURE(`smrsh',`/usr/sbin/smrsh')dnl
FEATURE(`mailertable',`hash -o /etc/mail/mailertable.db')dnl
FEATURE(`virtusertable',`hash -o /etc/mail/virtusertable.db')dnl
FEATURE(redirect)dnl
FEATURE(always_add_domain)dnl
FEATURE(use_cw_file)dnl
FEATURE(use_ct_file)dnl
dnl #
dnl # The following limits the number of processes sendmail can fork to accept
dnl # incoming messages or process its message queues to 12.) sendmail refuses
dnl # to accept connections once it has reached its quota of child processes.
dnl #
dnl define(`confMAX_DAEMON_CHILDREN', 12)dnl
dnl #
dnl # Limits the number of new connections per second. This caps the overhead
dnl # incurred due to forking new sendmail processes. May be useful against
dnl # DoS attacks or barrages of spam. (As mentioned below, a per-IP address
dnl # limit would be useful but is not available as an option at this writing.)
dnl #
dnl define(`confCONNECTION_RATE_THROTTLE', 3)dnl
dnl #
dnl # The -t option will retry delivery if e.g. the user runs over his quota.
dnl #
FEATURE(local_procmail,`',`procmail -t -Y -a $h -d $u')dnl
FEATURE(`access_db',`hash -T[[tmpf]] -o /etc/mail/access.db')dnl
FEATURE(`blacklist_recipients')dnl
EXPOSED_USER(`root')dnl
dnl #
dnl # For using Cyrus-IMAPd as POP3/IMAP server through LMTP delivery uncomment
dnl # the following 2 definitions and activate below in the MAILER section the
dnl # cyrusv2 mailer.
dnl #
dnl define(`confLOCAL_MAILER', `cyrusv2')dnl
dnl define(`CYRUSV2_MAILER_ARGS', `FILE /var/lib/imap/socket/lmtp')dnl
dnl #
dnl # The following causes sendmail to only listen on the IPv4 loopback address
dnl # 127.0.0.1 and not on any other network devices. Remove the loopback
dnl # address restriction to accept email from the internet or intranet.
dnl #
DAEMON_OPTIONS(`Port=smtp, Name=MTA')dnl
dnl #
dnl # The following causes sendmail to additionally listen to port 587 for
dnl # mail from MUAs that authenticate. Roaming users who can't reach their
dnl # preferred sendmail daemon due to port 25 being blocked or redirected find
dnl # this useful.
dnl #
dnl DAEMON_OPTIONS(`Port=submission, Name=MSA, M=Ea')dnl
dnl #
dnl # The following causes sendmail to additionally listen to port 465, but
dnl # starting immediately in TLS mode upon connecting. Port 25 or 587 followed
dnl # by STARTTLS is preferred, but roaming clients using Outlook Express can't
dnl # do STARTTLS on ports other than 25. Mozilla Mail can ONLY use STARTTLS
dnl # and doesn't support the deprecated smtps; Evolution
dnl # when SSL is enabled-- STARTTLS support is available in version 1.1.1.
dnl #
dnl # For this to work your OpenSSL certificates must be configured.
dnl #
dnl DAEMON_OPTIONS(`Port=smtps, Name=TLSMTA, M=s')dnl
dnl #
dnl # The following causes sendmail to additionally listen on the IPv6 loopback
dnl # device. Remove the loopback address restriction listen to the network.
dnl #
dnl DAEMON_OPTIONS(`port=smtp,Addr=::1, Name=MTA-v6, Family=inet6')dnl
dnl #
dnl # enable both ipv6 and ipv4 in sendmail:
dnl #
dnl DAEMON_OPTIONS(`Name=MTA-v4, Family=inet, Name=MTA-v6, Family=inet6')
dnl #
dnl # We strongly recommend not accepting unresolvable domains if you want to
dnl # protect yourself from spam. However, the laptop and users on computers
dnl # that do not have 24x7 DNS do need this.
dnl #
FEATURE(`accept_unresolvable_domains')dnl
dnl #
dnl FEATURE(`relay_based_on_MX')dnl
dnl #
dnl # Also accept email sent to "localhost.localdomain" as local email.
dnl #
LOCAL_DOMAIN(`localhost.localdomain')dnl
dnl #
dnl # The following example makes mail from this host and any additional
dnl # specified domains appear to be sent from mydomain.com
dnl #
dnl MASQUERADE_AS(`mydomain.com')dnl
dnl #
dnl # masquerade not just the headers, but the envelope as well
dnl #
dnl FEATURE(masquerade_envelope)dnl
dnl #
dnl # masquerade not just @mydomainalias.com, but @*.mydomainalias.com as well
dnl #
dnl FEATURE(masquerade_entire_domain)dnl
dnl #
dnl MASQUERADE_DOMAIN(localhost)dnl
dnl MASQUERADE_DOMAIN(localhost.localdomain)dnl
dnl MASQUERADE_DOMAIN(mydomainalias.com)dnl
dnl MASQUERADE_DOMAIN(mydomain.lan)dnl
MAILER(smtp)dnl
MAILER(procmail)dnl
dnl MAILER(cyrusv2)dnl

Sun, 06/07/2009 - 06:59
BenjaminVanWagner

:0w
VIRTUALMIN=|/etc/webmin/virtual-server/lookup-domain.pl $LOGNAME
:0
* ?test "$VIRTUALMIN" != ""
{
INCLUDERC=/etc/webmin/virtual-server/procmail/$VIRTUALMIN
}
DEFAULT=/var/spool/mail/$LOGNAME
DROPPRIVS=yes
:0
$DEFAULT

Mon, 02/20/2006 - 10:40
BenjaminVanWagner

is the procmailrc CORRECT for sendmail?

or is this perhaps for the postfix workaround?

what should the procmailrc be if using SENDMAIL ?

Mon, 02/20/2006 - 10:41
BenjaminVanWagner

anyone ??

Tue, 02/21/2006 - 10:07
BenjaminVanWagner

anyone ?

Thu, 02/23/2006 - 16:59
Joe
Joe's picture

Hi Benjamin,

I'm unfamiliar enough with Sendmail that I'm afraid I don't really have any good answers.

Is there anything in the procmail log for the user in question? If not, you might turn on logging (it's on by default, I think, but it might be getting disabled somewhere) with something like "VERBOSE=yes" and "LOGFILE=/var/log/procmail.log" in the procmailrc.

I also seem to recall some folks had trouble with the DEFAULT setting that Virtualmin uses by default. Perhaps simply removing that line will fix the problem. I don't know if this was in a Sendmail environment or not...but it's worth a try.

--

Check out the forum guidelines!

Fri, 02/24/2006 - 11:34 (Reply to #12)
BenjaminVanWagner

if I remove the virtualmin default config it will function..

but virtualmin puts it back in.

ill try it with the virtualmin settings "in" and verbose logging

Fri, 02/24/2006 - 11:40 (Reply to #13)
BenjaminVanWagner

okay.. obviously the first assignment is failing right off the bat

procmail: Assigning "VIRTUALMIN="
procmail: [[13610]] Fri Feb 24 12:18:42 2006
procmail: Executing "/usr/bin/test,,!=,"
procmail: [[13610]] Fri Feb 24 12:18:42 2006
procmail: Non-zero exitcode (1) from "/usr/bin/test"
procmail: No match on "/usr/bin/test != "
procmail: Assigning "PATH=/root/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/X11R6/bin"
From MAILER-DAEMON@shareweb.direclynx.net Fri Feb 24 12:18:40 2006
Subject: Postmaster notify: see transcript for details
Folder: /var/mail/root

Fri, 02/24/2006 - 11:45 (Reply to #14)
BenjaminVanWagner

tailed from the beginning of a test send

procmail: Executing "/etc/webmin/virtual-server/lookup-domain.pl,jlw@salinecounty.net"
procmail: Assigning "VIRTUALMIN="
procmail: [[13753]] Fri Feb 24 12:25:09 2006
procmail: Executing "/usr/bin/test,113638365818656,!=,"
procmail: [[13753]] Fri Feb 24 12:25:09 2006
procmail: Match on "/usr/bin/test 113638365818656 != "
procmail: Assigning "INCLUDERC=/etc/webmin/virtual-server/procmail/113638365818656"
procmail: Assigning "DROPPRIVS=yes"
procmail: Assuming identity of the recipient, VERBOSE=off
LibClamAV Error: Wrote 0 instead of 512 (/tmp/clamav-d0e6f59770f5ddd4/main.db).
cli_untgz: Disk quota exceeded
LibClamAV Error: cli_cvdload(): Can't unpack CVD file.
LibClamAV Error: Can't load /var/clamav/main.cvd: CVD extraction failure
ERROR: CVD extraction failure
procmail: Program failure (50) of "/usr/bin/clamscan"
From bvanwagner@oaklawn.com Fri Feb 24 12:25:07 2006
Subject:
Folder: /dev/null

Fri, 02/24/2006 - 14:39 (Reply to #15)
BenjaminVanWagner

i turned quotas off since i couldnt find any user whose quota would have been exceeded

now.. mail gets delivered but it doesnt look like its getting scanned by clamav.. there is no "scanned by" header and no record of it in the log

this is very odd

Fri, 02/24/2006 - 14:44 (Reply to #16)
BenjaminVanWagner

turned quotas on..

now it looks like clamav is back in the loop

but it fails again

procmail: Assigning "DROPPRIVS=yes"
procmail: Assuming identity of the recipient, VERBOSE=off
LibClamAV Error: Wrote 0 instead of 512 (/tmp/clamav-beb2644a5ec5d228/main.db).
cli_untgz: Disk quota exceeded
LibClamAV Error: cli_cvdload(): Can't unpack CVD file.
LibClamAV Error: Can't load /var/clamav/main.cvd: CVD extraction failure
ERROR: CVD extraction failure
procmail: Program failure (50) of "/usr/bin/clamscan"

Fri, 02/24/2006 - 14:54 (Reply to #17)
BenjaminVanWagner

turning virus filtering off for the domain..

leaving quotas on..

gives the same result.. the mail goes through fine..

Fri, 02/24/2006 - 15:28 (Reply to #18)
BenjaminVanWagner

included procmail file below

DROPPRIVS=yes
:0cw
| /usr/bin/clamscan -
:0e
/dev/null
:0fw
| /usr/bin/spamassassin --siteconfigpath /etc/webmin/virtual-server/spam/113630204230033

Fri, 02/24/2006 - 15:39 (Reply to #19)
BenjaminVanWagner

so this only happens if CLAMAV is on AND quotas on / are enabled.

even if the recipient AND clamav both have unlimited files and space

Fri, 02/24/2006 - 15:42 (Reply to #20)
BenjaminVanWagner

scratch that..

i think i may have figured it out

the quota was set to 5Mb on the recipient

he only had 4Kb of files.. but if the extracted AV files are greater than 5Mb it would fail..

Fri, 02/24/2006 - 16:56 (Reply to #21)
Joe
Joe's picture

Hey Benjamin,

Wow. Excellent troubleshooting!

So does that mean that small pure-text messages always go through with this configuration?

Sounds like we've learned a few things from this:

We need a warning when AV is enabled and quotas are small to strongly suggest increasing the quotas to allow for the extra file storage that Clam requires. We probably need to warn anytime hard quotas are used, since there are quite a few situations where hard quotas can cause serious and unrecoverable issues. Soft quotas don't have this kind of issue.

We need to prevent messages from being dropped on the floor in such a circumstance--I think I see why it's happening (Clam is failing), but I don't like that failure mode from Clam. This could possibly be solved by using a separate /tmp partition with no quotas, but that's really ugly and I don't want to add that requirement to the system. There is also a --max-space option, where we could probably feed in the users quota to keep clam from trying to scan files that would lead to quota overage.

We need to figure out a way to notify the server administrator if messages are getting dropped on the floor like this. Having mail disappear, apparently without warning is a terrible problem.

I also need to test to see if this occurs on a Postfix system.

--

Check out the forum guidelines!

Mon, 09/03/2007 - 01:15
CollinSchwagele

Hi Joe

Has there been any progress on this problem with Clamav yet, I have Virtulamin Pro, Postfix and Hard quotas.

Some clients make there users mailbox 15 or 10MB and it seems clam requires about 10-20MB of tmp space alone causing the mailbox to exceed its quota and the mail being bounced.

Any solutions for this.

Later
Collin

Mon, 09/03/2007 - 23:28 (Reply to #23)
Joe
Joe's picture

Yes, we're correctly bouncing messages if Clam doesn't have enough room to process them, so they are not dropping on the floor.

Very tiny quotas will never be a good idea when Spam/AV are involved. I'd suggest backing off on your quotas a bit, or don't use Clam. We can't avoid it requiring space to operate, and we can't safely process the message as any other user.

There is one way to avoid the /tmp file thing altogether, and that's to put /tmp on a filesystem without quotas enabled. e.g. create a new partition (a pretty big one since if you've got untrusted users they may find that they can take advantage of the "unlimited" storage in tmp). This will make most mail processing unrestricted by quotas. I don't necessarily recommend this, but it's not particularly dangerous (there is a potential DoS risk, if a malicious user decided to fill /tmp, as it would be unrestricted by quotas).

So, the technical solutions to this problem are already in place. Our procmail recipes are behaving correctly. But there probably should still be some stronger advice upon creation of a user with a tiny quota, to try to make folks creating users behave correctly, too. ;-)

--

Check out the forum guidelines!

Topic locked