Hi,
Spam and virus filtering stopped working on my server. /var/log/procmail.log is telling me:
procmail: Program failure (-25) of "/etc/webmin/virtual-server/lookup-domain.pl"
I though, I have solved it yesterday (at least temporarily) after discovering, that WEBMIN_CONFIG environmental variable was not being passed from /etc/webmin/virtual-server/lookup-domain.pl to /usr/libexec/webmin/virtual-server/lookup-domain.pl -- that is I was getting errors like WEBMIN_CONFIG is not defined in web-lib-funcs.pl and in the beginning of /usr/libexec/webmin/virtual-server/lookup-domain.pl WEBMIN_CONFIG was not defined, even though it was being set and correctly defined in /etc/webmin/virtual-server/lookup-domain.pl just before exec(/usr/libexec/webmin/virtual-server/lookup-domain.pl)).
To make the long story short, when I first got the error "procmail: Program failure (-25)" the following command would fail, when ran from command line
/etc/webmin/virtual-server/lookup-domain.pl username < sample-spam.txt
After adding $ENV{'WEBMIN_CONFIG'} = "/etc/webmin"; at the beginning of /usr/libexec/webmin/virtual-server/lookup-domain.pl the command above works when ran from command line and yet I am still getting
procmail: Program failure (-25) of "/etc/webmin/virtual-server/lookup-domain.pl"
most of the time (it worked for one email after the restart and then it stopped working again).
Would anyone know, what else could be going wrong?
Thanks a lot for any thoughts!
Cheers, tomiskou
P.S.: I have CentOS release 5.5 (Final), Webmin version 1.530 and Virtualmin version 3.82.gpl GPL
Howdy,
You may want to start by checking these two items --
Verify in Virtualmin's "Edit Mail and FTP Users" screen that the user in question isn't running low on their available quota.
Look in /etc/postfix/main.cf, and look for the parameters "message_size_limit" and "mailbox_size_limit"... what are those set to? And then, is there any chance either of those limits is being reached?
-Eric
Hi!
Thanks a lot for your reply!
It looks to be happening for all the users, even for those with no limits!
There is neither "message_size_limit" nor "mailbox_size_limit" in my /etc/postfix/main.cf :(.
I don't think the problem is in any limits, 'cause all the users I have are far away from reaching their quotas and I still have more than 10GB of space on my hdd left.
I am starting to get a feeling, that the problem might somehow be in perl or in it's configuration on my machine. But I don't know anything about it and so I am probably completely wrong. Yesterday, when I was experimenting, the /etc/webmin/virtual-server/lookup-domain.pl was from time to time running alright without needing any fixing, that this the problem with environmental variables in perl was not there. For example right after a machine restart or right after re-installation of perl package. After one run, the problem with environmental variables would appear again!
Btw., when I run /etc/webmin/virtual-server/lookup-domain.pl as postfix user, it doesn't work and therefore I wanted to know, under which user should it be running?
Thanks again!
Cheers, tomiskou
Howdy,
Well, do you know when this all stopped working? Had something changed immediately beforehand?
One thing you may want to do is review /var/log/procmail.log, and see if any additional errors are being output there.
Also, if you run this command as root, what output do you receive:
postconf -nAnother thing that may help -- can you post the contents of your /etc/procmailrc file?
As far as lookup-domain.pl goes -- I believe that would end up being run as the user the email is being delivered to.
Whenever you performed your installation, did you do that using the install.sh script? Do you by chance have any third party repositories enabled? You could see those in /etc/yum.repos.d.
Thanks,
-Eric
Hi again!
Well I am not aware of any change just before this started happening, although it is unlikely that it stopped working just on its own. Unfortunately, I don't remember making any relevant changes except installing and playing with dkim-milter. At that time, reading procmail.log a lot while debugging dkim-milter, however, lookup-domain.pl still appeared to be working correctly. I can try to disable dkim, though, to see what happens.
There are no other error messages in procmail.log, spam and virus filtering is not the only troubling thing happening after reviewing the log properly. I have an email alias for an email address to be forwarded to gmail and all the emails coming to this address are also being delivered locally, while there is no rule for that.
The output of postconf -n is:
alias_database = hash:/etc/aliasesalias_maps = hash:/etc/aliases
command_directory = /usr/sbin
config_directory = /etc/postfix
daemon_directory = /usr/libexec/postfix
debug_peer_level = 2
home_mailbox = Maildir/
html_directory = no
inet_interfaces = all
mail_owner = postfix
mailbox_command = /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME
mailq_path = /usr/bin/mailq.postfix
manpage_directory = /usr/share/man
milter_default_action = accept
mydestination = $myhostname, localhost.$mydomain, localhost, server1.myserver.com
newaliases_path = /usr/bin/newaliases.postfix
non_smtpd_milters = inet:localhost:8891
queue_directory = /var/spool/postfix
readme_directory = /usr/share/doc/postfix-2.3.3/README_FILES
sample_directory = /usr/share/doc/postfix-2.3.3/samples
sender_bcc_maps = hash:/etc/postfix/bcc
sendmail_path = /usr/sbin/sendmail.postfix
setgid_group = postdrop
smtpd_milters = inet:localhost:8891
smtpd_recipient_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_destination
smtpd_sasl_auth_enable = yes
smtpd_tls_cert_file = /etc/postfix/postfix.cert.pem
smtpd_tls_key_file = /etc/postfix/postfix.key.pem
smtpd_tls_security_level = may
unknown_local_recipient_reject_code = 550
virtual_alias_maps = hash:/etc/postfix/virtual
and the content of /etc/procmailrc file is
LOGFILE=/var/log/procmail.logTRAP=/etc/webmin/virtual-server/procmail-logger.pl
:0wi
VIRTUALMIN=|/etc/webmin/virtual-server/lookup-domain.pl $LOGNAME
:0
* ?/usr/bin/test "$VIRTUALMIN" != ""
{
INCLUDERC=/etc/webmin/virtual-server/procmail/$VIRTUALMIN
}
DEFAULT=$HOME/Maildir/
ORGMAIL=$HOME/Maildir/
DROPPRIVS=yes
Concerning the ownership of the process running /etc/webmin/virtual-server/lookup-domain.pl, on my machine it should work only if root runs it, since it is parsing /etc/webmin/minserv.conf which is owed by root.bin and has -rw------- permissions.
I did my installation using install.sh. I do have third party repos allowed and the content of my /etc/yum.d directory is:
-rw-r--r-- 1 root root 1333 Jul 30 17:51 atomic.repo-rw-r--r-- 1 root root 430 May 10 2010 centos-asterisk.repo
-rw-r--r-- 1 root root 2245 Apr 26 2010 CentOS-Base.repo
-rw-r--r-- 1 root root 414 May 10 2010 centos-digium.repo
-rw-r--r-- 1 root root 626 Apr 26 2010 CentOS-Media.repo
-rw-r--r-- 1 root root 739 Nov 13 04:55 mirrors-rpmforge
-rw-r--r-- 1 root root 717 Nov 13 04:55 mirrors-rpmforge-extras
-rw-r--r-- 1 root root 728 Nov 13 04:55 mirrors-rpmforge-testing
-rw-r--r-- 1 root root 1113 Nov 13 04:55 rpmforge.repo
-rwsr-sr-x 1 root root 410 Jul 20 2007 virtualmin.repo*
Indeed, since I am really a beginner in this I have managed to update a couple of packages using third-party repos, but I wouldn't know how to fix this easily except going from package to package and doing yum --disablerepo=... reinstall package. I have already fixed awstats like that. However, I wouldn't say, that this error would have anything to do with these third party repos, since it has been more than a week since the update and lookup-domain.pl stopped working only yesterday.
Thanks a lot again!
Greetings, tomiskou
Hmm... out of curiosity, what output do you receive if you run this command:
rpm -qa | grep perlI'm curious if you still have the standard CentOS Perl package, or if that one was pulled in from another repository.
-Eric
The output of rpm -qa | grep perl is:
perl-URI-1.35-3perl-IO-Zlib-1.04-4.2.1
perl-IO-Socket-INET6-2.51-2.fc6
perl-Net-IP-1.25-2.fc6
perl-HTML-Tagset-3.10-2.1.1
perl-Crypt-SSLeay-0.51-11.el5
perl-libwww-perl-5.805-1.1.1
perl-Razor-Agent-2.84-1.el5.rf
perl-Authen-PAM-0.16-9vm
perl-BSD-Resource-1.28-1.fc6.1
perl-IO-Socket-SSL-1.01-1.fc6
perl-Net-DNS-0.59-3.el5
perl-DBD-Pg-1.49-2.el5_3.1
perl-XML-Parser-2.34-6.1.2.2.1
perl-XML-Simple-2.14-4.fc6
perl-5.8.8-32.el5_5.2
perl-Convert-ASN1-0.20-1.1
perl-DBI-1.52-2.el5
perl-Compress-Zlib-1.42-1.fc6
perl-DBD-MySQL-3.0007-2.el5
perl-Net-SSLeay-1.30-4.fc6
perl-Socket6-0.19-3.fc6
perl-Digest-HMAC-1.01-15
perl-HTML-Parser-3.55-1.fc6
mod_perl-2.0.4-6.el5
perl-5.8.8-32.el5_5.2
perl-Digest-SHA1-2.11-1.2.1
perl-String-CRC32-1.4-2.fc6
perl-Archive-Tar-1.39.1-2.el5.art
Well, all of them except for the three listed below are from CentOS-5 - Base, CentOS-5 - Extras or CentOS-5 - Updates repository (using repoquery command, waw I am learning a lot today).
perl-Razor-Agent-2.84-1.el5.rf -- RHEL 5 - RPMforge.net - dagperl-Authen-PAM-0.16-9vm -- Virtualmin Distribution Neutral
perl-Archive-Tar-1.39.1-2.el5.art -- CentOS / Red Hat Enterprise Linux 5 - atomic
Besides that I have tried to disable DKIM without any effect. I also tried to do a little bit more detailed log search trying to spot a preceding event possibly leading to this failure, the only thing I was able to spot, however, is that this first failure happened right after logrotate.
Thank you!
tomiskou
Hi all!
Just to summarise, my problem appears to be in /usr/libexec/webmin/virtual-server/lookup-domain.pl or in perl itself.
Please, if anyone after reading everything below knows in which direction I should be looking next when trying to figure out the cause of my problem, I would appreciate any suggestions.
The symptoms are: procmail: Program failure (-25) of "/etc/webmin/virtual-server/lookup-domain.pl" (in /var/log/procmail.log) which means that no spam and virus filter is run on mail locally delivered by procmail.
For further reference (in the case anyone else has a similar issue), here is a little explanation of how things work, based on my freshman's understanding (for users knowing well how things work, please skip to the next paragraph): 1.) /etc/webmin/virtual-server/lookup-domain.pl is a standalone spamassassin and clamav wrapper. Note1: /etc/webmin/virtual-server/lookup-domain.pl parses /etc/webmin/minserv.conf to obtain webmin directory containing perl scripts (further referred to as $root) and then runs $root/virtual-server/lookup-domain.pl (in my case /usr/libexec/webmin/virtual-server/lookup-domain.pl). Note2: Since on my machine /etc/webmin/minserv.conf is accessible for reading only for root, /etc/webmin/virtual-server/lookup-domain.pl has better be run by root. 2.) a) When email is delivered to postfix, it runs /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME (see postconf -n | grep mailbox_command). 2.) b) /usr/bin/procmail-wrapper runs /usr/bin/procmail and the role of procmail-wrapper in all this is only to assure that procmail is run by root (since it has SUID bit), I guess. 2.) c) /usr/bin/procmail then executes /etc/procmailrc, where /etc/webmin/virtual-server/lookup-domain.pl is used to for extracting of the user, to which the email is being delivered. This is then used to include domain specific procmailrc (stored is /etc/webmin/virtual-server/procmail/) in which spam and virus filter is invoked.
Thus, if /usr/libexec/webmin/virtual-server/lookup-domain.pl doesn't work => spam and virus filtering doesn't work either. (Implication going the other way around is not true of course :).
To exclude other possible suspects for the cause of
procmail: Program failure (-25) of "/etc/webmin/virtual-server/lookup-domain.pl" (in /var/log/procmail.log)
I have generated output of lookup-domain.pl for all the domains accepting mail on my server -- and btw. the script /etc/webmin/virtual-server/lookup-domain.pl DOES work most of the time when run by me (root from terminal) and DOESN'T work most of the time when run by procmail -- and exchanged the original lookup-domain.pl with a simple "if, elsif" script (given user name, returns same as the original lookup-domain.pl). My observation is, that since this exchange there was no "Program failure (-25)" or any other failure concerning lookup-domain.pl detected on my system and already quite some mail has been delivered.
As mentioned in my previous posts, while running /etc/webmin/virtual-server/lookup-domain.pl, I have noticed that most of the time when the script fails, it fails because the environmental variable are not being passed from script to script as scripts are being nested. It always succeeds after system restart or reinstallation of perl (but this might be a coincidence).
My conclusion therefore is, that there is a nasty bug either in /usr/libexec/webmin/virtual-server/lookup-domain.pl or in perl itself (or somewhere even deeper).
My OS is CentOS release 5.5 (Final), my webmin version 1.530, my virtualmin version 3.82.gpl GPL, my perl version is 5.8.8-32.el5_5.2 (for the source repos of all my perl packages see my previous posts). I do have other packages from NON Cent-OS repo (like php and mysql) installed, but there was no upgrade immediately preceding the first occurrence of my problem.
Any suggestions about what could be the cause of my problem would be appreciated!
Thanks a lot!
Best, tomiskou