IMAP Extension '+' folder email addressing with Postfix and Procmail

Hi,

I am setting up IMAP mail for a client and want to use extension '+' addressing for IMAP folders e.g.

INBOX --test1 --test2

So the address jack+test1@example.com will go directly to the "test1" folder.

I found this post http://www.virtualmin.com/node/18217#comment-81249 and tried editing the mailbox delivery command in /etc/procmail/main.cf file, then adding a .procmailrc file as directed.

One question there though, is the directory in which to place the .procmailrc file, ~/homes/username?

Is a .forward file required with |/usr/bin/procmail as per http://doxfer.com/Webmin/ProcmailMailFilter? I tried adding a .forward file as well to ~/homes/username.

I also restarted postfix after making changes. In any case, so far no luck. Mail with the '+test1' or '+test2' added (user+test1@example.com) is still go to the INBOX and not the subfolder.

It would be great to get this working at the server level, rather than filtering with the email client.

Is there something I am missing here?

Cheers,

Noel

Status: 
Active

Comments

Howdy -- yeah I unfortunately don't know exactly how that procmailrc file would look in order to accomplish what you're after -- but we can certainly help make sure that procmail is being used, and that the .procmailrc file is being read. Once we have that working, all you'd have to do is develop the procmailrc syntax.

The first thing we'd want to do is verify that Postfix is configured to use procmail for mail delivery.

To do that, you can run this command:

postconf -n | grep mailbox_command

It should be using procmail-wrapper.

As far as the .procmailrc file goes -- yes, for an email user, that would go into $HOME/homes/USERNAME/.procmailrc.

You do not need a .forward file.

You would want to make sure the .procmailrc file is owned by the user and group of your email user.

Also, if there are any errors in the .procmailrc, those would show up in /var/log/procmail.log. You may want to review that log to see if any errors are displayed when sending an email to that particular user.

At some point in the past I had set this up for my personal email user on a Virtualmin installation, by putting the proper commands in its individual .procmailrc, located in /home/domain/homes/user. If I recall correctly, I didn't have to make modifications in the global or domain procmailrc.

Indeed you need to modify the delivery command of Postfix, like the forum post you linked explains. For me, it reads

mailbox_command = /usr/bin/procmail-wrapper -o -a $DOMAIN -a "$EXTENSION" -d $LOGNAME

The relevant contents of my procmailrc are as follows:

EXT=$2
:0
* EXT ?? ^^sent^^|^^sent\-mail^^
$HOME/Maildir/.$EXT/

This should cause Procmail to put the mail into the IMAP subfolder when the recipient is "user+folder@domain.tld". The part after the "??" is a Procmail-syntax regular expression that limits the allowed folders to certain names, here "sent" and "sent-mail". Note that there should be a backslash before the "-" in "sent-mail"! The forum software swallows that for some reason.

The regexp is there to prevent "random" IMAP folders from getting created when some joker sends mail to "myuser+crapfolder@domain.tld". :) (Without that restriction, Procmail would simply create the folder, if it doesn't exist yet!)

Thanks Eric,

Mailbox Command

# postconf -n | grep mailbox_command
mailbox_command = /usr/bin/procmail-wrapper -o -a $DOMAIN -a "$EXTENSION" -d $LOGNAME

I changed the ownership of the .procmailrc file to the user and group of the email user. It was owned by root, however that hasn't made a difference.

The procmail log has these errors for an email sent to a subfolder (but not sent to the address without the '+' addressing).

procmail: Unable to treat as directory "/var/www/Maildir"
procmail: Error while writing to "/var/www/Maildir"
procmail: Unable to treat as directory "/var/www/Maildir"
procmail: Lock failure on "/var/www/Maildir.lock"
procmail: Error while writing to "/var/www/Maildir"

However, these errors don't prevent the mail from being sent to the INBOX.

Thanks Locutus and for the explanation of the regex. This is my .procmailrc file derived from your linked post:

EXT=$2
:0
* EXT ?? ^^test^^|^^test1^^
$HOME/Maildir/.$EXT/

Emails are sent to user+test@example.com and user+test1@example.com

As mentioned above, delivery is still to the INBOX only and not the subfolder. Any other ideas?

The procmail log would suggest that it is trying to use /var/www as the user's home directory... which is probably wrong. :) Can you please verify the home directory for the mail user in question, to be found in the /etc/passwd file.

Thanks Locutus,

The home directory for the mail user in the /etc/passwd file is correct and in the format: /home/USER/homes/USERNAME

I can't find where the system may be assuming it is /var/www

Cheers, Noel

Okay, I have to admit I'm stumped then why Procmail would assume /var/www as home directory. It sure has to get that directory name from somewhere. Maybe for some reason it's being executed as a different user whose homedir IS /var/www...

It might help if you posted your full /etc/postfix/main.cf, your /etc/procmailrc and the user's individual procmailrc, as well as log excerpts from /var/log/mail.log and /var/log/procmail.log and /var/log/syslog at the time of delivery attempt.

Thanks Locutus.

The .procmailrc file is being read for this user and you can also see that in the log files.

Edit users->username->User-configured mail forwarding

User-configured mail forwarding
Forwarding conditon                             Action  ↓
Always                                          Deliver to your Inbox
Headers match EXT ?? ^^test^^|^^test1^^         Write to file $HOME/Maildir/.$EXT/

Following are the contents of the requested files.

One possibility: If you check mail.log, line 6 and syslog, line 11 you can see the "to" address is in the format, USERNAME+folder@HOSTNAME and that "orig_to" is in the format, USERNAME+folder@DOMAIN.com.

Probably the $HOME for host.ecomautomation.com is /var/www.

/etc/postfix/main.cf

# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname

smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

readme_directory = no

# TLS parameters
smtpd_tls_cert_file = /etc/postfix/postfix.cert.pem
smtpd_tls_key_file = /etc/postfix/postfix.key.pem
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.

myhostname = host.ecomautomation.com
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
mydestination = debian, host.ecomautomation.com, localhost.ecomautomation.com, localhost
relayhost =
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
# mailbox_command = /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME
# Edit to provide mailbox extension '+' addressing
mailbox_command = /usr/bin/procmail-wrapper -o -a $DOMAIN -a "$EXTENSION" -d $LOGNAME
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
virtual_alias_maps = hash:/etc/postfix/virtual
sender_bcc_maps = hash:/etc/postfix/bcc
home_mailbox = Maildir/
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
broken_sasl_auth_clients = yes
smtpd_recipient_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_destination
allow_percent_hack = no
local_destination_concurrency_limit = 5
smtpd_sasl_authenticated_header = yes
smtpd_tls_security_level = may
smtpd_tls_mandatory_ciphers = high

/etc/procmailrc

LOGFILE=/var/log/procmail.log
TRAP=/etc/webmin/virtual-server/procmail-logger.pl
:0wi
VIRTUALMIN=|/etc/webmin/virtual-server/lookup-domain.pl $LOGNAME
EXITCODE=$?
:0
* ?/usr/bin/test "$EXITCODE" = "73"
/dev/null
EXITCODE=0
:0
* ?/usr/bin/test "$VIRTUALMIN" != ""
{
INCLUDERC=/etc/webmin/virtual-server/procmail/$VIRTUALMIN
}
ORGMAIL=$HOME/Maildir/
DEFAULT=$HOME/Maildir/
DROPPRIVS=yes
:0
$DEFAULT

/home/USER/homes/USERNAME/.procmailrc

EXT=$2
:0
* EXT ?? ^^test^^|^^test1^^
$HOME/Maildir/.$EXT/

/var/log/mail.log

Aug 10 18:49:59 host spamd[32760]: spamd: connection from localhost [127.0.0.1] at port 33752
Aug 10 18:49:59 host spamd[32760]: spamd: setuid to stsadmin.brisbanests succeeded
Aug 10 18:49:59 host spamd[32760]: spamd: processing message <5206C388.4090003@gothisway.net> for stsadmin.brisbanests:1046
Aug 10 18:50:00 host spamd[32760]: spamd: clean message (-0.0/5.0) for stsadmin.brisbanests:1046 in 1.0 seconds, 1235 bytes.
Aug 10 18:50:00 host spamd[32760]: spamd: result: . 0 - SPF_HELO_PASS,SPF_PASS scantime=1.0,size=1235,user=stsadmin.brisbanests,uid=1046,required_score=5.0,rhost=localhost,raddr=127.0.0.1,rport=33752,mid=<5206C388.4090003@gothisway.net>,autolearn=ham
Aug 10 18:50:00 host postfix/local[27468]: 29B02B0936: to=<stsadmin.brisbanests+test@host.ecomautomation.com>, orig_to=<stsadmin+test@brisbanests.com>, relay=local, delay=13, delays=0.05/0.01/0/13, dsn=2.0.0, status=sent (delivered to command: /usr/bin/procmail-wrapper -o -a $DOMAIN -a "$EXTENSION" -d $LOGNAME)
Aug 10 18:50:00 host postfix/qmgr[18990]: 29B02B0936: removed
Aug 10 18:50:00 host spamd[32759]: prefork: child states: II

/var/log/procmail.log

From myemail@domain.net  Sat Aug 10 18:49:48 2013
Subject: #11 stsadmin+test
  Folder: /home/USER/homes/USERNAME/Maildir/new/1376174999.27469_1     1377
Time:1376175000 From:noelspringer@gothisway.net To:stsadmin+test@brisbanests.com User:stsadmin.brisbanests Size:1435 Dest:/home/brisbane/homes/stsadmin/Maildir/new/1376174999.27469_1.host.ecomautomation.com Mode:None
procmail: Unable to treat as directory "/var/www/Maildir"
procmail: Error while writing to "/var/www/Maildir"
procmail: Unable to treat as directory "/var/www/Maildir"
procmail: Lock failure on "/var/www/Maildir.lock"
procmail: Error while writing to "/var/www/Maildir"

/var/log/syslog

Aug 10 18:49:47 host postfix/smtpd[27464]: connect from mxout-07.mxes.net[216.86.168.182]
Aug 10 18:49:48 host postfix/smtpd[27464]: 29B02B0936: client=mxout-07.mxes.net[216.86.168.182]
Aug 10 18:49:48 host postfix/cleanup[27467]: 29B02B0936: message-id=<5206C388.4090003@gothisway.net>
Aug 10 18:49:48 host postfix/qmgr[18990]: 29B02B0936: from=<myemail@domain.net>, size=1057, nrcpt=1 (queue active)
Aug 10 18:49:48 host postfix/smtpd[27464]: disconnect from mxout-07.mxes.net[216.86.168.182]
Aug 10 18:49:59 host spamd[32760]: spamd: connection from localhost [127.0.0.1] at port 33752
Aug 10 18:49:59 host spamd[32760]: spamd: setuid to stsadmin.brisbanests succeeded
Aug 10 18:49:59 host spamd[32760]: spamd: processing message <5206C388.4090003@gothisway.net> for stsadmin.brisbanests:1046
Aug 10 18:50:00 host spamd[32760]: spamd: clean message (-0.0/5.0) for stsadmin.brisbanests:1046 in 1.0 seconds, 1235 bytes.
Aug 10 18:50:00 host spamd[32760]: spamd: result: . 0 - SPF_HELO_PASS,SPF_PASS scantime=1.0,size=1235,user=stsadmin.brisbanests,uid=1046,required_score=5.0,rhost=localhost,raddr=127.0.0.1,rport=33752,mid=<5206C388.4090003@gothisway.net>,autolearn=ham
Aug 10 18:50:00 host postfix/local[27468]: 29B02B0936: to=<stsadmin.brisbanests+test@host.ecomautomation.com>, orig_to=<stsadmin+test@brisbanests.com>, relay=local, delay=13, delays=0.05/0.01/0/13, dsn=2.0.0, status=sent (delivered to command: /usr/bin/procmail-wrapper -o -a $DOMAIN -a "$EXTENSION" -d $LOGNAME)
Aug 10 18:50:00 host postfix/qmgr[18990]: 29B02B0936: removed
Aug 10 18:50:00 host spamd[32759]: prefork: child states: II

Hmm, while in theory you shouldn't need this -- you could always try adding these two lines to the top of your user's .procmailrc file:

ORGMAIL=$HOME/Maildir/
DEFAULT=$HOME/Maildir/

That would explicitly set the email location

Okay, I went over your config files and the logs.

First, the thing you noticed, the "to/orig_to" line in the mail log, is okay this way. Virtualmin configures Postfix to use virtual domains as opposed to "real", "mydestination" ones. It maps users like "user@domain.tld" to "locallinuxuser@hostname.domain.tld", which in turn Postfix detects as local user and delivers it to their mailbox.

One difference between your config and mine is the two lines ":0 $DEFAULT" at the end of your global procmailrc. Those are not there in my procmailrc files, so you might want to try removing them. As far as I know Procmail syntax, those two lines define a delivery rule that unconditionally puts the mail into the default folder. I'm not sure if Procmail rules are processed on a first-match or best-match or last-match basis, but if it's first-match, that rule would override all subsequent ones that try to put the mail in a subfolder.

In your procmail.log, I noticed that the error messages occur AFTER the "folder" line, which to me kinda seems like at that point Procmail already made the choice where to put the email, and that the errors are like a follow-up issue and not directly related to the IMAP folder selection.

So you might try removing the two lines from your global procmailrc as stated above, maybe that helps.

Thanks Eric, though that change had no effect so I reverted to the original .procmailrc.

Locutus, I tried the change you suggested, removing the last two lines in /etc/procmailrc and it's getting closer.

The emails with the + addressing didn't arrive in the INBOX or the subfolders of "test" and "test1". I did check that email without the + addressing was still delivered to the INBOX and that's OK.

Now, although the mail wasn't received in the "test" or "test1" folders of the email client, in the /var/log/procmail.log, the messages were delived within the user's Maildir to the subdirectories .test/new and .test1/new as follows:

To:stsadmin+test@DOMAIN User:stsadmin.brisbanests Size:1435 Dest:/home/brisbane/homes/stsadmin/Maildir/.test/new/1376225916.1533_1.host.ecomautomation.com Mode:None

To:stsadmin+test1@DOMAIN User:stsadmin.brisbanests Size:1441 Dest:/home/brisbane/homes/stsadmin/Maildir/.test1/new/1376225955.1555_1.host.ecomautomation.com Mode:None

So the question now is why the email clients, Thunderbird or Roundcube webmail, cannot pickup the mail in the ".test" and ".test1" directories?

So, it's getting closer.

Cheers, Noel

Aah, this looks good! You probably just need to tell Thunderbird to subscribe to / show the newly created IMAP folder. They don't usually do that by themselves when a folder just "appears out of thin air". :) So you need to enter their folder management, probably do a "refresh", and then subscribe to the folders.

If that doesn't help, the next debugging step would be to take a look at the file system on the server, if the mails have been stored in their proper directories there. You should see one file per email.

Thanks Locutus! I it's working now.

The folders in Thunderbird were already subscribed since I created them within Thunderbird within the INBOX.

I checked the file system in Maildir and noticed the folders were actually named: .INBOX.test .INBOX.test1

So I amended the delivery line in .procmailrc to:

$HOME/Maildir/.INBOX.$EXT/

Problem solved! And I am one happy camper :-D.

Cheers, Noel

Yay, glad it works now! And yeah, my example was for IMAP folders on the root level, as opposed to subfolders of INBOX or similar. :)