Build a Mail Router with Qmail
Copyright © 2008 Rus Shuler, Enterprise Architect
What Does It Do?
It routes email. Specifically, it re-writes the email delivery address in the mail envelope, then delivers the email(s).
It could be used to-
What Do You Need?
Basically, you need a unix server running a patched and modified version of qmail, plus some custom scripts supplied here.
Specifically-
Assumptions
This is not a qmail tutorial nor a one-click install solution.
It assumes-
Setup Qmail + Add-Ons
Install qmail, patch it to version 1.05+, configure it, get it running, and add some things to it.
Setup Scripts
/etc/tcp.smtp
This script configures tcpserver, which is in the ucspi-tcp package. This dictates what can connect and what handles the connection.
192.168.0.:allow,RELAYCLIENT="",QMAILQUEUE="/var/qmail/filters/routemail.filter" 127.:allow,RELAYCLIENT="",QMAILQUEUE="/var/qmail/filters/routemail.filter" :allow,QMAILQUEUE="/var/qmail/filters/routemail.filter"
After you edit this file you must convert it to a binary file (/etc/tcp.smtp.cdb) for tcpserver. Use a script such as-
#!/bin/bash tcprules /etc/tcp.smtp.cdb /etc/tcp.smtp.tmp < /etc/tcp.smtp
/var/qmail/filters/routemail.filter
This is a shell script which runs qmail-qfilter. Qmail-qfilter invokes our Perl script which does the address transformation.
#!/bin/sh # invoked from /etc/tcp.smtp, runs qmail-qfilter instead of qmail-queue # qmail-qfilter invokes routemail.filter.pl to transform recipient addresses exec /var/qmail/bin/qmail-qfilter /var/qmail/filters/routemail.filter.pl
/var/qmail/filters/routemail.filter.pl
This Perl script reads a file line by line and transforms the email envelope addresses. The address transformations are stored in /var/qmail/addresses.
#!/usr/bin/perl # # rus.shuler - routemail.filter.pl - nov.2006 # # - reads /var/qmail/addresses for email addresses to transform. # - transforms mail envelope recipient(s) for redelivery by qmail by reading # file descriptor 3 and writing to descriptor 4. # - reads mail msgs from stdin and writes to stdout. # - this script is invoked via /etc/tcp.smtp and /var/qmail/filters/routemail.filter. # - qmail-qfilter is an add-on for qmail that makes this possible, see # http://untroubled.org/qmail-qfilter/ # read the address file and put the addresses in an array @addresses = (); $addrIndex = 0; open(ADDRESSES, " ) { if ( $line =~ m/^\s*(.*)\s*,\s*(.*)\s*\b/ && $line !~ m/.*#/ ) { $addresses[ $addrIndex++ ] = $1 . "," . $2; } } close ADDRESSES; # get envelope sender and envelope recipient from file descriptor 3, format is- # Fsender@somewhere.com\0Trecipient@here.com\0\0 # \0 = ascii zero (null), we translate these to tildes to make it easier to work with # read the envelope from file descriptor 3 open(ENVIN, "<&=3") or die "Can't open file descriptor 3 ($!), stopped"; $num_nulls = -1; if ( $line =) { $num_nulls = $line =~ tr/\0/~/; `logger -t routemail.filter.pl -p mail.info Envelope from qmail-smtpd is [$line]`; @envelope = split("~", $line); } else { `logger -t routemail.filter.pl -p mail.error No envelope from qmail-smtpd, rejecting`; exit 31; } close ENVIN; # iterate over the envelope addresses, remember the sender and transform the recipient(s) $sender = ""; for ( $i = 0 ; $i < scalar(@envelope) ; $i++ ) { # differentiate between the sender and recipient addresses if ( $envelope[$i] =~ m/^F(.+)$/ ) { # remember the sender address $sender = $1; } elsif ( $envelope[$i] =~ m/^T/ ) { # transform recipient addresses for ( $addrIndex = 0 ; $addrIndex < scalar(@addresses) ; $addrIndex++ ) { ( $oldAddr, $newAddr ) = split(/,/, $addresses[ $addrIndex ]); $envelope[$i] =~ s/$oldAddr/$newAddr/ig; } } } # rebuild the envelope and write it to file descriptor 4 $line = "F$sender~"; foreach $address ( @envelope ) { if ( $address =~ m/^T/ ) { $line = "$line$address~"; } } $line = "$line~"; `logger -t routemail.filter.pl -p mail.info Envelope to qmail-queue is [$line]`; $line =~ tr/~/\0/; open(ENVOUT, ">&=4") or die "Can't open file descriptor 4 ($!), stopped"; print ENVOUT $line; close ENVOUT; # open stdin to read the email message from qmail-smtpd via qmail-qfilter open(MSGIN, "<-") or die "Can't open STDIN ($!), stopped"; # open stdout to write the message to open(MSGOUT, ">-") or die "Can't open STDOUT ($!), stopped"; # define a var to track when the headers are done # ie $header=0 means we're done reading the headers $header = 1; # we're going to look for the Sender: header $senderHdr = ""; # read email line by line and write to stdout while ( $line = ) { # do header processing if ( $header ) { # look for and capture the Sender: header address if ( $line =~ m/^[Ss]ender:\s*(.+)\s*\n/ ) { $senderHdr = $1; } } # detect the end of the headers, a line with one char (a newline) should do it # see http://cr.yp.to/immhf/header.html for more info if ( $header == 1 && length($line) == 1 ) { $header = 0; # if there isn't a Sender: header then create one with the envelope sender if ( length($senderHdr) == 0 ) { print MSGOUT "Sender: " . $sender . "\n"; } } print MSGOUT $line; } # next email line close MSGIN; close MSGOUT; # exit codes for qmail-qfilter are: # # 0 = accept message and deliver it # 99 = accept message but do not deliver it # 31 = reject message with a 554 exit 0; # end
Permissions!
routemail.filter and routemail.filter.pl must be owned by root with their group set to qmail. They must be executable by everyone.
This will set everything correctly-
cd /var/qmail/filters chown root:qmail routemail.filter*; chmod 755 routemail.filter*
A directory listing of /var/qmail/filters should show this-
-rwxr-xr-x 1 root qmail 231 Nov 28 09:57 routemail.filter -rwxr-xr-x 1 root qmail 5030 Dec 1 11:25 routemail.filter.pl
Permissions are very important in qmail. If something isn't working as expected, check your permissions.
Configuring Transformations in /var/qmail/addresses
Here is an example /var/qmail/addresses file-
# addresses # --------- # # read by /var/qmail/filters/routemail.filter.pl # # format is [email address to replace], [replacement email address], or on more # general terms [anything to replace], [replacement] # # AN ADDRESS MAY GO THROUGH MULTIPLE TRANSFORMATIONS! # why? because the entire file is read for each address and processed line by line # e.g. support@mail.yourcompanysolutions.com > support@yourcompany.com > someone@yourcompanysolutions.com # --------------------- # local transformations # --------------------- # make sure all local accounts are accounted for if they may get mail we care about. postmaster@mail.mailserver.com, support@yourcompany.com # ----------------------- # corporate boxes & misc. # ----------------------- # use a "~T" to add another recipient address in the mail envelope support@yourcompany.com, dave@yourcompany.com~Ted@yourcompany.com # ------------ # ex-employees # ------------ hannibal@yourcompany.com, adam@yourcompany.com # ------------------------------------- # outbound or temporary transformations # ------------------------------------- contractor@yourcompany.com, judy@carolina.rr.com # -------------- # bullhorn users # -------------- # all bullhorn users must be specified jeff@yourcompany.com, jeff.logan@mail.bullhorn.com mark@yourcompany.com, mark.adams@mail.bullhorn.com # this should be the last transform, which will send all yourcompany.com mail to # another server. this must be the last transformation. @yourcompany.com, @yourcompany.provider.com
Other Stuff
Server With Dynamic IP
If you run this on a server with a dynamic IP address, e.g. at home, many mail servers will not accept your email. Yahoo is a good example. Yahoo will either not accept the email or relegate it to the spam folder. You can get around this by relaying mail through your ISP's mail server.
Edit /var/qmail/control/smtproutes-
# Artificial SMTP routes. # Each route has the form [domain]:[relay], without any extra spaces. If domain # matches host (cmd line param), qmail-remote will connect to relay, as if host # had relay as its only MX. (It will also avoid doing any CNAME lookups on recip.) # host may include a colon and a port number to use instead of the normal SMTP port. # # eg: somedomain.com:1.2.3.4 # somedomain.com:mail.somedomain.com # # make sure we deliver mail to [user]@theshulers.com to ourselves. if this line # weren't here then qmail-remote would attempt to connect to the firewall's # external address which doesn't work from inside. # this takes care of bounces generated from MAILER-DAEMON (qmail). theshulers.com:127.0.0.1 # route all mail through our local provider's server :smtp-server.carolina.rr.com
More on qmail-qfilter
Here are some notes regarding qmail-qfilter.
qmail-qfilter(1) qmail-qfilter(1) NAME qmail-qfilter - front end for qmail-queue that does filtering SYNOPSIS qmail-qfilter filter [ -- filter ... ] DESCRIPTION qmail-qfilter sends the message text through each of the filter commands named on the command line. Each filter is run separately, with standard input opened to the input email, and standard output opened to a new temporary file that will become the input to either the next filter, or qmail-queue. If the filter does not modify the message it passes unchanged to the next step. It also makes the envelope available to each filter as file descriptor 3. File descriptor 4 is opened to a new temporary file for the modified envelope, allowing the filter to modify the envelope or the message. If the fil- ter does not modify the envelope, the envelope remains unchanged for either the next filter or qmail-queue. This provides compatibility for existing filters that do not know about the envelope. qmail-qfilter also opens up file descriptor 5 to a temporary file. If this file is empty after all the filters have executed, its contents are read and used to specify a program to execute in place of qmail-queue. Each filter on the command line in separated with --. RETURN VALUES Returns 51 (out of memory), 53 (write error), or 81 (internal error) if it can’t create the temporary files or has prob- lems executing the filters. Returns 91 (bad envelope data) if it can’t read or parse the envelope data. If a filter returns anything other than 0 or 99, qmail-qfilter returns its exit code. If a filter returns 99, qmail- qfilter returns 0 immediately without running any other filters. Otherwise returns the exit code of qmail-queue. ENVIRONMENT For compatibility with previous versions, qmail-qfilter sets QMAILUSER and QMAILHOST to the user and host portions of the envelope sender address, and unsets QMAILNAME. It also sets QMAILRCPTS to the list of envelope recipients, each followed by a newline. It also sets ENVSIZE to the size of the envelope, MSGSIZE to the length of the message, and NUMRCPTS to the number of recipients. These values are updated before each filter is run. If QQF_QMAILQUEUE is set, its value is used in place of qmail-queue. SEE ALSO qmail-queue(8) NOTES $QMAILQUEUE is deliberately not used to override qmail-queue in order to avoid recursive loops with configurations that set $QMAILQUEUE to invoke qmail-qfilter itself. WARNINGS If you are using qmail-inject -n as one of the filters, you may want to unset MAILUSER, USER, and LOGNAME by using env -u QMAILNAME -u MAILNAME -u NAME qmail-inject -n as the command to invoke qmail-inject. Note that some the env command with some OS’s doesn’t support the -u option. A message with an excessive number of recipients (more than 64K bytes of recipient data on Linux) will cause execution of the filter programs to fail, and for the message to be rejected. The same temporary file is reused for file descriptor 5 for each filter. Make sure each filter writes a trailing ASCII NUL byte following the program name, as multiple filters could otherwise overwrite the value in undesirable ways. qmail-qfilter(1) -- The README file qmail-qfilter qmail-queue multi-filter front end Bruce GuenterVersion 2.1 2005-08-12 This program allows the body and/or envelope of a message to be filtered through a series of filters before being passed to the real qmail-queue program, and injected into the qmail queue. A mailing list has been set up to discuss this and other packages. To subscribe, send an email to: bgware-subscribe@lists.untroubled.org A mailing list archive is available at: http://lists.untroubled.org/?list=bgware Development versions of qmail-qfilter are available via Subversion at: svn://bruce-guenter.dyndns.org/qmail-qfilter/trunk Requirements: - bglibs is required for system dependancies. - This program is designed to take advantage of my QMAILQUEUE patch, which causes programs that would execute qmail-queue (such as qmail-smtpd etc.) to execute an alternative program. How to install: - Check the definitions at the top of qmail-qfilter.c, especially the value of TMPDIR. This should be set to a temporary directory that only the executor of qmail-qfilter has write access to. - Check the conf-* files for appropriate values for your compiler and linker, and installation paths. - Run "make" - As root, run "make install" How to use, with the QMAILQUEUE patch applied to qmail: - Create a script containing an invocation of qmail-qfilter. For example, a script that uses qmail-inject as a front end to qmail-queue would contain: #!/bin/sh exec /path/to/qmail-qfilter /var/qmail/bin/qmail-inject -n - Set the environment variable QMAILQUEUE to the location of this script. For example, in a SMTP rules files, put: A.B.C.D:allow,RELAYCLIENT="",QMAILQUEUE="/usr/local/bin/qmail-qftest" and rebuild the SMTP CDB file. - You're all set! In our example, all messages sent from the IP A.B.C.D will have their content filtered through qmail-inject, which will add missing "From:", "Date:", and "Message-Id:" headers. How to use, without the QMAILQUEUE patch: - Change the definition of QMAIL_QUEUE in qmail-qfilter.c to a different value, either by editing the source file or by modifying the DEFINES line in the Makefile to read: -DQMAIL_QUEUE=\"/var/qmail/bin/qmail-queue-old\" - Compile qmail-qfilter. - Rename qmail-queue to the new filename specified above. - Create a script to replace qmail-queue that contains an invocation of qmail-qfilter, as described in the previous example. - You're all set! All mail entering the queue will be filtered by your filter. Notes on writing a filter program: - If you want to block an email, exit from the filter with code 31. This will cause qmail-qfilter to exit with the same error code, and qmail-smtpd (for example) to send an error code to the client. - If you want to silently drop an email, exit with code 99. - The filter script that executes qmail-queue MUST NOT be setuid, and MUST BE readable. Only the real qmail-queue binary needs to be setuid. See the scripts in the "samples" directory for example scripts. This program is Copyright(C) 2001,2004-2005 Bruce Guenter, and may be copied according to the GNU GENERAL PUBLIC LICENSE (GPL) Version 2 or a later version. A copy of this license is included with this package. This package comes with no warranty of any kind. -- From http://untroubled.org/qmail-qfilter/ qmail-qfilter qmail-queue multi-filter front end Bruce Guenter Version 2.1 2005-08-12 This program allows the body and/or envelope of a message to be filtered through a series of filters before being passed to the real qmail-queue program, and injected into the qmail queue. A mailing list has been set up to discuss this and other packages. To subscribe, send an email to: bgware-subscribe@lists.untroubled.org A mailing list archive is available at: http://lists.untroubled.org/?list=bgware Development versions of qmail-qfilter are available via Subversion at: svn://bruce-guenter.dyndns.org/qmail-qfilter/trunk Requirements: - bglibs is required for system dependancies. - This program is designed to take advantage of my QMAILQUEUE patch, which causes programs that would execute qmail-queue (such as qmail-smtpd etc.) to execute an alternative program. How to install: - Check the definitions at the top of qmail-qfilter.c, especially the value of TMPDIR. This should be set to a temporary directory that only the executor of qmail-qfilter has write access to. - Check the conf-* files for appropriate values for your compiler and linker, and installation paths. - Run "make" - As root, run "make install" How to use, with the QMAILQUEUE patch applied to qmail: - Create a script containing an invocation of qmail-qfilter. For example, a script that uses qmail-inject as a front end to qmail-queue would contain: #!/bin/sh exec /path/to/qmail-qfilter /var/qmail/bin/qmail-inject -n - Set the environment variable QMAILQUEUE to the location of this script. For example, in a SMTP rules files, put: A.B.C.D:allow,RELAYCLIENT="",QMAILQUEUE="/usr/local/bin/qmail-qftest" and rebuild the SMTP CDB file. - You're all set! In our example, all messages sent from the IP A.B.C.D will have their content filtered through qmail-inject, which will add missing "From:", "Date:", and "Message-Id:" headers. How to use, without the QMAILQUEUE patch: - Change the definition of QMAIL_QUEUE in qmail-qfilter.c to a different value, either by editing the source file or by modifying the DEFINES line in the Makefile to read: -DQMAIL_QUEUE=\"/var/qmail/bin/qmail-queue-old\" - Compile qmail-qfilter. - Rename qmail-queue to the new filename specified above. - Create a script to replace qmail-queue that contains an invocation of qmail-qfilter, as described in the previous example. - You're all set! All mail entering the queue will be filtered by your filter. Notes on writing a filter program: - If you want to block an email, exit from the filter with code 31. This will cause qmail-qfilter to exit with the same error code, and qmail-smtpd (for example) to send an error code to the client. - If you want to silently drop an email, exit with code 99. - The filter script that executes qmail-queue MUST NOT be setuid, and MUST BE readable. Only the real qmail-queue binary needs to be setuid.
Enjoy!
Hope I haven't forgotten anything important. This really does assume you already have qmail running and want to add routing functionality. Send comments, suggestions and corrections to russhuler@yahoo.com.