Fighting Spam

created: Wed 11 July 2007 10:29:01
last updated:

Spam: its not going away and time soon. There are many strategies for dealing with it including, blacklists, whitelists, greylists, content filtering, bayesian filtering, checksum filters and many many more.

I am not going to provide a complete discussion here: just an explaination of my site-wide approach using qmail, courier-imap, maildrop and spamassassin. For a more general discussion check out Chris Hardie's qmail Anti-Spam HOWTO.

Overview

Incoming mail is handled by qmail and delivered as per each users ~/.qmail file. This allows users to choose whether or not they want spam to be handled by the server.

If there is no ~/.qmail file then mail is delivered by default to their Maildir, otherwise it is possible to specify for it to be handled by maildrop. If this is the case maildrop will pipe the email through spamassassin. Any mail thought to be spam will be moved to the users Spam folder, otherwise it will be moved to their Inbox.

qmail

There is nothing fancy about my qmail setup. If you don't already have qmail set up then a good guide is available at qmailrocks.org. In my system if there is no user specified option then the default in /etc/default/courier is: 

MAILDIR="Maildir"

This allows users to receive mail as normal and perhaps to do all the filtering with their mail client. Thunderbird does quite a good job of filtering spam as well as detecting scams.

If however the user wants the server to filter spam before it reaches their Inbox then they must create a file named .qmail in their home directory containing the line 

| maildrop

maildrop

This means email is now processed according to the instructions in /etc/maildroprc something like this:

SHELL="/bin/bash"

logfile "/var/log/maildrop.log"

DEFAULT="$HOME/Maildir"

xfilter "/usr/bin/spamc"

if (/^X-Spam-Status: *YES/)
{

    log "SPAM!!!"
        to "$DEFAULT/.Spam/"

}
else
{

 #   to "$DEFAULT"
 log "NOT SPAM!"

}

So here we are setting the default action to send mail to the Maildir. All mail is then filtered through spamassassin using the spamc client. It is then checked to see if spamassassin has determined it to be spam. If yes it is moved to the Spam folder which in this case must have been created by the user already. If it is not deemed to be spam it moves to the Inbox.

Learning Spam

If spamassassin has not correctly identified a message as spam then the user will have to manually move the message into a 'learning folder'. This should probably be a separate folder specifically for learning but I just used the same Spam folder again.

Every day a cron job runs to teach spamassassin using sa-learn. After 7 days messages in this folder are deleted. Create the file /etc/cron.daily/learnspam, make it executable and enter the following:

#!/bin/bash

/usr/bin/sa-learn --spam /home/*/Maildir/.Spam/cur/*

find /home/*/Maildir/.Spam/cur/ -type f -mtime +7 -exec rm {} \;

Double Bounces

I have also set maildrop to filter double bounces that go to the postmaster - which is me. I don't want to disable them completely: I'm just going to move all the failure notices to a separate folder. I do this using a .mailfilter file in my home directory so these rules apply only to me.

# personal mail filter

SHELL="/bin/bash"

logfile "$HOME/.mailfilter.log"

DEFAULT="$HOME/Maildir"

if (/^Subject:.*(failure notice)/)
{

    log "failed!!!"
    to "$DEFAULT/.Bounces/"

}
else
{

 #   to "$DEFAULT"
 log "pass my friend"

}

Thoughts

This is a very basic setup, maildrop in particular can do much more. My single folder here is not a good idea as false positives will be learnt as spam unless they are dealt with by the user the day they are received. I'll change this later.

Ideally spamassassin should be set up site-wide so that spam learnt for one user is learnt for everyone. Guidelines for doing that are on the spamassassin wiki