Re: [bitfolk] Shared spamd services

Hi David, On Tue, January 21, 2014 12:10 pm, David Stark wrote: nmap to find the open ports but I'm still at a loss on how this should be accessed, could anyone advise? I've been using Andy's spamd service for a few years now and am happy to share my method. There are many ways to skin this cat though so don't assume this is the 'best' way, however it has worked faultlessly for me so I wouldn't hesitate to recommend it. My mail routing/processing path is as follows: MAIL -> Postfix (via smtpd/port 25) -> spamc/spamd -> Postfix (via sendmail) -> Procmail -> User Note, in my setup Postfix doesn't actually perform any spam filtering but rather runs it via Andy's spamd service only for the purpose of adding spamassassin headers. I then use Procmail to deliver it to the user or, in the case of identified spam, handle it accordingly (see below). So, to tell Postfix to send all incoming mail via the spam check I modified /etc/postfix/master.cf to read: smtp inet n - - - - smtpd -o content_filter=spamcheck spamcheck unix - n n - - pipe flags=Rq user=spamcheck argv=/etc/postfix/spamcheckscript.sh -f ${sender} ${recipient} My /etc/postfix/spamcheckscript.sh looks like the following, the comments of which hopefully explain it: #!/bin/sh # spamcheckscript.sh Mathew Newton v0.1 08/03/2010 # # Script to submit mail for remote spam checking via spamc, handling # errors and responses accordingly without letting unchecked mail through # # Requires creation of 'spamcheck' system user and /var/spool/spamchecktemp # directory (owned by spamcheck:nogroup, permissions 0700) # # Intended to be called by Postfix's master.cf using the pipe command. Test by # by hand with /path/to/spamcheckscript.sh -f <sender> <recipient> < message-file # Set file locations and arguments INSPECT_DIR=/var/spool/spamchecktemp SPAMC="/usr/bin/spamc -x -d spamd.lon.bitfolk.com -u Debian-exim" SENDMAIL="/usr/sbin/sendmail -oi -G" LOGGER="/usr/bin/logger -s -p mail.warning -t spamcheckscript" # Exit code(s) from /usr/include/sysexits.h EX_TEMPFAIL=75 # Create a temporary file to store the spamc output (the scanned/modified message) cd $INSPECT_DIR || { $LOGGER "$INSPECT_DIR does not exist - nowhere to create temporary storage file"; exit $EX_TEMPFAIL; } SPAMC_OUTPUT="`mktemp -p $INSPECT_DIR`" || { $LOGGER "Cannot create temporary storage file"; exit $EX_TEMPFAIL; } # Clean up temporary file however the script ends trap "rm -f $SPAMC_OUTPUT" 0 1 2 3 15 # Submit mail for scanning, store the result (output) and assess response (status code) # Code will be 0 for a success (whether that be spam or ham) # Non-zero code (failure of some sort) will return $EX_TEMPFAIL thus causing Postfix to retry later $SPAMC > $SPAMC_OUTPUT RETURN="$?" if [ "$RETURN" != 0 ]; then $LOGGER "Temporary SpamAssassin execution failure (spamc returned error code $RETURN - see /usr/include/sysexits.h) - will retry later" exit $EX_TEMPFAIL fi # Optional success logging statement (comment out if not required) $LOGGER "Spamc returned code $RETURN (this does not reflect the level of spam suspicion - the analysis and score will be added to the message headers). Proceeding with mail delivery." # Mail scanned; proceed with delivery (with the headers modified by SpamAssassin) $SENDMAIL "$@" < $SPAMC_OUTPUT exit $? The script finishes off by submitting the message back to Postfix, but via sendmail and thus doesn't end getting checked again and stuck in a loop! The final step for Postfix is to deliver it to procmail - this is achieved with the following in /etc/postfix/main.cf: mailbox_command = procmail -a "$EXTENSION" I use procmail for final delivery so that I can perform whatever filtering I require e.g. out of office replies, mailing list postings to a certain folder, triggering SMS reports on selected e-mails etc. For the spam filtering aspect the following system-wide configuration in /etc/procmailrc directs mail accordingly (comments kept for guidance and alternative options): # /etc/procmailrc - System-wide procmail config file # Used as an alternative to Postfix's 'local' delivery agent in order to move # tagged spam (>5pts) into a user's Spam folder, and 'super spam' (>10pts) # somewhere else (e.g. /dev/null, another mailbox, etc - see below) # Recipes (rules) in here take precedence over per-user procmailrc files # (the recipient user's recipes are effectively tagged on the end) # Processing will stop after a receipe has 'fired' (e.g. delivered) # Set the 'ORiGinal MAILbox' as this is where mail will be dropped if the mail # cannot be delivered (if the disk is full it won't help much so will bounce) ORGMAIL=$HOME/Maildir/ # Default mailbox to use (set to be the same as above) if no recipes match DEFAULT=$ORGMAIL # Set system-wide logfile (intended only really to be used for testing) # Needs to be world-writable if we've set DROPPRIVS=YES as Procmail runs as # the recipient user? Root-only reads/writes are advised for privacy purposes LOGFILE=/var/log/procmail.log # Include an abstract of each delivered message to the logfile # Shows From:, Subject:, folder/file it was delivered to and the size (bytes) LOGABSTRACT=all # Keep root privileges for now to allow 'super spam' to be stored outside of # the original recipient's mailbox (particularly if we have a dedicated # spam mailbox without it having to be world writable) # Also good if performing detailed logging (in a root-read-only logfile) # We will drop the privileges later (uncomment the line below to drop now) #DROPPRIVS=YES # Set default mask for message files to ugo=rw (the default is u=rw but I # want to store system-wide 'super spam' in one of my mail folders hence I # need to be able to read/delete it despite it being owned by root:mail) # UMASK value will be subtracted from 777 to leave the mask to be used # (e.g. 000 gives 777 i.e. rwxrwxrwx although I think Procmail uses the x bits) # Will return to the default (077 i.e. rw-------) later UMASK=000 # Detect and handle all 'super spammy' (>10pts) mail to a central storage area # Option 1: Bin it permanently (in /dev/null) # Option 2: Place in a dedicated spam mailbox (e.g. /var/spool/spambin/) # Option 3: Forward to a dedicated spam address # Option 4 (Chosen): Place in a dedicated folder in my mailbox :0: * ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\* /home/mathew/Maildir/.Spam-SystemTrashed/ # Option 1: Bin it permanently (in /dev/null) #:0: #* ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\* #/dev/null # Option 2: Place in a dedicated spam mailbox (e.g. /var/spool/spambin/) #:0: #* ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\* #/var/spool/spambin/ # Option 3: Forward to a dedicated spam address #Don't use lockfile (i.e. no trailing : below) #:0 #Detect 'super spam' but only if it hasn't already been through this recipe #* ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\* #* !^X-Loop: definitespam(a)newtonnet\.co\.uk #Add a loop-detection header before forwarding it on (to avoid loops however #it still seems to loop if the original recipient was me as that's where the #'forwardee' will be redirected to) #Note: Requires a recipe in my .procmailrc to filter this off to a specific #folder based on the detection of the X-Loop header #| formail -A "X-Loop: definitespam(a)newtonnet.co.uk&quot; | /usr/sbin/sendmail -oi definitespam(a)newtonnet\.co\.uk # Return the mask to default (i.e. recipient user rw only) from here on UMASK=077 # Drop any high(er)-level privileges from here on so as to store mail # as the recipient user (and group) DROPPRIVS=YES # Detect and deliver tagged spam (>5pts) into the recipient's Spam folder # If the folder doesn't exist it will be created, but not subscribed to - # force a subscription by adding 'Spam' to the user's subscriptions file # e.g. echo Spam >> ~<user>/Maildir/subscriptions or use a client # Squirrelmail will automatically mark the 'Spam' folder as special hence # carry it at the top of the folder list # Note 1: Could look for ^X-Spam-Flag: YES (which defaults to 5 and above) # but using stars allows me to tweak if necessary # Note 2: Needs loop-detection activating if using Option 3 (forwarding) for # the 'super spam' test above :0: * ^X-Spam-Level: \*\*\*\*\* $HOME/Maildir/.Spam/ # Note: If processing gets this far (i.e. not spam) then the message is # delivered to $DEFAULT (i.e. the user's mailbox) unless a recipient's # .procmailrc instructs otherwise And that's it! Ham gets delivered as normal, suspect spam (5-9 points) gets put in the user's Spam folder and 'definite' spam (>10 points) gets put into a Spam-SystemTrashed folder in my mailbox which gets automatically pruned of old content. The reason I did this was so that nothing got outright rejected and I would always be able to recover a false positive for users even if it somehow failed the spam test massively. In practice this hasn't been necessary but it doesn't cause any issue other than having a message folder with 30k messages in it at any one time! Hope that helps, if only as food for thought. I'd be happy to provide further details if required. Mathew