SNIMTA (Keyword Filter for Mail Enable)

Home Page > Software > SNIMTA

Version 1.0.1626.35979 - Author: Jonathan Lampe

SNIMTA was used with the MTA facility of the Mail Enable email server to perform keyword matching on email messages and to delete or flag spam.

inaction.gif (77469 bytes)
Live screenshot from a mail server with SNIMTA logging to the Event Application Log and to a text statistics file (displayed/tailed here with "ntail.exe").

Installation

Minimum Requirements

No, I'm not kidding about these requirements!  Back in the day my company's primary mail server was a Pentium 133 with 128 megs of RAM, and it processed thousands of messages a day through Mail Enable, using about 500 restricted words, and only rarely did the whole processor get used!  (Idle usage was 67 total megs of memory - including virtual - when SNIMTA was in use memory use   "spiked" to about 70 megs.) 

Download

Final version 1.6.2.6:

Instructions

  1. Install the free .NET Framework from Microsoft (either version 1.0 or 1.1 will do) on your Mail Enable machine
  2. Make a subdirectory called "SNIMTA" in your "Mail Enable" directory.   (i.e. "c:\program files\mail enable\SNIMTA")
  3. Copy the contents of the SNIMTA.zip file into your new SNIMTA directory.

    mtainstall.gif (17207 bytes)
  4. Review and/or make changes to the SNIMTA configuration (see configuration section below).
  5. Configure your Mail Enable MTA:
    • Enable "Pickup Event"
    • Select "SNIMTA.exe" as the "program to execute on mail file"

mtaconfig.gif (26581 bytes)

Configuration

SNIMTA configuration is controlled by several different files:

Program Settings

Sample SNIMTA.cfg file:

DebugLevel = 5
WussMode = Yes
WussFlag = THIS_IS_SPAM
LogToEvent = Yes
LogToConsole = No
StatToFile = Yes
StatRecordOK = No
StatRecordLong = No
ReqToHeader = Yes
IgnoreSF = Yes

DebugLevel - Controls how much information is written to the (Event and/or Console) logs. In addition, if a debug level of 7 or more is specified, complete copies of control and message files will be copied to the "/stats" folder while messages are being processed; these copies can be used to reproduce an outright SNIMTA crash based on file content.  (Fortunately I haven't had to use this facility yet in practice, but if you do observe a crash, please loan me these file copies when you report the crash.) - Default = 3

chicken.gif (262 bytes) WussMode - If set to "Yes" SNIMTA will flag potential spam by putting the phrase "SNIMTA_SPAM" in the subject.  If set to "No" SNIMTA will simply delete potential spam. - Default = No

WussFlag - Controls which keyword will appear in the subjects of messages marked as spam. This option is only available in the production version. - Default = SNIMTA_SPAM

LogToEvent - If set to "Yes" SNIMTA will log information into the Windows NT/2K/XP Application Event Log - Default = Yes

LogToConsole - If set to "Yes" SNIMTA will attempt to log information to the command-line console, if any. - Default = No

StatToFile - If set to "Yes" SNIMTA will write out a rolling statistics log for each message and/or each potential spam message detected. - Default = No

StatRecordOK - If set to "Yes" SNIMTA will write out a statistics entry for every message processed.  If set to "No" SNIMTA will write out statistics entries only when potential spam is encountered. Applies only if StatToFile = Yes. - Default = Yes

StatRecordLong - If set to "Yes" SNIMTA will write out the full phrase in which a keyword or bad address was located into the last field of a statistics entry.  (This can be quite long if it is the complete body of a message.)   Applies only if StatToFile = Yes. - Default = Yes

ReqToHeader - If set to "Yes" SNIMTA will require that messages have someone listed as the TO or CC recipient.  (Setting this to "Yes" basically prohibits mail sent to 100% "BCC" addresses.) - Default = Yes

IgnoreSF - If set to "Yes" SNIMTA will NOT process messages forwarded in the system by the "SF" connector.  (You would probably want to use this option if you use "forward-only" mailboxes as aliases for other addresses on the system and don't want messages sent to an alias scanned twice.  You may want to turn this option off if you forward a lot of mail to other email addresses NOT hosted on the same machine.) - Default = Yes

LookupConfigByRecip - This option allows different domains to use different SNIMTA configurations. If set to "Yes" SNIMTA will look in an alternate configuration location for an SNIMTA.cfg file based on the domain->postoffice of the first recipient in the recipient list. By default, this alternate location is "[MailEnableProgramFolder]\PostOffices\[DomainPostOffice]\SNIMTA"; e.g., "d:\MailEnable\PostOffices\yahoo.com\SNIMTA". Any settings in an alternate configuration file OVERRIDE settings in the global SNIMTA.cfg file. Additional files (e.g. "bannedphrases.txt") must be replicated to individual configuration folders; the values in the global versions of these files are ignored if an alternate configuration is invoked. - Default = No

AltConfigLocation - This option allows administrators to safely delegate control over SNIMTA configurations to domain user by allowing SNIMTA configuration files to be anywhere on the system. This option is only used if LookupConfigByRecip is set to "Yes". If this option is set to any non-blank value, SNIMTA will replace the value of "[PostOffice]" in this value and use that location instead of a subfolder of a MailEnable post office. For example, an AltConfigLocation of "D:\Domains\[PostOffice]" would mean that SNIMTA would look for a configuration file named "D:\Domains\yahoo.com\SNIMTA.cfg". - Default = ""

Banned Addresses

Sample bannedaddresses.txt:

# BANNED ADDRESSES file
#
# List banned addresses, one per line.
# Partial addresses will match
# (e.g. "@yahoo.com" will match "ted@yahoo.com")
#
# Pound signs ("#") indicate comments
@sex.com

Banned Phrases

Sample bannedphrases.txt:

# BANNED PHRASES file
#
# List banned phrases, one per line.
# Partial addresses will match (e.g. "shit" will match "Takashita")
# Use special phrase "(b)" to indicate a leading/trailing space
# (e.g. "(b)cum" will match "Cum in" but not "scummy")
#
# Pound signs ("#") indicate comments
#
# Naughty words
adult channel
amateur tv

Ignore From Addresses

Sample ignorefromaddresses.txt:

# IGNORE FROM ADDRESSES file
#
# List addresses from which mail will not be scanned, one per line.
# (Use this file especially if you have a mass-mailer!)
# Partial addresses will match (e.g. "@yahoo.com" will match "ted@yahoo.com")
#
# Pound signs ("#") indicate comments
automation@mycompany.com
notify@mycompany.com

Ignore To Addresses

Sample ignoretoaddresses.txt:
(this file is only used in the production version) 

# IGNORE TO ADDRESSES file
#
# List addresses to which mail will not be scanned, one per line.
# (Use this file especially if you have multiple domains and want mail
# to one or more of your domains to NOT be scanned by SNIMTA!)
# Partial addresses will match (e.g. "@yahoo.com" will match "ted@yahoo.com")
#
# Pound signs ("#") indicate comments
@mydomain.com
fred@myotherdomain.com

Testing and Tuning

There are a lot of filters in the default set and you WILL want to make your own tweaks.  To do so, I suggest you build a test environment as described below.

First, grab a log tailing program (such as "ntail.exe") and set it to watch the current day's spam statistics file.  (i.e. "stats\statYYMMDD.csv", e.g. "stats\stat030224.csv")

Next, try sending some emails you KNOW will trigger the spam filter through the mail server.  If everything is working, new entries will show up in today's statistics file.  Now send some "tricky" emails with formatted text, large attachments, etc. to see if there is anything the filter is missing. 

Let the filter run "stock" for a day or two and eyeball it after a few hours to see if there were any false positives.  Also check your email and ask your coworkers if any spam slipped through that they expected the filter to have caught. 

When making changes, specific filters are usually better than general.  For example, "hot offers" might be a good phrase, but "offers" might catch too many things. When you make a change, immediately test it with the email which "slipped through" (if available) to see if your change actually did what you thought it would.

Technical Details

Banned Address Filtering

Addresses are checked against several fields in an email message:

Banned Keyword Filtering

Keywords are checked against several fields in an email message:

Special Leading/Trailing Space Designation

Use a "(b)" to indicate a leading or trailing space in a keyword list.   For example:

"top" will match "stop" but "(b)top" will not.

Phrase Normalization

The raw message body is never used for keyword comparison; instead it is first "normallized" to increase the likelihood of matching a smaller list of banned keywords.  For example, the following message:

"Hello Dolly,
You are the apple of my eye.  Why don't you c*m over tonight?"

Would be normallized to the following string before being scanned:

" hello dolly you are the apple of my eye why don't you c m over tonight "

Specific transitions which occur during "normalization":

A separate pass removes all HTML tags and then normallizes a string for a second comparison.  For example:

"Hot te<!-- apple pie-->en <b>action</b>"

Would be de-tagged and normallized to the following string before being scanned:

" hot teen action "

Frequently Asked Questions

Q: What does "SNIMTA" stand for?
A: "Standard Networks, Inc. Mail Transfer Agent"  More specifically, Standard Networks, Inc. is where I worked and was the first establishment to benefit from this little utility.  "Mail Transfer Agent" is the Mail Enable component into which this spam filter plugs.

Q: Is there any support available for "SNIMTA"?
A: No.

Q. Do I really have to install Microsoft's .NET framework?
A. Yes. You're free to port it to something else if you want. 

Q. How do I change the "SNIMTA_SPAM" subject tag to something else?
A. You now have two choices. You may either purchase the production version and use the "WUSSFLAG" option or purchase the source and do it yourself.   

Q: Why did you write SNIMTA?
A: I "celebrated" having the same email address for five whole years in 2003. Even though I answered about 50 legitimate emails a day, the ratio of spam/content in my unfiltered inbox would be about 98%/2%.  I had been using Eudora's filters for the past two years, but they are just not enough and a little unwieldy once you get past 50 filters.  (Also, after about 800 filters, they finally seemed to give up and started to send everything into the trash.)  After I noticed how ridiculously easy it was to plug into Mail Enable's MTA with a console application, I wrote a little anti-spam ditty to not only keep spam out of my mailbox, but everyone else's company mailboxes too.  

Q. Does SNIMTA handle attachments?
A. No.  I got SNIMTA working to the point where it doesn't butcher attachments to likely spam messages, nor does it try to do keyword searching against binary attachments, but it doesn't save off and process attachments.  (When and/or if SNIMTA does save off and process attachments, it will also become an anti-virus filter because anti-virus programs have a tendency to wipe, delete or lock viral files.)

Q. How do I report an error or send you my log files?
A. You don't - I no longer support this utility.

Q. Do you recommend using other anti-spam measures in conjuction with SNIMTA?
A. Absolutely. I used Mail Enable's DNS blacklists in addition to SNIMTA.

Q. How do I get the source code and/or the production version?
A. Use the "download" link above.

License

This software is provided AS IS, without warranty or guarantee of any kind.

As of May 4, 2010, this code has been released in to the public domain. Support is not provided under any circumstances.

Revision History

0.9.1151.18160 (February 25, 2003)

First public release.

0.9.1165.18952 (March 11, 2003)

New Feature: ReqToHeader configuration option
New Feature: IgnoreSF configuration option
New Feature: Ignore From address list
New Feature: Temporary control and message file copies made with high debug levels
Bug Fix: More loop prevention code added.

0.9.1259.14499 (June 13, 2003)

Bug Fix: Fixed code which caused all messages obtained through the MailEnable POP Retrieval interface to be marked as spam because MailEnable puts a blank value in the "sender=" header. (The fix both allows blank values in the "sender=" header when the using the POP interface AND properly checks the value of ReqToHeader when using other interfaces such as SMTP or SF.)

1.0.1325.37527 (August 18, 2003)

a.k.a. PRODUCTION RELEASE 1.0
New Feature: WussFlag configuration option
New Feature: Ignore To address list

1.0.1371.28520 (October 3, 2003)

Bug Fix: Fixed code which assumed that the format of "now()" was always United States string format. This fix addresses the "Failed to log statistics (IndexOutOfRangeException)" error encounted by many non-United States users (such as Germans).

1.0.1626.35979 (June 14, 2004)

New Feature: Added ability to pick up different configuration files for different domains. This was added with the LookupConfigByRecip and AltConfigLocation features.