Antispam Frameworks Explained

There are a vast number of different email hygiene solutions on the market today offering protection from viruses, malware, phishing, and spam for customers of all sizes.  Typically these products are built on a combination of several prevention techniques such as content filtering, RBLs, reputation filtering, and safe lists.

Some products also support one or more of a relatively new type of prevention – email authentication.

What is Email Authentication?

When the SMTP protocol was first created all users were trustworthy and hence there was no need to include any significant level of security within the protocol.  This has lead to many of today’s problems such as address spoofing.  Several  email authentication schemes have appeared on the scene to try and authenticate that an email using different methods, each with positive and negative aspects.

Sender Policy Framework

Sender Policy Framework (SPF) allows domain owners to use DNS TXT records to specify which email servers are allowed to send email for that domain.  This technique works on the assumption that the DNS records for a domain name are correct and trustworthy.  However there are a few weaknesses with this approach.

Firstly there has not been widespread adoption of this method by domain owners.  As such it is not practical for email administrators to block emails that fail an SPF test.  For example, if the owners of the domain example.com have no SPF record in their DNS zone then spammers are free to continue forging example.com email addresses.

The method also does not prevent spam being sent through authorized servers.  Senders can still forge the email address of other users of that domain, which makes it ineffective for blocking spam sent from popular web-based email systems that have millions of users.  Furthermore, the SPF test assumes that the authorized server is not an open relay or otherwise compromised.

SPF also breaks completely when email forwarding occurs, which is very common.

Finally, SPF is impractical in any scenario where emails are being sent via an unexpected server, such as home users sending via their ISP.  The domain owner would need to add SPF records for all possible servers that their end users might need to use, which is a significant administrative burden.

DomainKeys Identified Mail

DomainKeys Identified Mail (DKIM) is a technique whereby an email sender adds a cryptographic signature to the email header that can be used by the recipient to verify both the source and the integrity of the message content itself.  A receiving server can check the signature using a public key published in the sending domain’s DNS zone to prove that it was sent by an authorized server.  Although this may be more effective than SPF it also carries some weaknesses.

The signature-based authentication can be broken by any modification of the email in transit, such as a message footer inserted by anti-virus software.  Other common factors such as mailing list servers also cause message contents to be modified went forwarded on to their destination.

As with SPF DKIM is also ineffective at preventing spam sent by people abusing authorized servers, again this is a serious problem with popular web-based email systems.

DKIM also imposes significant processing overhead on the receiving server, which may cause load and scalability issues for businesses wishing to adopt this technique to prevent spam.

However one of the advantages of DKIM is that it can be used to better identify non-spam emails when used in combination with other techniques.  For example, an email that passes an open relay test for the sending server (either at the time of receipt or by membership on a list of known safe mail servers) that also passes a DKIM test is likely to not be spam and can bypass any further processor intensive scanning such as content filtering.

Penny Black

Penny Black is the codename for a Microsoft project taking a unique look at preventing spam using a “sender pays” system.  Under such as system the cost of sending unsolicited email is increased to make it impractical for spammers to send millions of spam emails.  The cost is not monetary, rather it is applied as computational effort.  Where currently an email message takes fractions of a second to send, Microsoft proposes increasing this to a much larger amount such as 10 seconds.

For normal email users this would cause few problems because they send a low volume of email.  Spammers would be forced to invest in more computing resources to continue sending out millions of spam emails, however this would not be an issue for them as long as they can still leverage large botnets of compromised pc’s across the internet.

The proposal only impacts unsolicited email.  For large corporate networks who engage in high volumes of email they can whitelist the domains of trusted partners so as not to impose the additional costs on them.  This may become easier as enterprise email systems such as Exchange 2010 introduce new features like Federation that allow secure authentication and sharing between organisations.

Summary

Development of anti-spam frameworks continues and there are positive outcomes within reach.  At this stage though it is unclear which of the frameworks will become widely adopted and become part of the standard for email communication.  A likely outcome will be that more than one framework is adopted and each is treated a separate calculation of the “likelihood” of an email being spam as opposed to a definitive pass/fail result.  For businesses choosing an anti-spam solution today the best approach is to implement one that already contains support for one or more of these developing frameworks.