Why is it Really So Hard to Tackle Spam?

Written by Paul Cunningham on August 5, 2009

damMy last post on international spam fighting attracted a comment from reader Andreas Kroll.  Andreas asks “Why is it really so hard to tackle spam?

That is a good question, and one we don’t often stop and think about.  The war against spam carries on with each side adjusting to the other’s new techniques with new ways of defeating them.  This constant shifting of the landscape makes anti-spam a very fluid, dynamic industry with rapid technology changes.  Of course to the regular person using their computer for email and internet access they are probably wondering what all of the clever people at anti-spam company are really doing about it.

Let’s take Andreas’ comment for example.

“Spam in itself is the repeated sending of (nearly) identical messages to a lot of people.”

This would be true if all spam messages were created equally.  I’m sure we’re all familiar with viagra spam, or Nigerian 419 spam, or lottery spam, but if you sat and looked at 10 viagra spam emails in your Junk email folder you won’t find two the same.  Spammers will simply use an email template with a series of variable portions, and run scripts to insert a variety of values into those fields.  A short spam email with just 10 fields, each with 10 possible values, means 10,000,000,000 unique spam emails can be produced.

“It cannot be so hard to distinguish a mail that reaches several thousand people from a mail intended for a small group or a single person.  (With the exception of newsletter postings, which you have (hopefully) opted in)”

Legitimate commercial email and newsletters contain many similar characteristics to a typical spam email such as commercial terminology, urgent tone, and a call to action (eg “click here”).

This is one of the major challenges faced by both anti-spam companies and by email marketers.  The anti-spam company doesn’t want to inconvenience businesses and email recipients by blocking legitimate marketing, and the email marketer wants to avoid being labeled a spammer and losing all of their customers because they can’t guarantee a high delivery rate.

“So if you can identify the mails and you can identify the routes these mail take, why can’t you go backwards in the routing step by step, identify the responsible server, identify the responsible admin and give him/her the choice to cooperate in the fight against spam or be excluded from mail traffic by a BAN list.”

There are two answers for this.  Firstly, not all spam originates from email servers.  A lot of spam will come from armies of compromised home and business computers called botnets.  Some of it will come from compromised or misconfigured servers such as open relays, or email servers not correctly set up to avoid backscatter.  Many of these botnets and open relays do end up on blocklists such as Spamhaus that can be used by email administrators to perform connection filtering.

“ISPs not willing to shut of spam senders will have to be shut off from the network completely. I cannot understand why a provider allowing to distribute that crap through his network is still on the internet.”

Back in November 2008 a victory against spam was won when an ISP responsible for as much as 75% of the internet’s spam was shutdown.  As effective as this was in the fight against spam unfortunately it only further highlighted the challenges of international spam fighting as the operators were able to simply set up shop elsewhere in another country and continue spamming the internet.

But for most legitimate ISPs outbound spam filtering is not something they can be very aggressive with, nor is it always going to be effective.  Firstly, an ISP does not want to disrupt their customers’ email by causing any false positives (legitimate email marked as spam).  False positives may damage their customers’ businesses and cause financial losses, which could lead to law suits against the ISP.

Secondly, most ISPs permit customers to send directly out to the internet over SMTP without having to relay through any ISP mail servers.  This is actually the preferred method for most email administrators because it makes troubleshooting email delivery much easier.

“Local law should make the ceo personally responsible for the damages.”

This is the main issue in the war on spam.  Anti-spam legislation (where it exists at all) is local, but the problem is global.  According to some reports Australia is one of the most spammed countries in the world, despite having strict anti-spam laws.  These laws are only effective at stopping Australian businesses from sending spam.  If a spammer in Russia or Korea sends me some spam then Australian authorities have no power to punish or stop them.

My answers to Andreas’ comment may make it seem like the fight against spam is hopeless, and that spammers are winning.  The reality is that while spam continues to be a problem causing nuisance and financial loss for businesses and individuals everywhere, any place that has an effective anti-spam solution in place that is from a quality vendor, is well configured, and is well maintained will be blocking most spam already.  Fighting spam is difficult and costs both time and time, but it can be done.

Comments

Donovan Hill August 7, 2009

Having been an email admin for a small (and now defunct) ISP, I found the most effective thing to preventing spam was to start by using a list like spamhaus, then block connections from hostnames that read like “ppp” or “client” or “dynamic” or any other indicator of a consumer internet connection. The final step was to block subnets as the spam came in. It was a lot of work for the first few months, but then is slowed down until I was spending maybe an hour a week dealing with the problem.

RobertSeattle August 7, 2009

I agree with Donovan, for the most part, no legitimate email should becomeing from a dynamic IP address pool, and those “legitimate” that are dynamic (perhaps like from a webcam on an dynamic IP that sends out an email when activity occurs) should be white listed manually.

Pingback: Understanding Blocklist Providers

  • (required)
  • (required)