How Spamfilters Work

AllSpammedUp has a post describing the primary techniques anti-spam filters use to identify mail as spam or not spam. While is this not sender or delivery focused knowledge, it is important for people sending mail to have a basic understanding of filtering mechanisms. Without that base knowledge, it’s difficult to troubleshoot problems and resolve issues.

Any anti-spam system that is worth using will contain a range of preventative measures and features that are used to determine whether an email is likely to be spam or not.  As a complete solution they can be very effective, but taken individually and their weaknesses become more apparent. […] when you combine a number of different techniques into a single system, with each technique applying a “likelihood” score to each email that is checked, the system can be quite effective.
For example, if an email is from an IP address that is not considered a likely spam source (no score increase), but contains spam-like content (score increased according to severity), and fails sender verification (increases score again) , the combined “likelihood” score may reach the configured threshold for the system and cause the email to be treated as spam.

This is the concept I try to convey by using my bucket metaphor.

Related Posts

Delivery lore

Number of people believing outrageous statements on the Internet
(Image from Bad Astronomy)
Almost every delivery consultant, delivery expert or deliverability blog offers their secrets to understanding spam filters. As a reader, though, how do you know if the author knows what they’re talking about? For instance, on one of the major delivery blogs had an article today saying that emails with a specific subject line will not get past spam filters.
This type of statement is nothing new. The lore around spam filters and what they do and do not do permeates our industry. Most of the has achieved the status of urban legend, and yet is still repeated as gospel. Proof? I sent an email with the subject line quoted in the above blog post to my aol, yahoo, gmail and hotmail accounts. Within 3 minutes of sending the email it was in the inbox of all 4 accounts
I can come up with any number of reasons why the email ended up in my inbox, rather than being caught by spam filters as the delivery expert originally claimed. But none of those reasons really matter. The expert in question is spreading delivery lore that is demonstrably false. Emails with that subject line will get through spam filters. I even added an extra 4 exclamation points in the subject line.
Not all delivery lore is true. In fact, most lore involving “always” “all” “never” or “none” is not going to be true. Just because you read it on the internet, and because it came from someone claiming to know what they’re talking about does not absolve individual senders from critically thinking about the information.

Read More

Who is Julia and why won't she leave me alone?

There seems to be some new spam software in use. Julia <random last name> keeps telling me about her new webcam, how much she wants to date me and wants to know when I want to visit. These spams started February 1. I’ve had 179 caught by my MUA filters, and 152 caught by spamassassin (SA score >7 are filtered to a special account).
This is exactly the type of pattern that causes people to write filters that years later people look at and ask why someone thought this was a reasonable marker for spam.
The good folks over at MailChimp have examined some of the scoring rules that their clients trigger. They found some “Julia” type markers. Some oddities they reported on:

Read More

Confirmed opt-in

I spent the morning in multiple venues correcting mis-understandings of confirmed opt-in. The misunderstandings weren’t so much that people didn’t understand how COI works, but more they didn’t understand all the implications.
In one venue, the conversation centered around how small a portion of deliverability the initial subscription process affects. Sure, sending unwanted, unexpected email can and does cause reputation problems, but merely using COI as a subscription methodolgy doesn’t automatically give a sender a good reputation or good delivery. Senders using COI as a subscription practice need to also need to send relevant and engaging mail that their recipients expect to receive. They need to handle their bounces well and purge or re-engage inactive subscribers. They need to keep their complaints low and their responses high.
How you manage subscriptions is only one factor in reputation schemes, and even if the subscription method is COI other factors can negate any bonus involved.
The second conversation involved Ken challenging me on the comment I left on his quiz yesterday. I said COI wasn’t foolproof and he challenged me to explain how. I did, and he’ll be following up next week.

Read More