Insight into Gmail filtering

Last week I posted a link to an article discussing how Gmail builds defenses to protect their users from malicious mail. One of the things I found very interesting in that article was the discussion about how Gmail deploys many changes at once, to prevent people from figuring out what the change was.
Let’s take a look at what Gmail said.

Make it hard for attackers to understand your defenses – Use overwhelming force and deploy many countermeasures at once
This is probably the most subtle of the lessons. Attackers constantly probe systems to find loopholes. For example, at some point one of Gmail’s spammers became very astute at finding bugs in our parsers and started to find very subtle bugs he could exploit. For example, he realized he could use the @ ambiguity (it is used in email addresses and in http links) to confuse our parsers and for a brief period of time he successfully evaded detection. This is why it is very important to make probing more difficult for attackers by rolling out multiple changes. That way they are overwhelmed by the number of things to test and can’t easily figure out what changed.
Bottom line: When rolling out change in your defenses, don’t rush (too much) and release multiple changes at once.

I cannot tell you the number of people who have approached me – in person at conferences, on twitter, through email, on LinkedIn – asking if I knew, “What gmail changed this week.” Now, at least, I have an answer. “Gmail changes a lot of things at once in order to stop people from figuring out the filters.”
I’ll be honest, I stopped trying to probe Gmail’s filters to identify ways around them a couple years ago. They are just too hard to evaluate. Sure, I can identify certain things to change that will get email into the inbox, briefly. But unless the underlying issues were fixed, the filters catch up and the mail will go back to the bulk folder. Sometimes it takes the filters days to catch up, sometimes it can take hours.
In any case, probing the filters to see what they’re doing is a very short term, limited fix to Gmail problems.
What I’ve focused on, with my clients, is getting the filters to work for them. We know that modern filters don’t treat all mail from a single company, IP or domain equally. Instead they make delivery decisions for each individual recipient of that mail. Those of you who have seen some of my talks may have seen this image before.

Things like IP reputation, domain reputation, content reputation and link reputation all contribute to the reputation of an email. If the reputation is very bad, the mail is bounced and no body receives it. But if the mail isn’t bounced, then they go through the individual recipient preferences. It is the combination of individual preferences and email reputation that determine where the mail ends up for each recipient. Different recipients may get mail differently.
This is why engagement is so important in email. Sending to people who want to receive the mail improves overall inbox delivery. If most of your recipients want your mail than chances are if you mail someone new, they’ll want your mail, too.
Gmail has a goal with their email delivery. You can make filters work for you by sending mail that users want and engage with. If you’re having problems with Gmail delivery focus on the recipients and making them happy. Don’t waste time trying to troubleshoot a filter change. Gmail isn’t going to make it easy for you.