Insight into Gmail filtering

Last week I posted a link to an article discussing how Gmail builds defenses to protect their users from malicious mail. One of the things I found very interesting in that article was the discussion about how Gmail deploys many changes at once, to prevent people from figuring out what the change was.
Let’s take a look at what Gmail said.

Make it hard for attackers to understand your defenses – Use overwhelming force and deploy many countermeasures at once
This is probably the most subtle of the lessons. Attackers constantly probe systems to find loopholes. For example, at some point one of Gmail’s spammers became very astute at finding bugs in our parsers and started to find very subtle bugs he could exploit. For example, he realized he could use the @ ambiguity (it is used in email addresses and in http links) to confuse our parsers and for a brief period of time he successfully evaded detection. This is why it is very important to make probing more difficult for attackers by rolling out multiple changes. That way they are overwhelmed by the number of things to test and can’t easily figure out what changed.
Bottom line: When rolling out change in your defenses, don’t rush (too much) and release multiple changes at once.

I cannot tell you the number of people who have approached me – in person at conferences, on twitter, through email, on LinkedIn – asking if I knew, “What gmail changed this week.” Now, at least, I have an answer. “Gmail changes a lot of things at once in order to stop people from figuring out the filters.”
I’ll be honest, I stopped trying to probe Gmail’s filters to identify ways around them a couple years ago. They are just too hard to evaluate. Sure, I can identify certain things to change that will get email into the inbox, briefly. But unless the underlying issues were fixed, the filters catch up and the mail will go back to the bulk folder. Sometimes it takes the filters days to catch up, sometimes it can take hours.
In any case, probing the filters to see what they’re doing is a very short term, limited fix to Gmail problems.
What I’ve focused on, with my clients, is getting the filters to work for them. We know that modern filters don’t treat all mail from a single company, IP or domain equally. Instead they make delivery decisions for each individual recipient of that mail. Those of you who have seen some of my talks may have seen this image before.
IndividualRecipPreferences
Things like IP reputation, domain reputation, content reputation and link reputation all contribute to the reputation of an email. If the reputation is very bad, the mail is bounced and no body receives it. But if the mail isn’t bounced, then they go through the individual recipient preferences. It is the combination of individual preferences and email reputation that determine where the mail ends up for each recipient. Different recipients may get mail differently.
This is why engagement is so important in email. Sending to people who want to receive the mail improves overall inbox delivery. If most of your recipients want your mail than chances are if you mail someone new, they’ll want your mail, too.
Gmail has a goal with their email delivery. You can make filters work for you by sending mail that users want and engage with. If you’re having problems with Gmail delivery focus on the recipients and making them happy. Don’t waste time trying to troubleshoot a filter change. Gmail isn’t going to make it easy for you.
 
 

Related Posts

Thanks for the great session

I had a great time answering questions at the 2015 All About eMail Virtual Conference & Expo today. Thanks so much to everyone who participated and asked questions. They were great and I’m sorry we didn’t have more time.
I did get some questions on twitter (@wise_laura) afterwards. One was about an example I gave to explain how filters are complex. There have been rumors going around recently that Gmail is filtering mail with more than 3 URLs in it. Let me just say right now THIS IS NOT TRUE emails with more than 3 URLs in them are being delivered just fine to Gmail.
There is a situation involving the number (and type) of URLs that I think are a useful example of the filter complexity happening at some places, like Gmail. I started working on it, but don’t quite have time to finish it today, but will keep working on and it should go up in the next day or so.
Thanks again to everyone who joined the session. You asked some great questions and I had fun answering them.
 

Read More

Setting expectations at the point of sale

In my consulting, I emphasize that senders must set recipient expectations correctly. Receiver sites spend a lot of time listening to their users and design filters to let wanted and expected mail through. Senders that treat recipients as partners in their success usually have much better email delivery than those senders that treat recipients as targets or marks.
Over the years I’ve heard just about every excuse as to why a particular client can’t set expectations well. One of the most common is that no one does it. My experience this weekend at a PetSmart indicates otherwise.
As I was checking out I showed my loyalty card to the cashier. He ran it through the machine and then started talking about the program.
Cashier: Did you give us your email address when you signed up for the program?
Me: I’m not sure, probably not. I get a lot of email already.
Cashier: Well, if you do give us an email address associated with the card every purchase will trigger coupons sent to your email address. These aren’t random, they’re based on your purchase. So if you purchase cat stuff we won’t send you coupons for horse supplies.
I have to admit, I was impressed. PetSmart has email address processes that I recommend to clients on a regular basis. No, they’re not a client so I can’t directly take credit. But whoever runs their email program knows recipients are an important part of email delivery. They’re investing time and training into making sure their floor staff communicate what the email address will be used for, what the emails will offer and how often they’ll arrive.
It’s certainly possible PetSmart has the occasional email delivery problem despite this, but I expect they’re as close to 100% inbox delivery as anyone else out there.

Read More

Thoughts on Gmail filtering

Gmail has some extremely complex filters. They’re machine learning based and measure hundreds of things about incoming mail. The filters are continually adjusting to changes and updating how they treat specific mail.
One consequence of continually adjusting machine learning filters is that filtering is not static. What passes to the inbox now, may not pass in a couple hours.
One of the other challenges with Gmail filters is that they look at all the mail mentioning a particular domain and so affiliate mail and 3rd party mail can affect delivery of corporate mail.
The good news is that continually adjusting filters adapt to positive changes as well as negative ones. In fact, I recently made a segmentation suggestion to a client and they saw a significant increase in inbox delivery at Gmail the next day.
Gmail can be a challenge for delivery, but send mail users want and mail does go to the inbox.

Read More