BLOG
Thinking about filters
Much of the current deliverability advice focuses on a few key ideas:
- Authenticate your mail with SPF, DKIM and DMARC
- Use a dedicated IP.
- Monitor delivery.
- Clean your data.
All of these things are absolutely things you should be doing, but senders can do all these things and still have cruddy delivery. These things are great and can help your mail deliver better. But they’re not enough to get mail into the inbox.
As I think about email filters I can put them in some different broad categories based on the email types they target.
- Safety filters – block phishing and malware
- Unsolicited filters – block mail that looks like it’s unsolicited
- Unwanted filters – block mail that is unwanted by the recipient
Safety filters are pretty self explanatory. They use different signals like source IP, links, and virus signatures to block mail. Some safety filters are about protecting the infrastructure, so they act on senders that open too many connections, or send non-RFC compliant mail or act in ways that are not well-behaved.
Unsolicited filters have their own signals they use to determine if a message is unsolicited. These include a lot of the things we talk about in delivery. Things like spam complaints, bounces, and spamtrap hits are all things that indicate recipients never opted in to receive a particular email.
Unwanted filters use some of the same signals as unsolicited filters, the difference between unsolicited email and unwanted email is somewhat subtle. Signals for unwanted filters are the things we describe as engagement metrics. When emails are read and saved and moved between folders that tells the ISP these mails are wanted. Emails that are deleted without opening are likely unwanted.
Understanding what kind of mail filters are targeting helps drive what types of fixes we do. If the problem is our users don’t want the mail then removing bounces isn’t going to affect the signals driving the filter.
On the flip side, if the problem is the mail looks unsolicited because it has too many dead addresses and hits spamtraps, we can sometimes fix them using the same techniques we use for addressing unwanted filters. When we limit sending to people who have opened and engaged with the messages, we’re removing the addresses that signal the email is unwanted and the email is unsolicited.
Filters like Spamhaus focus on unsolicited emails. That’s why they focus on making sure that senders have permission rather than focusing on engagement metrics.
Understanding what type a mail a filter is attempting to protect users from is crucial for solving delivery problems.
Unfortunately the “Unwanted filters” would rely on the recipients’ behavior, such as “are those messages read?” or “would the user take it out of junk folder?”. While both actions usually means something, there are cases where the machine can’t guess what the user wants. And that’s ok.
What’s not ok is that the machine is still trying, and fails.
There are emails I want to get and keep even if I don’t read them often, or at all. Could be transactional, order confirmation, invoice, monitoring alerts, or even informative content I like but barely have time to read. I wouldn’t agree to seeing those in junk folder, or rejected.
My educated opinion is that users can understand how a mechanism works, if it is consistent, while unpredictable machine learning actions are at least confusing for the user, if not frustrating.
Do those emails you want tend to end up in your inbox or your bulk folder?
Depends on the webmail. Gmail most of the time gets it right (only the “important” flag seems to be used randomly), Outlook behaves erratically, Yahoo seems fine
Hotmail is a hot mess, so I’m not surprised they’re being erratic. Gmail and Yahoo getting it right doesn’t surprise me, either. They’ve had pretty complex (and mostly accurate) filtering for a while.
But think your own experience says that the filters are pretty accurate, even with transactional mail that’s not often interacted with. And I’m not seeing how it’s that confusing for the general recipient – the filters mostly do what they want.
Ok I’m directly looking at Hotmess then. Wanting to keep spam out of inbox makes sense. Wanting to guess what the user would maybe want is prone to mistakes. If a system is imperfect, but consistently, then the user will work around the imperfections (or find another system). But if it works imperfectly and erratically, only one option remains.