Updating the filtering model

laura
Industry
April 6, 2018

One thing I really like about going to conferences is they’re often one of the few times I get to sit and think about the bigger email picture. Hearing other people talk about their marketing experiences, their email experiences, and their blocking experiences usually triggers big picture style thoughts.
Earlier this week I was at Activate18, hosted by Iterable. The sessions I attended were interesting and insightful. Of course, I went to the deliverability session. While listening to the presentation, I realized my previous model of email filtering needed to be updated.

The old model

The old model was pretty simple. It was based on the idea that ISPs ask two fundamental questions about email when it comes in.

Is it safe?
Is it wanted?

If the answer to both those questions is yes, the mail is delivered to the inbox.
Safe is something we don’t talk much about in the marketing space, because generally our mail is safe. But our mail has to go through the exact same filters that are set out to catch the bad guys. And sometimes we do things that trigger that set of filters unintentionally.
The wanted is where marketers can really shine. ISPs look at their user behavior to determine if mail is wanted or not. While the measurements are slightly different than what marketers use, Marketers must also look at how wanted mail is. Engagement with the email is part of it. But, you can also use other metrics you have. Do they visit your website? Are they active on your FB page? What other data do we have that says this person is engaged with your brand and won’t object to increased volume.
Spammers send unwanted mail. Spammers send mail to a lot of addresses where the address owner doesn’t log in. Spammers send mail to people who don’t want it, so they delete it immediately. If you are sending lots of mail and your recipient demographic looks like the typical spammer recipient demographic, then your mail will be treated like spam. It’s ALL about the recipient. That’s what the ISP uses to measure your mail. And if your recipients are reacting to your mail in the same way they react to spammer mail, then you’re going to face deliverability challenges.
Increasing volume is expected, but you need to be strategic with how you increase it. I’m working with one of my clients right now to help them with the final touches on the holiday marketing program. They are increasing mail, but we made some changes to their proposed program to protect them against deliverability challenges. They’re in the early stages of warmup (yes, you need to warmup for significant volume changes!) and it’s looking good.
(That’s from an email I wrote to answer a question on the Only Influencers mailing list back in 2015)

The newer model

This week, though, I realized that another question snuck into the equation. The ISPs are still asking if mail is safe. Does it have harmful content? Phishing? Viruses? Is it coming from a botnet? ISPs use IP reputation and domain/URL reputation to answer the bulk of these questions during the SMTP transaction. If an ISP determines the mail isn’t safe, they’ll reject it out of hand.
But with technology improvements and machine learning, ISPs are able to split the second question into two sub questions.

Is it unsolicited?
Is it wanted by the recipient?

The unsolicited piece was always part of the equation. Many of the metrics used to answer the “is it wanted” question were actually trying to determine if the mail was unsolicited. These measurements include the things we talk about: bounce rate, complaint rate, unknown user rate. There are two reasons I think this is worth pointing out. The first is that I have often glossed over how much unsolicited mail “legitimate” senders actually send. Every company who purchases a list, who uses lead gen, who collects addresses at point of sale send unsolicited email. They may not mean to, but they do. The second reason is this is the piece of spam filtering that data hygiene companies are addressing. Everyone who cleans your list, identifies bounces, finds bad users, they’re specifically addressing this filtering. And many senders don’t understand why their mail is still going to bulk even after they’ve purchased very expensive hygiene services.
The reality is, that hygiene services make mail look less unsolicited, by removing many of the markers that tell ISPs the mail is unsolicited. But that doesn’t make the mail any more wanted by the recipient. Hence the current focus on engagement and individual delivery metrics. These are metrics that can’t be faked by the sender. Third parties can’t identify those recipients that want a particular piece of mail. And, with privacy laws like GDPR, it’s unlikely those business models will be cost effective.

Email marketing ulcers for the holiday

I’ve mentioned here before that I can usually tell when the big ISPs are making changes to their spam filtering as that ISP dominates my discussions with current and potential clients and many discussions on delivery mailing lists.
The last two weeks the culprit has been Yahoo. They seem to be making a lot of changes to their filtering schemes right at the busiest email marketing time of the year. Senders are increasing their volume trying to extract that last little bit of cash out of holiday shoppers, but they’re seeing unpredictable delivery results. What worked to get mail into the inbox a month ago isn’t working, or isn’t working as well, now.
Some of this could be holiday volume related. Many marketers have drastically increased their mail volume over the last few weeks. But I don’t think the whole issue is simply that there is more email marketing flowing into our mailboxes.
As I’ve been talking with folks, I have started to see a pattern and have some ideas of what may be happening. It seems a lot of the issue revolves around bulk foldering. Getting mail accepted by the MXs seems to be no different than it has been. The change seems to be based on the reputation of the URLs and domains in the email.
Have a domain with a poor reputation? Bulk. Have a URL seen in mail people aren’t interested in? Bulk. Have a URL pointing to a website with problematic content? Bulk.
In the past IPs that were whitelisted or had very good reputations could improve delivery of email with neutral or even borderline poor reputations. It seems that is no longer an effect senders can rely on. It may even be that Yahoo, and other ISPs, are going to start splitting IP reputation from content reputation. IP reputation is critical for getting mail in the door, and without a good IP reputation you’ll see slow delivery. But once the mail has been accepted, there’s a whole other level of filtering, most of it on the content and generally unaffected by the IP reputation.
I don’t think the changes are going to go away any time soon. I think they may be refined, but I do think that reputation on email content (particularly domains and URLs and target IP addresses) is going to play a bigger and bigger role in email delivery.
What, specifically, is going to happen at Yahoo? Only they can tell you and I’m not sure I have enough of a feel for the pattern to speculate about the future. I do think that it’s going to take a few weeks for things to settle down and be consistent enough that we can start to poke the black box and map how it works.

Ask Laura: Should we allow tagged addresses?

Do system administrators have too much power?

Yesterday, Laura brought a thread from last week to my attention, and the old-school ISP admin and mail geek in me felt the need to jump up and say something in response to Paul’s comment. My text here is all my own, and is based upon personal experience as well as those of my friends. That said, I’m not speaking on their behalf, either. 🙂
I found Paul’s use of the word ‘SysAdmin’ to be a mighty wide (and — in my experience — probably incorrect) brush to be painting with, particularly when referring to operations at ISPs with any significant number of mailboxes. My fundamental opposition to use of the term comes down to this: It’s no longer 1998.
The sort of rogue (or perhaps ‘maverick’) behavior to which you refer absolutely used to be a thing, back when a clean 56k dial-up connection was the stuff of dreams and any ISP that had gone through the trouble to figure out how to get past the 64k user limit in the UNIX password file was considered both large and technically competent. Outside of a few edge cases, I don’t know many system administrators these days who are able to (whether by policy or by access controls) — much less want to — make such unilateral deliverability decisions.
While specialization may be for insects, it’s also inevitable whenever a system grows past a certain point. When I started in the field, there were entire ISPs that were one-man shows (at least on the technical side). This simply doesn’t scale. Eventually, you start breaking things up into departments, then into services, then teams assigned to services, then parts of services assigned to teams, and back up the other side of the mountain, until you end up with a whole department whose job it is to run one component of one service.
For instance, let’s take inbound (just inbound) email. It’s not uncommon for a large ISP to have several technical teams responsible for the processing of mail being sent to their users: