The inbox is a moving target

The more I look at the industry, the more convinced I am that we’re in the middle of a fundamental shift in how email is filtered. This shift will change how we handle email deliverability and what tools we have and what information we can use as senders to address challenges to getting to the inbox.

Early deliverability

This period is roughly between 2001 and 2006. Many of the email filters were focused on blocking unsolicited email. They were mostly IP based and were very all or nothing. Mail is either accepted or it’s not. It wasn’t until 2003 that Yahoo and AOL (now OATH) announced spam folders that they would put mail into. There were few formal communication channels between ISPs and mail senders.
We did, of course, have informal channels. Often delivery could be improved by asking the right people to make some adjustments to their internal filters. And, to be honest, there were some people who used this position of power to personally enrich themselves (the good news is most of them are no longer in the business).
As senders, there was very little information we had to work with. Most communication with ISPs went through gatekeepers. The primary gatekeepers were the companies providing certification services: Bonded Sender, Habeas, Goodmail.

First inflection point

The first inflection point happened in the middle of the 2000s. Over the course of a few years, there were some significant changes all of which served to create the email industry as we now know it. This post certainly isn’t going to cover the full range of changes, but there are a few notable things that happened.
Expansion of broadband. As broadband availability expanded and became more affordable, many of the old dialup ISPs simply folded. With it, folks stopped using ISP provided email addresses and moved to free email addresses provided by companies like Yahoo and Hotmail.
Gmail. Google launched the beta for Gmail in 2004. Most of what I remember about this was invites to the private beta being shared wildly. Everyone wanted to get in. Initial buzz aside, Gmail is consistently one of the top 3 domains for consumer and business (through hosted G Suite accounts) mailings alike.
Postmaster pages. ISPs started codifying what it took to get mail delivered to that ISP. One example is how pages published what they expected in terms of connections and messages per connection. Those pages also started explaining some of the SMTP return codes ISPs were using.
Feedback loops. In 2006 ISPs all over the US started announcing Feedback Loops. What a lot of folks don’t realize is this expansion in FBLs was driven in part by Return Path’s willingness to provide infrastructure to the ISPs to manage FBL registration.
These changes, along with some others, worked together to create a delivery environment that, while not always welcoming to bulk mail, was understandable.
For the next 5 to 8 years, this was the environment we worked in. Sure, there was some consolidation in the email space: ESPs were launched and bought, new deliverability companies arose, some accreditation companies folded. Overall, though, it was pretty solid and companies that followed a few simple rules were able to reach the inbox.

Present day

Today, though, I’m seeing a lot of those rules changing. Companies who keep their bounces and complaints low are still sometimes seeing delivery problems. They’re not able to reach the inbox, for at least some of their mail streams. Again, there are a number of things happening concurrently that is driving this change.
ISPs are changing. Two of the 4 major consumer ISPs Microsoft, AOL, Gmail, Yahoo (MAGY) providers have merged, to form what I’m fondly calling OMG (OATH, Microsoft, Gmail). We’re also seeing consolidation in the cable space with the Time Warner / ATT merger. Other cable companies are merging, too, and rethinking things like their postmaster pages. Additionally, FBLs are going away and no ISP offers whittling anymore.
Privacy changes. We’re also seeing a global focus on privacy and tracking and user data. A lot of this is translating into how the email providers and ISPs are handling incoming email. A lot of the changes we’ve seen in the last 12 months was driven, in part, by ISPs trying to upgrade their infrastructure to deal with the effects of GDPR. But California is passing its own GDPR style law, which will affect technology companies across the US.
Security Concerns. Email is a primary source of infections inside corporations. All of the major breeches we’ve heard about in the past few years have stared with email. Filters aren’t about spam any more, they’re about protecting end users and corporations from malicious traffic. This is going to affect how mail is processed and delivered and what can and cannot reach the inbox.
This is a very abbreviated list of the things that were happening at each stage in the process.
One of the bigger challenge  is the ISPs are moving to less human involvement in filters and answering deliverability questions. They’re moving to the Hotmail “fill out the form and parse the boilerplate” model of deliverability support. Or they’re moving even further to the Gmail style “here’s a help page, have fun!” Even folks who know people inside the ISPs, are having problems getting the same helpfulness of answers that they had in the past. I’m not blaming the ISPs or the folks who work there. But, as I see it, providing support to senders was often a way for ISPs to tweak their filters. Now that the ISPs are using machine learning and AI as a way to filter mail, they need less in the way of direct feedback.
Overall, I think we’re in the early stages of another major shift in how email is managed at the ISPs. This will change how we have to send emails in order to reach the inbox. We have to think about the filters and what their goals are. Having good stats isn’t going to be enough. Almost anyone can have good stats, and there are large companies senders can pay to clean up their stats to be within the ISP thresholds. Once you can buy low bounce rates or low complaint rates, those stats are useless for determining the health of your email program.
Smart senders will start thinking about these issues sooner rather than later.