Reputation is in the eye of the beholder

A few years ago reputation was generally recognised as one thing. If a sending reputation or IP reputation was good in one place it was likely good in other places. Different entities mostly reputation using the same set of signals albeit slightly tweaked to meet their own needs. More recently there is a divergence in how reputation is measured, meaning delivery can be vastly different across entities.

In simpler times reputation was based on the number of complaints an email generated and the number of non-existent addresses a sender hit with some spamtrap and blocklist data thrown in for good measure. These were pretty easy for each recipient domain to measure and, in fact, most senders could measure them as well. These measurements mostly reflected how well the sender collected and managed data. We could assume that data management was the same no matter what the recipient ISP was. Thus, we could assume reputation was mostly the same across domains.

As spammers adapted to reputation, filters had to evolve to keep mailboxes clean for their users. In most cases, the evolution was based on the data the recipient domains had access to. For webmail providers that manage the mail client as well as the mail server, they had a lot of data related to how users interacted with mail. Thus, engagement emerged as an important metric for those providers. There are a lot of providers, including cable companies, third party filters and blocklists, that don’t have any access to engagement data. They had to find other things to use.

The different data sources drove significant divergence among filters and that changes how we have to cope with them.

For example, engagement filters are really only relevant at the consumer webmail providers. There are two reasons for this. First, they have the data to see how folks interact with mail. Second, they’re the ones who care. The webmail providers want to delivery mail their users want. Thus, they’re going to measure what mail their users want. Other providers can’t get the data, as they don’t control the mail client. In some cases the user’s needs are not as important as the customer needs, like when the provider manages inbound mail for a business.

Cloud MX providers, that accept mail filter it and either deliver directly to the users inbox or to another MTA, often can’t or won’t look at engagement data. But they do have access to traffic data across thousands of different domains. They can use that traffic data as part of their reputation calculations.

There are non-email sources of data as well. For instance, many of the commercial email filters are maintained by companies that also track all types of security events online. Many of these companies include this data in their email reputation metrics as well.

All of this means that we can’t treat reputation as a monolith. It’s no longer the case that the inputs are the same and the algorithms are tuned a little differently. Instead, we’re in a situation where filter maintainers have access to very different types of data and thus the reputations are really specific to the provider measuring the reputation.