BLOG

Twisting information around

One of my mailing lists was asking questions today about an increase in invitation mailings from Spotify. I’d heard about them recently, so I started digging through my mailbox to see if I’d received one of these invites. I hadn’t, but it clued me into a blog post from early this year that I hadn’t seen before.
Research: ESPs might get you blacklisted.
That article is full of FUD, and the author quite clearly doesn’t understand what the data he is relying on means. He also doesn’t provide us with enough information that we can repeat what he did.
But I think his take on the publicly available data is common. There are a lot of people who don’t quite understand what the public data means or how it is collected. We can use his post as a starting off point for understanding what publicly available data tells us.
The author chooses 7 different commercial mailers as his examples. He claims the data on these senders will let us evaluate ESPs, but these aren’t ESPs. At best they’re ESP customers, but we don’t know that for sure. He claims that shared IPs means shared reputation, which is true. But he doesn’t claim that these are shared IPs. In fact, I would bet my own reputation on Pizza Hut having dedicated IP addresses.
The author chooses 4 different publicly available reputation services to check the “marketing emails” against. I am assuming he means he checked the sending IP addresses because none of these services let you check emails.
He then claims these 4 measures

give a representation of how an ESP operates.
[…]
This includes whether it follows best principles and sends authenticated emails, unsubscribes Feed Back loop (FBL) complaints etc.

Well, no, not even a little bit.
The 4 measures he included are SenderScore, Sender Base, Trusted Source and MxToolbox’s blacklist checker. The first 4 are proprietary scores generated by commercial companies. Sender Base is a proprietary reputation scheme stream run by Cisco/Ironport. Trusted Source is a proprietary reputation evaluation run by McAfee.
In all cases, the scores are proprietary and are closely guarded secrets and we don’t know much about how they are generated. There are a few things I’m comfortable saying about them.
Scores reflect information provided by receiving mail servers. These scores are sometimes, but not always, applicable to receivers that use a different filtering system. Likewise, good senders can have poor scores and poor senders can have good scores.
In many of the scores volume plays an important role. Volume changes, whether up or down can cause unexpected and transient changes in scores.
Publicly available reputation scores don’t actually tell you that much about the policies of an ESP or the deliverability at a certain ESP.  Content is playing a bigger and bigger role in filtering at major ISPs, and good IP reputation scores aren’t sufficient to overcome bad content.
The only thing that actually tells you about delivery rates is: actually looking at your delivery rates.
The other source the author relied on to analyze deliverability is a scan of 100+ blocklists. He points out that some “ESPs” are listed and blocked by those blocklists. He never mentions which ESPs are listed, or which blocklists are listing them. There are a lot of published blocklists that are not very widely used, and many senders, ESP and otherwise, don’t notice or care. The time and energy to get delisted does nothing to improve delivery. So they just ignore it.
As we’ve demonstrated here recently, even listings on widely used lists are not sufficient to demonstrate poor practices on the part of the sender. Sometimes the blocklists are wrong.
So the author wrote an entire blog post about analyzing deliverability, without actually analyzing deliverability.
And, when he reported the results of his analysis, he left out all the relevant information that would allow us to repeat his analysis. We can’t look at the IP addresses (or the ESPs) that he used as samples because he reported neither bit of information. We can’t look at the blocklists that these IP addresses (or the ESPs) are listed on because he didn’t report the blocklists.
His delivery analysis is full of problems. Tomorrow we’ll look at the errant conclusions he drew from his “analysis.”

4 comments

  1. Al Iverson says

    You are so spot on here. Joshua Geake’s post was so utterly ignorant, so utterly wrong, so utterly stupid, that I have half a mind to bill him for the five minutes I wasted reading that piece of garbage. I did get a good laugh from him trying to dress down Andrew Barrett based on Andrew’s blog supposedly being hosted by WordPress, though. Because Joshua Geake is an expert, and he knows that no legit blog would ever be hosted by WordPress. Snort.

  2. Mark Brownlow says

    Laura: you say, ” Content is playing a bigger and bigger role in filtering at major ISPs”. Could you expand on that? Do you mean traditional filtering based on the actual text and code in the mails or rather how people engage with the content?

    1. laura says

      I’ve got a little more to say about picking an ESP, but that’s a useful thing to write about.
      The short version is that IP reputation is more about getting in the door and content reputation (mostly domains and URLs) is more about getting into the inbox. Having a decent IP reputation is important as a minimum, but once you get past the SMTP session, IP reputation doesn’t affect inbox placement as much as content reputation.

  3. Mark Brownlow says

    Super, thanks – I’m learning all the time from your posts!

Comment:

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.