Measurements

One of the things I’ve been spending a lot of time thinking about lately is how we measure deliverability. Standard deliverability measurements include: opens, bounces, complaints, and clicks. There are also other tools like probe accounts, panel data, and public blocklists. Taken together these measurements and metrics give us an overall view of how our mail is doing.


A globe with the words innovation, creativity, inspiration and idea written on it.

More and more, though, I see senders meeting all the standard metrics for these measurements, yet still struggling with deliverability. In many ways this isn’t surprising. There are a whole host of tools out there that allow senders to manipulate the underlying metrics without changing their underlying practices. To complicate matters even more, there are tools that manipulate open and click rates by following every link in an email. Finally, we know that some ISPs don’t send 100% of the “this is spam” messages to their FBL. Other metrics, like probe accounts, are inaccurate in an era of personalised delivery based on activity.

All in all, these metrics were built to tell us things about a mail system that no longer exists. Our next challenge is to figure out what metrics to use in the future. How do we monitor the effectiveness of our address collection processes and our deliverability?

One thing I’ve started having customers look at, especially my ESP clients, is how the consumer ISPs are accepting their mail. Are they seeing temp failures and if they are, what specific mailstreams are the temp failures related to? It’s a little early to tell if this is an effective measurement for ESP compliance purposes. It’s definitely helping identify problematic mail streams for my brand clients and allowing us to make adjustments to get to the inbox.

What I do know is that we in the deliverability space need to continue innovating and thinking about how to measure our deliverability. Mail filters are evolving, and we must evolve as well.

Related Posts

Meaningless metrics

I’ve been having some conversations with fellow delivery folks about metrics and delivery and bad practices. Sometimes, a sender will have what appear to be good metrics, but really aren’t getting them through any good practices. They’re managing to avoid the clear indicators of bad practices (complaints, SBL listings, blocks, etc), but only because the metrics aren’t good.
This made me laugh when a friend posted a link to a Business Insider article about how many website metrics aren’t useful indicators of the business value of a website.  Then I found the original blog post referenced in the article: Bullshit Metrics. It’s a great post, you should go read it.
I’d say the concluding paragraph has as much relevance to email marketing as to web marketing.

Read More

Reporting email disposition

Most regular readers know I think open and click through rates are actually proxy measurements. That is they measure things that correlate with reading and interacting with an email and can be used to estimate how much an email is wanted by the recipients.
The holy grail is, of course, having ISPs report back exact metrics on what a user did with an email. Did the user read it? Did it stay open on their screen a long time? Did the user just mark it read or throw it away? What happened to the message. Marketers would love this information.
It’s unlikely the ISPs will ever provide this information to marketers. Take away all the technical challenges, and there are some significant ones there are still social challenges to making this data available. Current user contracts protect the privacy of the user, local laws prohibit sharing this data. And, there is the vocal group of privacy advocates that will protest and raise a big stink.
I’m not sure why email is gets the special treatment of expecting the channel owners to provide detailed disposition data. In no other direct marketing venue is that information collected or provided. TV stations can’t tell advertisers whether or not someone watched a commercial, fast forwarded through it or got up to grab a beer from the fridge. The post office can’t tell direct mail marketers whether or not a recipient read the mail or just dumped it in the big recycling bin the post office provides for unwanted messages. Billboard owners can’t tell advertisers how many people read the billboard.
Since we can’t get exact read rates from ISPs, what do we do? We look at proxy numbers.
Read rate directly measures who opened the message. Open rate is a proxy. It’s who displayed images in the message.
Read rate can be measured only by people who have access to the user’s inbox. The ISPs can measure read rate because they have full access to the mailbox, but this requires the user to access the mailbox through webmail or IMAP. Some third party mailbox addons can measure it, but this requires the cooperation of the mailbox owner. If the mailbox owner doesn’t install the reporting tool, then the 3rd party doesn’t have access to the data. Only groups with access to the end users mailbox can measure this rate.
Open rate can be measured by people who have access to the server images are hosted. Senders and ESPs and 3rd parties can measure it if they provide unique image IDs or tracking pixels in their emails. Open tracking does require the cooperation of the recipient – they have to have images on. No images on, no open tracking. Ironically, ISPs cannot measure open rate, because they have no access to the image hosting servers.
Click rate can be measured by people who have access to the server that hosts the website. The same people who can measure opens can measure clicks. Some ISPs can measure clicks, Hotmail used to pass every URL through a proxy they hosted and they could count clicks this way. AOL controls the client so they could measure number of clicks on a link. I’ve heard trustworthy folks claim that ISPs are measuring clicks and that they’re not measuring clicks (any of the Barry’s want to comment?).
Without controlling the inbox, though, senders have to rely on proxy measurements to judge the effectiveness of any particular campaign. But at least email marketers have proxies to use for measurement.

Read More

Subject lines

There has been a lot of discussion in various places recently about subject line length and how it affects email marketing. There have been multiple studies done on how the subject line affects opens and clicks. (Mailchimp, Alchemy Worx, Mailer Mailer, Adestra). The discussion has even spilled over into Ken Magill’s newsletter today.
I’ve had a couple people ask me my opinion on subject line over the years. My general response is that subject line length is not directly measured by spamfilters and so don’t fret about the length. It is true that consistently crafting poor subject lines can indirectly cause delivery problems. Send mail few people open and that will hurt your reputation over time.
I think Ken really said it best, though.

Read More