Justin Coffey commented on my check your assumptions post pointing out his data on opens related to ISPs. He says:
I can say that users at webmail are easily as likely to click on a message that they have opened than users at pay-for ISPs.
Who else collects data on opens per ISP? And Monkeys, I’m looking at you, I know you have this data.
Not every delivery failure is due to poor reputation or spam. Sometimes ISPs just have problems on their mailservers and so mail doesn’t get through. It’s often hard for delivery experts (and their bosses and their customers and their clients) to watch email delays or rejections without being able to do anything about it.
Sometimes, though, there is nothing to do. The rejections are because something broke at the ISP and they have to sort through it. Just this week there’s been a lot of twitter traffic about problems at a major cable company. They are rate limiting senders with very good reputations. They have admitted there is a problem, but they don’t have a fix or an ETA. From what I’ve heard it they’re working with their hardware vendor to fix the problem.
Hardware breaks and backhoes eat fiber. Yes, ISPs should (and all of the large ones do) have backups and redundancies. But those backups and redundancies can’t always handle the firehose worth of mail coming to the ISPs. As a result, the ISPs start rejecting some percentage of mail from everyone. Yahoo even has a specific error message to distinguish between “we’re blocking just you” from “we’re shedding load and temp failing everyone.”
The EEC has been working on standardizing metrics used in email marketing. They have published a set of definitions for different terms many email marketers use. They published their Support the Adoption of Email Metrics (S.A.M.E) guide in June.
Under the new EEC definitions an open is measured when either a tracking pixel is displayed or a user clicks on any link in the email, including the unsubscribe link. Open rate is defined as the number of opens (either unique or total) divided by the number of accepted emails. Accepted emails equals the number of emails sent minus the number of emails rejected by the ISP for any reason.
The authors do caution, however, that even their measurements may under count the number of email subscribers that actually open or read an email. Some readers don’t load images or click on links but happily read and digest the content being sent. Others may not click on a link but actually visit a website or brick and mortar store to purchase something based on the email.
Overall, I think the definitions created by the S.A.M.E. group accurately reflect the things they want to measure within the limits of what is actually measurable. Their definitions won’t affect conversations in the short term, but are likely to drive change to standard terminology over the longer term. I do strongly encourage people to grab a copy of their document and see how their definitions compare with your current measurements.
There has been an lot written about open rates in the past, but there are two posts that stand out to me. One was the EEC’s post on renaming open rate to render rate and Mark Brownlow’s excellent post on what open rate does and does not measure. I’ve also weighed in on the subject. The issue is still very confused.
If asked, most people will tell you that open rate is the number of emails that were opened by the recipient. The problem is that this isn’t actually true. Open rate is measured by the number of people that display an image in an email. Traditionally this has been a uniquely tagged 1×1 pixel, until some filters and mail clients stopped displaying 1×1 pixels. More recently, every image in an email is tagged, so opening one image would record as an open.
So open rate doesn’t actually tell a sender how many people opened and read an email. It really only records that an image in a particular email is loaded. It does not record when an email is opened. Some people don’t load images by default. Some people don’t load images at all, even when they open and actively read the text portion of the email.
Clearly, there are some uses for open rates. It can give a useful metric when comparing different forms of the same email (A/B testing) and when looking at user engagement over time. However, we have also recently seen that open rate is not predictive for click through rate.