Open Rate

About the Apple thing

A lot of folks are talking about Apple’s recent announcement about building privacy protection into email. I have somewhat stayed out of the conversation and I’m not sure what I really think about it. This is a change to how a lot of folks use email and no one really likes change.

Stop obsessing about open rates

In 2020:

250OK says open rates were much lower than ESPs report.
The Only Influencers list hosts a discussion about the value and use of open rates.
A potential client contacts me asking if I can get their open rates to a certain percentage.
A client shows me evidence of 100% inboxing but wants to improve their open rate.
An industry group runs sessions at multiple meetings discussing how inaccurate open rates are.

The industry needs to stop obsessing over open rates.

Forget about engagement, think inboxing

While answering a question about how to improve IP reputation at Gmail I realized that I no longer treat Gmail opens as anything about how a user is interacting with email. There are so many cases and ways that a pixel load can be triggered, without the user actually caring about the mail that it’s not a measure of the user at all.

Oh, Microsoft

Things have been a little unsettled at Microsoft webmail properties over the last few months. A number of ESPs reported significantly increased deferrals from Microsoft properties starting sometime late in November. Others saw reduced open rates across their customer base starting in late October. More recently, people are noticing higher complaint rates as well as an increase in mail being dropped on the floor. Additionally, Return Path announced certification changes at the end of November lowering the Microsoft overall complaint rate to 0.2%, half of what is was previously.

Overall, sending mail to Microsoft is a challenge lately. This is all correlated with visible changes which may seem unrelated to deliverability, but actually are. What are the changes we know about?

Implied permission

Codified into law in CASL, implied permission describes the situation where a company can legally mail someone. The law includes caveats and restrictions about when this is a legitimate assumption on the part of the company. It is, in fact, a kludge. There isn’t such a thing as implied permission. Someone either gives you permission to send them email or they don’t.
We use the term implied permission to describe a situation where the recipient didn’t actually ask for the mail, but isn’t that bothered about receiving it. The mail is there. If it has a particularly good deal the recipient might buy something. The flip side of not being bothered about receiving mail, is not being bothered about not receiving mail. If it’s not there, eh, no biggie.

Implied permission isn’t real permission, no matter what the law says.
Now, many deliverability folks, including myself, understand that there are recipients who don’t mind getting mail from vendors. We know this is a valid and effective way of marketing. Implied permission is a thing and doesn’t always hurt delivery.
However, that does not mean that implied permission is identical to explicit permission. It’s one of the things I think CASL gets very right. Implied permission has a shelf life and expires. Explicit permission doesn’t have a shelf life.
Implied permission is real, but not a guarantee that the recipient really wants a particular email from a sender, even if they want other emails from that sender.

Ask Laura: Does changing ESPs hurt deliverability?

Monitoring Your Mail Stream

One of the most important things for any mail sender to do is monitor their mail stream. There are a number of things that every mailer should pay attention to. Some are things to monitor during delivery, some are things to monitor after delivery. All of these things tell senders important information about how their mail is being received by their recipients and the ISPs.

What is an open?

I was having a discussion today with a few industry colleagues about engagement and open rates. It was a good discussion and inspired a couple blog posts. Engagement totally matters, Engagement affects deliverability, and ISPs should be the last of your concerns.
I think they’ve covered the engagement issue pretty well, but what I wanted to talk about was metrics, specifically opens. Open is a fairly simple word, and it’s used in email all the time. Recipients open email. Mailbox providers measure that open. Senders measure that open.
It’s critical to remember, though, that open rates as measured by free mailbox provider and open rates tracked by a sender are not really the same thing. They’re measured in very different ways, and there is not a 1:1 mapping between the two measurements.

Gmail opens… anyone seeing changes?

I’m wondering if people are seeing any changes in open rates now that gmail defaulted to on.
Anyone got any quick feedback?

Meaningless metrics

I’ve been having some conversations with fellow delivery folks about metrics and delivery and bad practices. Sometimes, a sender will have what appear to be good metrics, but really aren’t getting them through any good practices. They’re managing to avoid the clear indicators of bad practices (complaints, SBL listings, blocks, etc), but only because the metrics aren’t good.
This made me laugh when a friend posted a link to a Business Insider article about how many website metrics aren’t useful indicators of the business value of a website. Then I found the original blog post referenced in the article: Bullshit Metrics. It’s a great post, you should go read it.
I’d say the concluding paragraph has as much relevance to email marketing as to web marketing.

Subject lines

There has been a lot of discussion in various places recently about subject line length and how it affects email marketing. There have been multiple studies done on how the subject line affects opens and clicks. (Mailchimp, Alchemy Worx, Mailer Mailer, Adestra). The discussion has even spilled over into Ken Magill’s newsletter today.
I’ve had a couple people ask me my opinion on subject line over the years. My general response is that subject line length is not directly measured by spamfilters and so don’t fret about the length. It is true that consistently crafting poor subject lines can indirectly cause delivery problems. Send mail few people open and that will hurt your reputation over time.
I think Ken really said it best, though.

Metrics, metrics, metrics

I’ve been sitting on this one for about a week, after the folks over at IBM/Pivotal Veracity called me to tell me about this. But now their post is out, so I can share.
There are ISPs providing real metrics to senders: QQ and Mail.ru. Check out Laura Villevieille’s blog post for the full details.

Reporting email disposition

Most regular readers know I think open and click through rates are actually proxy measurements. That is they measure things that correlate with reading and interacting with an email and can be used to estimate how much an email is wanted by the recipients.
The holy grail is, of course, having ISPs report back exact metrics on what a user did with an email. Did the user read it? Did it stay open on their screen a long time? Did the user just mark it read or throw it away? What happened to the message. Marketers would love this information.
It’s unlikely the ISPs will ever provide this information to marketers. Take away all the technical challenges, and there are some significant ones there are still social challenges to making this data available. Current user contracts protect the privacy of the user, local laws prohibit sharing this data. And, there is the vocal group of privacy advocates that will protest and raise a big stink.
I’m not sure why email is gets the special treatment of expecting the channel owners to provide detailed disposition data. In no other direct marketing venue is that information collected or provided. TV stations can’t tell advertisers whether or not someone watched a commercial, fast forwarded through it or got up to grab a beer from the fridge. The post office can’t tell direct mail marketers whether or not a recipient read the mail or just dumped it in the big recycling bin the post office provides for unwanted messages. Billboard owners can’t tell advertisers how many people read the billboard.
Since we can’t get exact read rates from ISPs, what do we do? We look at proxy numbers.
Read rate directly measures who opened the message. Open rate is a proxy. It’s who displayed images in the message.
Read rate can be measured only by people who have access to the user’s inbox. The ISPs can measure read rate because they have full access to the mailbox, but this requires the user to access the mailbox through webmail or IMAP. Some third party mailbox addons can measure it, but this requires the cooperation of the mailbox owner. If the mailbox owner doesn’t install the reporting tool, then the 3rd party doesn’t have access to the data. Only groups with access to the end users mailbox can measure this rate.
Open rate can be measured by people who have access to the server images are hosted. Senders and ESPs and 3rd parties can measure it if they provide unique image IDs or tracking pixels in their emails. Open tracking does require the cooperation of the recipient – they have to have images on. No images on, no open tracking. Ironically, ISPs cannot measure open rate, because they have no access to the image hosting servers.
Click rate can be measured by people who have access to the server that hosts the website. The same people who can measure opens can measure clicks. Some ISPs can measure clicks, Hotmail used to pass every URL through a proxy they hosted and they could count clicks this way. AOL controls the client so they could measure number of clicks on a link. I’ve heard trustworthy folks claim that ISPs are measuring clicks and that they’re not measuring clicks (any of the Barry’s want to comment?).
Without controlling the inbox, though, senders have to rely on proxy measurements to judge the effectiveness of any particular campaign. But at least email marketers have proxies to use for measurement.

Forcing those opens

Most email marketers want to see their open rates go up. This particular marketer has come up with a new way to force recipients to load their mail.

Inbox rates and conversion rates

Jeanne Jennings published an interesting bit of research on open rates and inbox rates at ClickZ recently. Essentially she looked at two different industry studies and compared their results.
The first study was the Return Path Global Delivery Survey and the second was the Epsilon North American Trend Results. What Jeanne found is that while Return Path shows a decrease in inbox placement, Epsilon is seeing an increase in average open rate.

There are any number of reasons this could be happening, including simply different ways the numbers are calculated. I am not sure it’s just a numbers issue, though. Many of Epsilon’s clients are very big companies with a very experienced marketing team. The Return Path data is across their whole user base, which is a much broader range of marketers at different levels of sophistication.
I expect that the Epsilon data is a subset of the Return Path data, and a subset at the high end at that. It does hint, though, that when the inbox is less cluttered, recipients are more likely to open the commercial mail that does get in there.

Freemail opens

Justin Coffey commented on my check your assumptions post pointing out his data on opens related to ISPs. He says:

Standard Email Metrics

The EEC has been working on standardizing metrics used in email marketing. They have published a set of definitions for different terms many email marketers use. They published their Support the Adoption of Email Metrics (S.A.M.E) guide in June.
Under the new EEC definitions an open is measured when either a tracking pixel is displayed or a user clicks on any link in the email, including the unsubscribe link. Open rate is defined as the number of opens (either unique or total) divided by the number of accepted emails. Accepted emails equals the number of emails sent minus the number of emails rejected by the ISP for any reason.
The authors do caution, however, that even their measurements may under count the number of email subscribers that actually open or read an email. Some readers don’t load images or click on links but happily read and digest the content being sent. Others may not click on a link but actually visit a website or brick and mortar store to purchase something based on the email.
Overall, I think the definitions created by the S.A.M.E. group accurately reflect the things they want to measure within the limits of what is actually measurable. Their definitions won’t affect conversations in the short term, but are likely to drive change to standard terminology over the longer term. I do strongly encourage people to grab a copy of their document and see how their definitions compare with your current measurements.

What does open rate tell you

There has been an lot written about open rates in the past, but there are two posts that stand out to me. One was the EEC’s post on renaming open rate to render rate and Mark Brownlow’s excellent post on what open rate does and does not measure. I’ve also weighed in on the subject. The issue is still very confused.
If asked, most people will tell you that open rate is the number of emails that were opened by the recipient. The problem is that this isn’t actually true. Open rate is measured by the number of people that display an image in an email. Traditionally this has been a uniquely tagged 1×1 pixel, until some filters and mail clients stopped displaying 1×1 pixels. More recently, every image in an email is tagged, so opening one image would record as an open.
So open rate doesn’t actually tell a sender how many people opened and read an email. It really only records that an image in a particular email is loaded. It does not record when an email is opened. Some people don’t load images by default. Some people don’t load images at all, even when they open and actively read the text portion of the email.
Clearly, there are some uses for open rates. It can give a useful metric when comparing different forms of the same email (A/B testing) and when looking at user engagement over time. However, we have also recently seen that open rate is not predictive for click through rate.

When an open is not a sign of interest

A lot of people, including myself, are using opens as one of the measures of engagement. This, as a general rule, is not a bad measure. However, there are people who will open email not because they’re interested in it, but because they know it is spam.
Take, for instance, the email address I acquired in 1993. Yes, I still have this address. I stopped using it to sign up for lists in 1999 and stopped using it for most of the rest of my mail around 2001. This address, though, is on any number of spam mailing lists. The spam that gets through is usually sent by hard-core spammers. The ISP that hosts that mailbox uses Communigate Pro to filter mail, so much of the casual spam is filtered.
Generally, if I open an email (and load images or click through) on that account it is only in order to track down a spammer. For instance, I’m getting a lot of spam there from affiliates offering me the opportunity to purchase printing services for a very low price. I have actually been opening the mail, and clicking through. But I’m not clicking through because I’m interested in purchasing. I’m clicking through to see if my reports to abuse@ printer are resulting in any action against the spammers. (They’re not).
The thing is, though, I know that by clicking through on ads, I’ve now been promoted by the spammer to the “clicks on emails! it’s a live address!” list. Which only means I’m going to get more spam from them. Lucky me.
Using clicks and opens as a measure of engagement isn’t necessarily bad. But when using them you have to understand the limitations of the measurement and that what you may think it’s telling you isn’t actually what it’s telling you.

Standardizing email metrics

Slogging towards e-mail metrics standardization a report by Direct Mag on the efforts of the Email Experience Council to standardize definitions related to email marketing.

Best time to send email: analysis and discussion

Mark Brownlow (who I don’t think is here in Ams, much to my disappointment) wrote a long assessment of how to determine what is the best time to send email. He walks through the questions and the data that a sender should evaluate when making the decision when to best send email.
I have previously posted about my views on the best time to send email. There is no one best time to send email. In fact, my experience leads me to believe if someone said the best time to send email is at 4pm on Tuesday afternoon then 4pm on Tuesday afternoon would rapidly become the absolute worst time to send email.
It should come as no surprise, then, that I really like Mark’s #4 recommendation.

Best time to send marketing email

Pages and pages have been written about the best time to send email. Marketers spend significant amounts of energy discussing and researching the best time of the day and the best day of the week to send email. I have long thought that these discussions do not put enough attention on individual end users and how the recipients interact with email.
Researchers recently developed a model for email user behaviour that splits email users into two classes “e-mailaholics” that send, and presumably read, email all the time and “day labourers” that send, and presumably read, email during standard business hours. There is very little transition between groups, 75% of users stayed in the same usage group over the 2 years of the study.
What does this mean for senders? Senders need to know know how their recipients use email and which user group recipients are. By analyzing clicks and opens, senders can classify recipients and use that data to send mail that is more relevant and better targeted.
h/t arXiv blog at Technology Review

Measuring open rate

In this part of my series on Campaign Stats and Measurements I will be examining open rates, how they are used, where they fail and how the can be effectively used.
There has been an lot written about open rates recently, but there are two posts that stand out to me. One was the EEC’s post on renaming open rate to render rate and Mark Brownlow’s excellent post on what open rate does and does not measure. I’ve also weighed in on the subject.
Overall, I find open rates to be a very frustrating metric. Some senders, particularly those relatively new to email marketing, are so sure they know what open rate is and what it means, that they don’t take any time to actually understand the number. While the name “open rate” seems self explanatory, it’s actually not. Open rate is actually not a measure of how many recipients open an email. However, there are times where open rate is a useful metric for measuring a marketing program over time.
What is an open?
If asked, most people will tell you that open rate is the number of emails that were opened by the recipients. The problem is that this isn’t actually true. An open is counted when a tagged image in an email is rendered by the recipient’s email client. Not all mail clients render images by default, but the emails are still available for the recipient to read. If a user clicks on a link in an email that has not had an image rendered, some ESPs count that as an open as well as a click. In other cases, visiting a link in an email with no image rendered is just a click, no open is recorded.
What is the open rate?
Open rate is generally the percentage of email opens divided by some number representing the number of emails sent. Many senders use the number of emails sent minus the number of bounced emails, others use just the number of emails sent without factoring in the number of emails bounced.
Open rate is a secondary metric. While it does not measure the success, or failure, of a campaign directly, it can be used as a indicator for campaigns. Many people use open rate as a metric because it’s easy to measure. Direct metrics, such as clicks or average purchase or total purchase, may take days or even weeks to collect and analyze. Open rates can be calculated quickly and easily.
What the open rate isn’t
Open rate is not a measure of how many people opened a mail. It is not a measure of how many people read a mail. It really only records that an image in a particular email is loaded and, sometimes, that a link was clicked on. Open rates can be wildly different depending on how the sender measures opens and how the sender measures sends.
What senders use open rates for
To compare their open rates with industry averages
As I talked about above, this use of open rates is problematic at best. You cannot compare numbers, even when they have the same name, if the numbers were arrived at using different calculations. Open rate is not open rate and unless you know the underlying algorithm used you cannot compare two open rates. This is a poor use of open rate.
As a metric for advertising rates
Since a sender can manipulate the open rate by using different calculation methods, this is a good metric for the advertiser to use. It is not so great for the purchaser though, who is at the mercy of the sender’s metrics. There are contractual ways a purchaser can protect herself from an unscrupulous marketer, but only if she understands how open rate can be manipulated and takes steps to define what open rate is in use.
To judge the success of campaigns over time
A single open data point doesn’t mean very much, however, using consistently measured open rates a sender can measure trends. Open trends over time are one area that open rates can help senders judge the success, or failure, of a marketing campaign.
As one metric in A/B testing
Comparing open rates in A/B testing gives some indication of which campaigns recipients may be more interested in. As with trends over time, the lone measurement isn’t useful, but as a comparative metric, it may provide senders with insight into a particular mailing.
To judge the engagement of recipients
Over the long term, recipients who do not interact with a mailing become dead weight on the list. Too many non responders can hurt a sender’s reputation at an ISP. List hygiene, in the form of removing people who never open or click on an email, is an important part of reputation management.
As metrics for email campaigns go, open rate is limited in what it measures about an email campaign. However, as a quick way to measure trending or do head to head comparisons it is a useful metric.

Open rates climbing, click rates dropping

Ken Magill reported on a study published by Epsilon (pdf link) on Tuesday. This report shows open rates are climbing but click-through rates are falling.

Open rates

Right now, there is no way to compare open rates as everyone calculates them differently. Mark Brownlow covers this today.

Asking the right question

My job as a consultant does involve answering questions and solving problems. Often the most important, and most overlooked, thing that I do is change the question that clients are asking. It is not that this changes the problem or even, really, changes the solution. It does change how people think of the problem, and changing how they think of the problem drives better solutions.
This can be applied to the current Email Experience Council (EEC) discussion on metrics and defining what a render rate was. Loren has a post up today detailing a number of common email situations and explaining in which cases an email is counted as open and in which cases an email is counted as unopened.
Right now an open in email terms is actually quite simple: a tagged image on a remote webserver was loaded. That’s all an open is. It used to be that no one was blocking images by default, so this was actually quite an accurate way to measure how many people were opening and presumably reading an email (at least for people using mail clients that display HTML and images).
But, as spammers started including more and more explicit images in email, recipients started asking for images to be blocked. In response to recipient requests, ISPs started blocking images by default. No longer was open rate a measure of which recipients opened and read an email, it became a measure of something completely different.
The EEC has recognized this is a problem and have decided that standardization would be a solution. As the first step to standardization they have identified two problems: open rate isn’t calculated in any standard way and the resulting ratio doesn’t describe what most people think it describes. Their recent publication The Email Render Rate defines standard calculations for render rates. This way render rates as reported by different ESPs can be directly compared. Changing the name from open rate to render rate changes what most people expect that the term means. No longer is this a measure of how many recipients opened the mail, but rather it is a measure of how many email clients rendered the images in the mail.
Maybe a better solution could be arrived at by changing the question? Instead of “how can we standardize render rate?” perhaps they should ask the question: “What do people think they’re measuring when they talk about open rates?”
Once the “what?” question is answered, perhaps a good solution to the “how?” question will become more obvious.

Open Rate? Render Rate?

The EEC is pushing the term render rate to replace the term open rate. In addition to changing the name the EEC is attempting to standardize how the render rate is calculated. Loren McDonald, co-chair of the EEC Measurement Accuracy Roundtable posted his views on the discussion today. He presents 3 reasons why we should care about using render rate.

Old lists have bad delivery

This is something we all know is true, and something that everyone believes. But, Mailchimp has actually published numbers demonstrating just how bad old lists are.

The overlooked secret of marketing

Seth Godin posted recently about the overlooked secret of marketing: time

Spam or not spam

I have been a bit behind on my blog reading recently, and am slowly going through my RSS feed catching up with what everyone has had to say about spam in the last few weeks.
One of the articles that caught my attention was a post from VerticalResponse discussing the response to a marketing campaign from one of their customers. It seems to me the point of the post is to defend the VerticalResponse mail to the customer. The mail VerticalResponse sent was not spam. Why this is true is not made clear, other than the mail was not pills spam, phishing or porn.
Contrasting with that article is a post a friend pointed out to me today. This article goes to the other extreme, and seems to say that any one-to-many email is spam and should not be sent. While trying to find his point, the author does take the step of exempting any opt-in marketing from his definition. The confusing bit is that the statistics he is using are compiled by MailerMailer, who have a very clear anti-spam policy and allow only permission based marketing.
What both posts seem to be missing is that, these days, spam is in the eye of the receiver, not the sender. There are customers who groan every time they receive mail from their vendor. Eventually, they may lash out at a sender and complain about the email. At that point, a sender is now dealing with an angry person, and arguing the mail is no spam is not going to diffuse the situation. On the flip side, there are people who are very happy to receive mail, even advertising and marketing mail, from vendors. Even if they do not “open” the mail (read: load images in the email), they may be opening, reading and acting on the offers in the email.
Email marketing is a valuable tool, when it is done correctly and focuses on the receiver’s needs and wants. It is when marketers ignore the individuals they are mailing that they are more likely to see complaints or problems.