Opens

No, Gmail did not just break all open tracking

I was avoiding commenting on the email open tracking bad take that seems to be going viral round the more gullible corners of LinkedIn.

I always want to say “Emails, and Opens, and Clicks… Oh My!” when I’m talking about them.

Apple MPP

You’ve probably heard about Apple Mail Privacy Protection. Email marketing chat has been all a-twitter about it since it was announced in June.

Stop obsessing about open rates

In 2020:

250OK says open rates were much lower than ESPs report.
The Only Influencers list hosts a discussion about the value and use of open rates.
A potential client contacts me asking if I can get their open rates to a certain percentage.
A client shows me evidence of 100% inboxing but wants to improve their open rate.
An industry group runs sessions at multiple meetings discussing how inaccurate open rates are.

The industry needs to stop obsessing over open rates.

One of the things I’ve been spending a lot of time thinking about lately is how we measure deliverability. Standard deliverability measurements include: opens, bounces, complaints, and clicks. There are also other tools like probe accounts, panel data, and public blocklists. Taken together these measurements and metrics give us an overall view of how our mail is doing.

A must read on engagement

I really can’t add anything to what Chad wrote in Opens, Clicks, And Blocks In The Third Age Of Email Deliverability

FAQ about opens and Gmail caching

I had hoped to blog about something else today, but this still seems to be a big concern for a number of people. There are a lot of questions running around, some of which we don’t have answers to, others of which we have answers based on some evidence.
It’s important to remember that we’ve seen Gmail roll things out and then roll things back and do phased transitions during deployment. What various people are reporting about images and caching and headers are accurate at the time they are tested. But they may not be accurate tomorrow or in a week or in a month.
I’ve also discovered through this process that a lot of different providers use significantly different image tracking in order to record image loads. Some of these techniques seem to be more resistant to Google’s new image loading process than others.
Why is this all so important?
Image tracking has become a fundamental part of email marketing. It’s something that can be measured, and so a lot of marketers evaluate the effectiveness of an email send based partially on open rate.
How does open tracking work?
For open tracking, ESPs inject a uniquely tagged image into the email. When the recipient opens an email and has images on, the email client calls to the sender server and asks the sender server for all the images in the email. When the tagged image is returned to the recipient, the server records an “open.”
How does caching break open tracking?
Caching means that only the first load of an image is provided by the sender’s server. Subsequent loads of an image are served by the caching proxy. Caching proxies are nothing new; they just haven’t affected email enough in the past for us to have to talk about it.
Why are some people reporting zero problems?
The first load of a unique image always happens. Some folks don’t measure repeat opens, so they’re not even noticing any changes in their reporting thus are saying they’re seeing no problems.
What else is image tracking used for?
Image tracking can also be used for device detection by reading the “user-agent” string that each device returns. Gmail is currently rewriting the “user-agent” string thus breaking all device detection. The string is unique enough that it would be possible to tag those opens as “opened through gmail web interface.” Gmail may decide to pass through the user agent in the future, the HTTP standard does allow for that.
Image tracking can also be used for geolocation. Some senders use the location of an IP address to return images relevant to a user’s location. The accuracy of geolocation is totally dependent on the accuracy of the IP to location database used; it is a best guess of the user’s location. Gmail is currently not passing through the user’s IP address when requesting the original image. I don’t expect them to start, given they also don’t reveal user IPs when Gmail web users send mail. This falls in the same category of privacy protection.
Is there a workaround?
I have heard of a few people claiming they have a fix. The problem is all of the fixes I have seen involve doing things that violate the HTTP RFCs. For instance, the “fix” or “workaround” discussed at E-Mail Marketing Tipps is to not send back an image at all. This is working now to track repeat opens, but Gmail may adapt and block this as well. It’s also possible that Gmail may decide people trying to “work around” Gmail’s cache should be blocked outright for violating the HTTP spec.
Where can I find more information?
Other blog posts on the issue, including research on what people have seen.

Gmail deploys image proxy servers

This afternoon Justin Foster of LiveClicker posted to the OnlyInfluencers list asking about Gmail rewriting links.

Tor cleans up their lists

Recently I got an email from Tor. Apparently they’re watching their opens and clicks and they noticed I hadn’t loaded any images recently.

Meaningless metrics

I’ve been having some conversations with fellow delivery folks about metrics and delivery and bad practices. Sometimes, a sender will have what appear to be good metrics, but really aren’t getting them through any good practices. They’re managing to avoid the clear indicators of bad practices (complaints, SBL listings, blocks, etc), but only because the metrics aren’t good.
This made me laugh when a friend posted a link to a Business Insider article about how many website metrics aren’t useful indicators of the business value of a website. Then I found the original blog post referenced in the article: Bullshit Metrics. It’s a great post, you should go read it.
I’d say the concluding paragraph has as much relevance to email marketing as to web marketing.

Subject lines

There has been a lot of discussion in various places recently about subject line length and how it affects email marketing. There have been multiple studies done on how the subject line affects opens and clicks. (Mailchimp, Alchemy Worx, Mailer Mailer, Adestra). The discussion has even spilled over into Ken Magill’s newsletter today.
I’ve had a couple people ask me my opinion on subject line over the years. My general response is that subject line length is not directly measured by spamfilters and so don’t fret about the length. It is true that consistently crafting poor subject lines can indirectly cause delivery problems. Send mail few people open and that will hurt your reputation over time.
I think Ken really said it best, though.

Reporting email disposition

Most regular readers know I think open and click through rates are actually proxy measurements. That is they measure things that correlate with reading and interacting with an email and can be used to estimate how much an email is wanted by the recipients.
The holy grail is, of course, having ISPs report back exact metrics on what a user did with an email. Did the user read it? Did it stay open on their screen a long time? Did the user just mark it read or throw it away? What happened to the message. Marketers would love this information.
It’s unlikely the ISPs will ever provide this information to marketers. Take away all the technical challenges, and there are some significant ones there are still social challenges to making this data available. Current user contracts protect the privacy of the user, local laws prohibit sharing this data. And, there is the vocal group of privacy advocates that will protest and raise a big stink.
I’m not sure why email is gets the special treatment of expecting the channel owners to provide detailed disposition data. In no other direct marketing venue is that information collected or provided. TV stations can’t tell advertisers whether or not someone watched a commercial, fast forwarded through it or got up to grab a beer from the fridge. The post office can’t tell direct mail marketers whether or not a recipient read the mail or just dumped it in the big recycling bin the post office provides for unwanted messages. Billboard owners can’t tell advertisers how many people read the billboard.
Since we can’t get exact read rates from ISPs, what do we do? We look at proxy numbers.
Read rate directly measures who opened the message. Open rate is a proxy. It’s who displayed images in the message.
Read rate can be measured only by people who have access to the user’s inbox. The ISPs can measure read rate because they have full access to the mailbox, but this requires the user to access the mailbox through webmail or IMAP. Some third party mailbox addons can measure it, but this requires the cooperation of the mailbox owner. If the mailbox owner doesn’t install the reporting tool, then the 3rd party doesn’t have access to the data. Only groups with access to the end users mailbox can measure this rate.
Open rate can be measured by people who have access to the server images are hosted. Senders and ESPs and 3rd parties can measure it if they provide unique image IDs or tracking pixels in their emails. Open tracking does require the cooperation of the recipient – they have to have images on. No images on, no open tracking. Ironically, ISPs cannot measure open rate, because they have no access to the image hosting servers.
Click rate can be measured by people who have access to the server that hosts the website. The same people who can measure opens can measure clicks. Some ISPs can measure clicks, Hotmail used to pass every URL through a proxy they hosted and they could count clicks this way. AOL controls the client so they could measure number of clicks on a link. I’ve heard trustworthy folks claim that ISPs are measuring clicks and that they’re not measuring clicks (any of the Barry’s want to comment?).
Without controlling the inbox, though, senders have to rely on proxy measurements to judge the effectiveness of any particular campaign. But at least email marketers have proxies to use for measurement.

Forcing those opens

Most email marketers want to see their open rates go up. This particular marketer has come up with a new way to force recipients to load their mail.

What email metrics do you use?

Vertical Response talks about email metrics that are useful on a dashboard.
Metrics are an ongoing challenge for all marketers. The underlying need for metrics is to evaluate how effective a particular marketing program is. Picking metrics involves understanding what the goal is for a particular program. If your goal is brand recognition then perhaps sales and click-through figures aren’t a good metric. If your goal is sales then opens is not as good a metric as average order value or revenue per email.
Measuring email success is important. But how you choose to measure it is a critical decision. Too many marketers just use canned metrics and don’t think about what they really want to know.

Don't take my subscribers away!

Tom Sather has a good summary of the problems with inactive email addresses and why data hygiene is critical to maintain high deliverability. These recommendations are some of the most difficult to convince people to implement.
Some of my clients even show me numbers that show that a recipient that hadn’t opened or read and email in 18 months, suddenly made a multi-hundred dollar purchase. Another client had clear numbers that showed even recipients that didn’t open for an entire year were responsible for 10% of revenue.
They tell me I can’t expect them to let their customers go. These are significant amounts of money and they won’t let any potential revenue go without a fight.
I understand this, I really do. The bottom line numbers do make it tough to argue that inactive subscribers should be removed. Particularly when the best we can offer is vague statements about how delivery may be affected by sending mail to unengaged users.
I don’t think many senders realize that when they talk about unengaged users they are actually talking about two distinct groups of recipients.
The first group is that group of users that actively receive email, but who aren’t opening or reading emails from particular senders. This could be because of their personal filters, or because the mail is going to the bulk folder or even simply because they don’t load images by default. This is the pool that most senders think of when they’re arguing against removing unengaged users.
The second group is that group of users that never logs in ever. They have abandoned the email address and never check it. I wrote a series of posts on Zombie Emails (Part 1, 2, 3) last September, finishing with suggestions on how to fight zombie email addresses.
Unlike senders ISPs can trivially separate the abandoned accounts from the recipients who just don’t load images. Sending to a significant percentage of zombie accounts makes you look like a spammer. Not just because spammers send mail to really old address lists, but a number of spammers pad their lists with zombie accounts in order to hide their complaint rates. The ISPs caught onto this trick pretty quickly and also discovered this was a good metric to use as part of their filtering.
I know it’s difficult to face the end of any relationship. But an email subscription isn’t forever and if you try to make it forever then you may face delivery problems with your new subscribers.