Deliveries and Opens and Clicks

I always want to say “Emails, and Opens, and Clicks… Oh My!” when I’m talking about them.

We really want to understand how a mailbox provider perceives the streams of email we send them. Some of the things the provider looks at are technical and objective – is the email coming from who it claims to be coming from? Is there an attachment that contains a virus? We have pretty good insight into those things, as we can look at the mail we’re sending and have much the same information as the receiving mailbox provider has. Did we attach a virus? Is our authentication correct¹?

But a mailbox provider also cares about how their users perceive the email. A consumer ISP wants their users to be happy using their service, including seeing mail they want to receive, and not seeing mail they didn’t want. That reduces user churn, which makes them money, either directly or via putting eyeballs on ads. An enterprise mailbox provider wants the corporation paying for the service to be happy, and amongst other things that means that they want their users to be able to do their job efficiently, to have no problems receiving work-related email and not to be distracted by other email.

Engagement

Mailbox providers have varying amounts of insight into what their users are doing with an email. Some have users reading mail via IMAP, and they can tell when a user has opened an email in their mail client (they can see the IMAP \Seen flag is set), deleted it (the \Deleted flag) replied to it (the \Answered flag) and when they’ve moved it from one folder to another, including from or to the spam folder. Others have users reading mail via an instrumented webmail client, or using a tightly integrated mobile client. Those providers can see all those things, but also can see whether the user clicks on anything, scrolls through the whole message, how long it’s open on their screen, where the pointer hovered, and any other measurement they want to implement, limited more by privacy concerns than any technical limitations.

Mailbox providers use all that information to see how much a user interacted with an email and to estimate how interested they were in it. Mailbox providers call that engagement, and use it to estimate how much a user wants to see mail that’s from the same mail stream, or that has similar content. They’ll also use engagement measures across users – if this user engaged with mail in this stream, then it’s likely that this other user might like it. If these two users have engaged with similar email in the past, then if one of them likes mail from a given mail stream it’s likely the other will too.

Also engagement?

As a mail sender we don’t have access to any of that subjective engagement data, let alone the deeply nuanced version that webmail providers have. We’d love to have it, we’d love to make decisions based on it.

But we don’t have access to it (unless we’re paying for an expensive service that partners with ISPs to share a subset of it). Instead we’ll look at the data we do have access to, call that engagement and make decisions based on that.

The information we can gather as a sender about the life of a single email is whether it was accepted for delivery by the mailbox provider, whether some external content – typically an image – in the email was fetched, and whether the webpage a link in the email points to was fetched.

Delivery

Delivery happens when our ESP connects to the recipients mailserver, sends the email and gets a nice “250 2.0.0 Ok: queued” message in response. This means that the recipients MX has accepted the email and agreed in principle to deliver it to the recipient, or at least to do it’s best to do so. It’s possible for the mail to be undeliverable after this point, in which case you’ll probably get an asynchronous bounce later, but that’s really rare on the modern internet – once the MX has accepted it you can consider it delivered. If you do get an asynchronous bounce later you’ll just have to go back and mark the mail as not delivered.

If the mail was rejected then there are usually useful details in the rejection message about why it was, but that’s not really engagement related so I’ll save talking about that for another time.

Opens

Typically when we talk, as senders, about an “open” we mean that we’ve added a tiny, invisible 1×1 pixel image at the bottom of the email, that has a unique URL that encodes the recipient, the sender, the mailing and any other metadata about the email or campaign we want to record. We host that image on a specialized webserver that as well as returning the image in response to a request for it will also record that this image, with this metadata was fetched at this date and time.

We call the fact that that image was fetched an “open” of the email it was embedded in.

Does that mean that the user had the email open in their mail client at that date and time? Maybe. Maybe not.

If you did see the image fetched it could mean any of:

• The user opened the email in their mail client, and read it
• The user’s mailbox provider or mail client prefetched the image because it expected the user to see it, and the user later read it
• The user’s mailbox provider or mail client prefetched the image because it expected the user to see it, and the user didn’t read it
• The user’s mailbox provider or mail client fetches all images as a privacy feature, explicitly so you can’t tell whether the user read it or not
• The user’s mailbox provider or mail client fetches all images not previously categorized as spam, at some random time after delivery, explicitly so you can’t tell whether the user read it or not
• A spam / malware filter somewhere along the delivery path fetched the image so it could make decisions based on the content
• A spam / malware filter at the MX fetched the image during the delivery process, then accepts, defers or rejects the mail – so you may see an image load hours before the mail is successfully delivered, or even if it was rejected
• The users mailbox provider shared the URL of the image with a search engine, and the search engine crawled it (yes, this happens)
• The user previously read the email on a mail client that didn’t fetch images but then, possibly hours later, reads it again on their ‘phone that does fetch images – so you may see click activity from a recipient hours or days before the first “open” happens
• The user configured their mail client not to load images by default, but later chose to fetch them – again you may see other activity before the first “open”

If you didn’t see the image loaded that could mean any of:

• The mail was never delivered
• The mail was delivered to the spam folder so the user never saw it
• The user saw the mail in their inbox but decided not to open it
• The user uses a mail client that doesn’t support linked images (rare on the modern desktop, but not that unusual in some ticketing systems)
• The image was stripped by content filters before delivery
• The user configured their mail client not to load images, but still opened and read your mail
• The user configured their mail client not to load images by default, but still opened and read your mail
• The user configured their mail client to load images “privately” using, for example, Apple’s Mail Privacy Protection, but they’re also running a corporate VPN client that sometimes breaks that functionality, so the image failed to load
• Your email was over 102k long, so it was truncated at Gmail, cutting off the tracking pixel you put at the bottom of the email – the user did read the email, but didn’t click the link to load the whole thing
• The user’s mail client loads images lazily, and they didn’t scroll to the bottom of the long email, so the image was never fetched
• The user fetched your mail to their ‘phone, then later read it on a plane or a train or somewhere else without network connectivity, so the image wasn’t loaded
• Something went wrong – network outage, DNS problems, your stunt image server fell over – so the user read your mail but the image wasn’t loaded, or it was loaded but wasn’t recorded

Oh, and also:

• Some ESPs will mark an email as “opened” if either they see image load activity or if they see click activity – so your reported open rates may have to take into account accuracy of click measurement too. That also makes comparing open rates across different ESPs problematic.

Well, that was kind of a lot. So what can an “open” “image load” tell you?

If the image was loaded that tells you that the mail was probably delivered, and not to a spam folder.

If you measure image load rates over time and look for changes that can be a useful piece of data. If open rates suddenly drop at gmail it may mean something technical went wrong (did you send a longer mail than usual that got truncated) or it may mean something about that mail caused gmail to deliver it to the spam folder. If open rates are gradually declining at Yahoo maybe you’re increasingly being delivered to the spam folder.

Clicks

Click tracking is similar to open tracking, in that it embeds metadata about the recipient or the campaign in a URL. Instead of being the URL of an image that points at an image load tracking server it’s the URL of a link in the body of the email that points at a click tracking server. When the user clicks on a link in the email the click tracking server records the fact, then redirects the user to the content they’re expecting.

Click tracking is simpler than image load tracking for several reasons. The big one is that mailbox providers and mail clients aren’t pre-loading and caching the pages a recipient is clicking through to. That means that you won’t see an automated fetch of that link content that will later be displayed to the user without you knowing about it.

When a user clicks on a link in the email, you’re going to see their web browser contact your webserver.

But the user isn’t the only thing that may follow links in email. Some filters will fetch the web pages linked to from an email to look at the content on the linked page to see if it looks suspicious – it’s a very effective way of detecting phishing or malware, for instance.

That’s beginning to be done at scale by some consumer mailbox providers – there’ve been a lot of reports over the past few months that an ESP is suddenly seeing a huge spike in clicks from mail sent to Microsoft properties. It looks like they’re testing something.

The automated link fetches may drown out the actual human clicks by number, so your click rate for recipients at those mailbox providers may be artificially inflated to a ridiculous degree.

Does this make click rate a useless metric? It seems to make naive measurement of “clicks” fairly pointless, but unlike images we’ve got a lot more data to work with.

The automated link followers aren’t trying to hide anything for privacy reasons, they just want to see the content before delivering the mail. They’re not being opaque or deceptive in the way privacy preserving images fetches are. So we can reasonably assume that any clicks more than a few minutes after delivery are probably from actual humans. Some of the ones before that may well be too, but we’re just using it as s rough measure of “this human clicked on links in some fraction of mails we sent” – if we’re a bit off we can still make useful decisions based on it.

Even better, we can follow a users behaviour on our website after the click. We can use any of the off the shelf “is this an authentic user” services to categorize that user; we can follow their behaviour in looking at other pages once they’ve clicked through from the email, tracking it all the way through to a purchase.

“Bots”, “Non-human interactions”, …

You’ll see many folks describe this behaviour as “bot traffic” or “non-human interactions” or similar. And, eh, when you’re generating reports you’ve got to call it something.

But it can imply that the recipients of these emails aren’t real, and didn’t read your email, and maybe should be removed from your list as “inauthentic” or something. Some folks recommend doing that, even.

But the vast majority of the automated image loading and link following is done for emails that may well then be delivered to the inbox of a human who opens and reads them. Those are happy recipients; removing them from your list to make your reporting look less messy isn’t a great idea.

Conversely there are people who – out of choice, or for security requirements – never load images from emails. For some list demographics we’ve seen 10% or less of recipients loading images. Those recipients are real, they just work in computer security. Removing them to make your “open rate” or “engagement” look better isn’t a great idea either.

Engagement?

When we’re discussing email delivery we’ll often talk about delivery decisions being made by mailbox providers based on user engagement with a mail stream. We’ll also talk about engagement as ESPs and mail senders use the term to describe data derived from clicks and opens.

These are entirely different things.

Open and click data is useful to us, as senders, but we should be aware of the nuance in what they measure.

We need to understand exactly what we’re really measuring, so we can model what changes in those measurements may really mean, and make informed strategic decisions based on more than opens-go-up.

1. Quick ad for https://aboutmy.email/ – you can use that to check if the mail you’re sending is authenticated. ↩︎