Bulk Folder

Forget about engagement, think inboxing

While answering a question about how to improve IP reputation at Gmail I realized that I no longer treat Gmail opens as anything about how a user is interacting with email. There are so many cases and ways that a pixel load can be triggered, without the user actually caring about the mail that it’s not a measure of the user at all.

Fun with spam filters

I recently had a challenging travel experience in the Netherlands, trying to get from Schipol airport to a conference I was speaking at. As part of my attempt to get out of the airport, I installed UBER on my phone. There were some challenges with getting UBER to authorise my phone number, so I tried linking it to my Gmail account.

Why is my cold email going to the spam folder?

Because that’s what the spam folder is for unsolicited email.

Did the algorithm change?

When faced with unexplained deliverability changes one of the first questions many folks ask is “Did the algorithm change.” In many ways this is an meaningless question. Why? Because there are two obvious answers to the question.
A1: Of course it didn’t.
A2: Of course it did.
Both answers are correct, but they’re answering different underlying questions. When we understand how two diametrically opposed answers are both correct, we understand much more about filtering.

Doing email right

Over on the MarketingLand website, Len Shneyder talks about 3 companies (Uber, REI and eBay) that do email right. In there he shows how the companies use email to further their business goals while understanding and meeting the needs of their customers.
Meeting the needs of recipients is the way to get your mail to the inbox. Send email that your users want, and they will tell the ISPs when they don’t get your mail. It’s sometimes hard to convince senders of this. Instead they want to tweak URLs or authentication or IPs or domains. But none of those things are what deliverability is all about. Deliverability is about the recipient. Deliverability is about the relationship between the sender and recipient.
Send to the right people – and the right people are those who have asked for and want your mail – and deliverability problems don’t materialize. Sure, every once in a while something might happen that throws mail into the bulk folder for one reason or another. But fighting to get to the inbox isn’t an every day thing. Instead, senders can focus on knowing their users and sending mail that makes them happy when it shows up in the inbox.

Filters do what we tell them

In the email space we talk about filters as if they were sentient beings. “The filters decided…” “The filters said…” This is convenient shorthand, but tends to mask that filters aren’t actually deciding or saying anything. Filters are software processes that follow rules dictated by the people who create and maintain them. The rules flow from the goals set by the mailbox provider. The mailbox provider sets goals based on what their users tell them. Users communicate what they want by how they interact with email.

What we end up with is a model where a set of people make decisions about what mail should be let in. They pass that decision on to the people who write the filters. The people who write the filters create software that evaluates email based on those goals using information collected from many places, including the endusers.
What mail should be let in is an interesting question, with answers that differ depending on the environment the filter is deployed in.
Consumer ISPs typically want to keep their users happy and safe. Their goals are to stop harmful mail like phishing, or mail containing viruses or malware. They also want to deliver mail that makes their users happy. As one ISP employee put it, “We want our users to be delighted with your mail.”
Businesses have a few other goals when it comes to filters. They, too, need filters to protect their network from malicious actors. As businesses are often directly targeted by bad actors, this is even more important. They also want to get business related email, whether that be from customers or vendors. They may want to ensure that certain records are kept and laws are followed.
Governments have another set of goals. Universities and schools have yet another set of goals. And, of course, there are folks who run their own systems for their own use.
Complicating the whole thing is that some groups have different tolerances for mistakes. For instance, many of our customers are folks dealing with being blocked by commercial filters. Therefore, we don’t run commercial filters. That does mean we see a lot of viruses and malware and rely on other strategies to stop a compromise, strategies that wouldn’t be as viable in a different environment.
Filters are built to meet specific user needs. What they do isn’t random, it’s not unknowable. They are designed to accomplished certain goals and generally they’re pretty good at what they do. Understanding the underlying goals of filters can help drive solutions to poor delivery.
Use the shorthand, talk about what filters are doing. But remember that there are people behind the filters. Those filters are constantly maintained in order to keep up with ever changing mail streams. They aren’t static and they aren’t forgotten. They are updated regularly. They are fluid, just like the mail they act on.

From the archives: Taking Permission

From February 2010, Taking Permission.

Permission is always a hot topic in email marketing. Permission is key! the experts tell us. Get permission to send email! the ISPs tell us.
Marketers have responded by setting up processes to “get” permission from recipients before adding them to mailing lists. They point to their privacy polices and signup forms and say “Look! the recipient gave us permission.”
In many cases, though, the permission isn’t given to the sender, permission is taken from the recipient.
Yes, permission is being TAKEN by the sender. At the point of address collection many senders set the default to be the recipient gets mail. These processes take any notion of giving permission out of the equation. The recipient doesn’t have to give permission, permission is assumed.
This isn’t real permission. No process that requires the user to take action to stop themselves from being opted in is real permission. A default state of yes takes the actual opt-in step away from the recipient.
Permission just isn’t about saying “well, we told the user if they gave us an email address we’d send them mail and they gave us an email address anyway.” Permission is about giving the recipients a choice in what they want to receive. All too often senders take permission from recipients instead of asking for permission to be given.
Since that post was originally written, some things have changed.
CASL has come into effect. CASL prevents marketers from taking permission as egregiously as what prompted this post. Under CASL, pre-checked opt-in boxes do not count as explicit permission. The law does have a category of implicit permission, which consists of an active consumer / vendor relationship. This implicit permission is limited in scope and senders have to stop mailing 2 years after the last activity.
The other change is in Gmail filters. Whatever they’re doing these days seems to really pick out mail that doesn’t have great permission. Business models that would work a few years ago are now struggling to get to the inbox at Gmail. Many of these are non-relationship emails – one off confirmations, tickets, receipts. There isn’t much of a relationship between the sender and the recipient, so the filters are biased against the mail.
Permission is still key, but these days I’m not sure even informed permission is enough.

Use all the channels

One of the hardest deliverability situations to address is when all mail from a certain sender is going to the bulk folder. I’ve had numerous clients come to me to address this situation over the years. Ideally, clients come to me before all their mail is going to bulk. Then we can make some tweaks and changes to their mail program, repair the reputation and then recover other addresses. We have knobs we can twist to fix things if some people are still getting messages in their inbox. We have data to measure.
When all mail is going to bulk, though, we lose access to the knobs and the data. There are zero complaints if mail is going to bulk. There are no opens or clicks, because many ISPs disable images and links in the bulk folder. Our normal “fixing reputation” tools are taken away from us.
Senders with all their mail going to bulk are faced with a profound challenge. How can they engage customers who are unengaged and who are not seeing mail at all? How can we fix deliverability when our normal tools and metrics are unavailable?
If we can get even a small percentage of recipients to go pull mail out of bulk or spam and move it to their inbox, then we’re well on our way to repairing reputation. But how can we get them to go look for the mail in the bulk folder. Recent Litmus research suggests that a significant percentage of folks regularly check their spam folder, but this isn’t always enough to repair reputation,
The question becomes how can the senders encourage recipients to go digging through their spam folder.
This is the point where I start quizzing clients on what other channels they use to communicate with their customers. I’ll run through the whole list: social media, snail mail, push notices through apps, SMS, website popups, Facebook ads. I work with them to identify users who are engaged with their brand and brainstorm ways to get those users to look for mail.
I’m always pleased to see large brands using these strategies. Just recently Blizzard used twitter to communicate with their users about email problems. They tweeted.
BlizzardTweet
The link takes you to the Blizzard support site. Where they give specific instructions on how to whitelist mail and what mail to whitelist.

Google takes on intrusive interstitials

Starting next January, Google will be modifying its mobile search results to lower the ranking of sites that use interstitials that interfere with the users experience. In a blog post announcing the change they explain:

Yahoo changes

Thanks to tips by a couple blog readers and some clients, I have been looking into Yahoo disabling links in the bulk folder. It does appear Yahoo is no longer allowing users to click on links in emails that Yahoo places in the bulk folder.
In fact, some of the spam in my Yahoo mailbox even has a notice about this.

Gmail and the bulk folder

Earlier this week Gmail announced they were providing reasons for why they delivered a particular mail to the bulk folder. I’m sure a lot of senders are rejoicing over the clear feedback. After all this is exactly what they’ve been asking for “tell us why you’re filtering our mail and we’ll fix it.”
I am not sure, however, that this is going to help the majority of senders seeing mail going to the bulk folder. On the Gmail support pages, they list a number of the explanations they’re be providing.

Delivery challenges increasing

Return Path published their most recent Global Deliverability report this morning. (Get the Report) This shows that inbox placement of mail has decreased 6% in the second half of 2011. This decrease is the largest decrease Return Path has seen in their years of doing this report.
To be honest, I’m not surprised at the decrease. Filters are getting more sophisticated. This means they’re not relying on simply IP reputation for inbox delivery any longer. IP reputation gets mail through the SMTP transaction, but after that mail is subject to content filters. Those content filters are getting a lot better at sorting out “wanted” from “unwanted” mail.
I’m also hearing a lot of anecdotal reports that bulk folder placements at a couple large ISPs increased in the first quarter of 2012. This is after the RP study was finished, and tells me increased bulk folder placement is more likely to be a trend and not a blip.
One of the other interesting things from the RP study is that the differences are not across all mail streams, but are concentrated in certain streams and they vary across different regions.

About that Junk Folder

I use a pretty standard mail filtering setup – a fairly vanilla SpamAssassin setup on the front end, combined with naive bayesian content filters in my mail client. So I don’t reject any mail, it just ends up in one of my inboxes or a junk folder. And I have a mix of normal consumer mail – facebook, twitter, lots of commercial newsletters, mail from friends and colleagues and spam. (As well as that I have a lot of high traffic industry mailing lists, but overall it’s a fairly normal mix.)
My bayesian filter gets trained mostly by me hitting “this is spam” when spam makes it to my inbox. If I’m expecting an email “immediately” – something like a mailing list COI confirmation or email as part of buying something online – I’ll check my spam filter and move the mail to my inbox in the rare case it ended up there. Other than that I let it and spamassassin chug along with no tweaking.
I’m starting a data analysis project, based on my own inboxes, and as part of that I’m using some tools to look for false positives in my junk folders, and manually fixing anything that’s misclassified. I’ve been doing this for a couple of hours now, and I’ve found some interesting things.

Social media to improve email delivery

Mail delivered to the bulk folder is likely to continue landing in the bulk folder without intervention. Sometimes a sender can talk to the ISP involved and get mail moved back to the inbox. Sometimes a sender can make hygiene changes and get mail moved back to the inbox.
The most effective way to get mail delivered to the inbox, however, is for recipients to go into the bulk folder and mark the mail as “not spam.” Nothing is more effective at getting mail delivered to the inbox.
But there is a bit of a catch 22 there. If mail ends up in the bulk folder consistently, recipients tend to forget about it. Many people trawl through their bulk folder sporadically, if at all. If recipients aren’t engaged with mail and don’t know when they should see it, then they won’t miss it and won’t look for it.
So if mail is ending up in the bulk folder and recipients aren’t expecting it what can a sender do? One of the obvious answers is find another channel. Let recipients know through some channel besides email that they need to look in their bulk folder for a particular email.
In the past it was difficult to find non-email ways to connect recipients. I worked with customers who really had no other way to interact with recipients than email. They weren’t running a website, they didn’t have any other contact methods, they were really stuck. But a recent tweet from AppSumo shows how social media can be used to improve email delivery.

Engagement based delivery makes testing tricky

Yesterday I wrote about how important recipients are to achieving good delivery. The short version of yesterday’s post is that delivery is all about engagement, and how the ISPs were really focusing on engagement and proving custom user experiences.
This is great, for the user. Take the common example where a commercial list has some highly engaged recipients and a bunch of recipients that can take or leave the mail. The ISP delivers the newsletter into the inbox of the highly engaged recipients and leaves it in the bulk folder of less engaged recipients.
With user focused delivery people get the mail they are interested in where they can read it and interact it. People who have demonstrated a lack of interest for a topic or a sender don’t see that mail.
This can get complicated for those of us trying to troubleshoot deliver problems, though. I have a couple mail accounts I use for testing at various ISPs. Even though I do very little to try and personalize the account I am seeing behaviour that leads me to wonder if ISP personalizing the inbox experience is going to make it that much more difficult to troubleshoot delivery issues.
I have to wonder, too, where this leaves delivery monitoring services in the future. If delivery is personalized, how can you know that the delivery monitoring addresses are representative any longer? Is there even a “representative” mailbox any longer?

Recipients are the secret to good delivery

Many, many people hire me to educate them on delivery and fix their email problems. This is good, it’s what I do. And I’m quite good at helping clients see where their email program isn’t meeting expectations. I can translate tech speak into marketing. I can explain things in a way that shifts a client’s perception of what the underlying issues are. I can help them find their own way into the inbox.
But…
Most of what I do is simply think about email delivery from the point of view of a recipient and help clients better meet their recipient’s expectations. This works. This works really well. If you send mail that your recipients want your mail gets to the inbox.
Here’s the secret: ISPs and most spam filters have a design goal to deliver mail their users want. They only want to block mail their users don’t want.
Filters are not designed to block wanted mail.
Sure there are complicated situations where senders have gotten behind the 8 ball and need some help cleaning up. There are situations where filters screw up and block mail they shouldn’t (and aren’t quite designed to). Spam filters are complicated bits of code and sometimes they do things unexpectedly. All of these things do happen.
But these situations happen a lot less than most senders think. Most of the time when mail is hitting the bulk folder, or is throttled at the MTA the issue is that recipients don’t care about the mail.
Recipients aren’t engaged with a particular sender or particular brand. So ISPs react accordingly and that mail ends up slowly delivered or bulked. This upsets the senders to no end, but the recipients? The recipients often don’t care that some mail shows up in bulk or arrives Wednesday afternoon instead of Tuesday evening.
When recipients are engaged with a particular sender or brand, though? Delivery is fast and reliable. Mail is rarely delayed or bulked. When recipients want mail, they interact with it. They look in the bulk folder. They miss it when it’s not there. They complain to the ISPs when they don’t get it. The ISPs react accordingly and prioritize or “red carpet” that email.
The secret to really good delivery is to get your recipients to handle your ISP relations for you. Send mail they miss when they don’t get it, and you’ll discover most of your delivery problems go away.

Mailing old addresses: 5 questions to ask first

James asked the question on twitter:

If you haven’t mailed an address in 5-10 yrs, would you include it in a re-engagement mail?
Read More

When the inbox isn't the inbox

There was a discussion today on the OI list about email filtering that brought up something I usually don’t mention in delivery discussions. Most email marketers treat the inbox as the holy grail of delivery. Everything about delivery is focused on getting to the magical inbox.
I think, though, that inbox is often just shorthand for “not landing in the bulk or spam folders.”
For some recipients, particularly those of us who get lots of mail, sometimes it’s better to land in a folder rather than the inbox. I have a folder set up, where most of my commercial mail goes. It’s labeled “commercial.” I check it once or twice a day.
This is beneficial to me and to the senders. Why? Because when I check that folder I’m ready to actually look at my commercial mail. I’m looking for those offers.
For someone like me, who does most of their work in their inbox, commercial interruptions are a problem. Commercial mail that ends up in my inbox, which can happen if I’ve been lazy about filters, interrupts me and usually doesn’t get read. But when it’s in my commercial folder? Well, then I can look at it, visit websites and make purchases.
So just remember, it’s not that you want mail in the inbox as much as you want mail somewhere that the recipient will notice it.

Is your mail being bulk foldered?

Daisy at Signup.to posted a list of 11 things to do when mail is going to the bulk folder. Her suggestions are a good start to troubleshooting and fixing persistent bulk foldering of mail.
One thing she doesn’t mention is that while bulk foldering can sometimes be the result of poor content, more often it’s the result of unengaged recipients. Think of bulk foldering this way: the ISP has some subscribers they’re pretty sure want your mail, so they’re not going to block your mail. But they’re pretty sure a lot of subscribers don’t want your mail so they’re not going to deliver it to the inbox.
The trick to getting mail moved out of the bulk folder is to get more people engaged with your email marketing. This is tough to do if they’re not actively checking their bulk folder for mail but there are some ways I’ve helped clients get mail into the inbox.