Delivery Improvement

Life of an Email

I’m repeating the presentation I gave at M3AAWG in London for the Certified Senders Alliance.

It’s all about how to send an email by hand, and how knowing the mechanics of how an email is sent can help us diagnose email delivery issues.

We’re starting in about five hours from when I post this.

Register at https://register.gotowebinar.com/register/2268789893122531343

Troubleshooting delivery problems

Everyone has their own way of troubleshooting problems. I thought I would list out the steps I take when I’m trying to troubleshoot them.

Details matter

I field a lot of delivery questions on various online fora. Often people try and anonymise what they’re asking about by abstracting out the question. The problem is that there are very few answers we can give in the abstract.

ESPs and deliverability

There’s an ongoing discussion, one I normally avoid, regarding how much impact an ESP has on deliverability. Overall, my opinion is that as long as you have a half way decent ESP they have no impact on deliverability. Then I started writing an email and realised that my thoughts are more complex than that.

Wildfires and deliverability

A few weeks ago we took a drive down I5 to attend a service at Bakersfield National Cemetery. Amid the acres and acres of almond farms there were patches of black from recent grassfires. Typical but boring California landscape. Wildfires are a hugely destructive but continual threat in California. Growing up on the east coast, I never really understood wildfires. How can acres and acres and square miles just burn?
Having lived in California for almost as long as I lived on the east coast, I understand a bit better. In some ways, I have to. Even living right on the bay, there’s still some risk of fire. Like the grass fire a few miles from here across the street from the FB headquarters a few years ago. Further up the hills, there’s an even bigger risk of fire. Every driver can see the signs and precautions. Fields have plowed firebreaks around the edges. CAL FIRE posts signs alerting the public to the current fire risk status.

What do wildfires have to do with deliverability?
I associate wildfires and deliverability together because of a radio show I did a few years ago. It was pitched as a “showdown” between marketers and deliverability. I was the representative of deliverability. During the conversation, one of the marketers mentioned that deliverability people were too focused on the worst case scenario. That we spoke like we expected a fire to break out at any moment. His point was that deliverability spent too much time focused on what could happen and not enough time actually just letting marketers send mail.
His overall point was deliverability people should put out the fires, rather than trying to prevent them in the first place.
I thought about that conversation during the long drive down I5 the other day. I saw the firebreaks plowed into fields at the side of the road. And I saw the patches of blackness from fires reach along the highway where there were no firebreaks.
There are a group of marketers who really hate the entire concept of deliverability. Their point of view is that deliverability is hampering their ability to make money. I’ve even heard some of them assert they don’t care if 70% of their mail goes to the bulk folder. They should be allowed to send blasts of mail and deliverability shouldn’t tell them what they can do. Deliverability, so the complaint goes, is simply out to hurt marketers.
The only good deliverability is that which gets them unblocked when their behavior triggers IP based blocks. When the field is burning down, they’d like us to come spray water on it. And then go away and let them keep throwing lit cigarettes out their car windows.
But that’s not all that firefighting is about. Much of the work is preventing fires in the first place. In the US, a lot of that work is done through building codes. There are mandates like smoke detectors, fuel free spaces around dwellings, and sprinklers for some buildings. Monitoring local conditions and enforcing burn bans are also a large part of what the fire service does.
I like the fire fighter motif a lot. Much of what deliverability does is actually about preventing the block. ESPs have building code like standards for what mail is good and what is bad and what can be sent on their networks. Many of us publicly speak and educate about good practices and preventing blocks in the first place.
Fire prevention is about risk management and understanding how little things add up. Deliverability is similar. All the little things senders do to improve their deliverability adds up to a lower risk of fire. Yes, things like listbombing happen where even the best deliverability advice wouldn’t have prevented it. But, overall, deliverability wants to help senders get their mail in front of the people who can act on it. Some of that advice, though, takes the form of risk management and saying no.

What does good IP Reputation get you?

Today I was discussing some mailing list posts with an ESP colleague. He was telling me some interesting numbers he’d collected from different IP pools they maintain. He was testing routing mail through IPs based on subscription process and routing based on engagement metrics. The data showed that inboxing rates were similar across the test groups. As he put it, “IP reputation didn’t have much impact on inbox delivery.”

I’m not surprised. I’ve been talking for a while about how IP reputation is less important in reaching the inbox. In fact, it was almost 5 years ago now that I wrote The Death of IP Based Reputation. I updated it in 2015 with Deliverability and IP Reputation. Overall, IP reputation is a much smaller piece of reaching the inbox now than it has been in the past. I’ve talked about the reasons for this in the above posts. The short version is:

Looking forward

I had a number of very good talks with folks at the Email Innovations Summit earlier this week. I’m still digesting it all. It’s clear that getting to the inbox isn’t a solved problem. Around a decade ago I figured that the explosion of complaint feedback loops would make my job obsolete. That more data would mean anyone could manage delivery. That’s not the case for a couple reasons. The biggest is that filters don’t look just at complaints and there aren’t FBLs for all the other factors.
For whatever reason, many companies are still struggling with delivery.
Even more interesting is how changes in filters and inboxes are making it harder to measure delivery. In some ways I feel like we’re losing ground on inbox measurement. Filters changes and will keep changing, both to address emerging threats and to meet the needs and wants of subscribers. Gone are the days where Panels have their problems. Seed lists have their problems. There’s a longer blog post here, but it’s nearly the weekend and I’ve had a long week.
Hope you have something great planned.

Naming Names

One of the things that regularly happens at email conferences is a bunch of representatives from various ISPs and sometimes deliverability companies get up on stage and entertain questions from the audience about how to get email to the inbox. I’ve sat in many of these sessions – on both sides of the stage. The questions are completely predictable.
Almost invariably, someone asks if they can quote the ISP representative, because there is this belief that if you connect a statement with an employee name that will give the statement more weight. Except it doesn’t really. People who aren’t going to listen to the advice won’t listen to it even if there are names attached.
A lot of what I publish here is based on things the ISP reps have said. In some cases the reps actually review and comment on the post before I publish it. I don’t really believe attaching names to these posts will make them any more accurate. In fact, it will decrease the amount of information I can share and will increase the amount of time it takes to get posts out.
Last night I was joking with some folks that I should just make up names for attribution. Al did that many years ago, coining the pseudonym Barry for ISP reps. Even better, many of the ISP employees adopted Barry personas and used them to participate in different online spaces. Barry A. says X. Barry B. says Y. Barry C. says W. Barry D.
It doesn’t matter what names I attach.
I think I’m going to start adding this disclaimer to the appropriate blog posts:
Any resemblance to persons living or dead should be plainly apparent to them and those that know them. All events described herein actually happened, though on occasion the author has taken certain, very small, liberties with narrative.
Because, really.

The perfect email

More and more I’m moving away from consulting on technical setup issues as the solution to delivery problems. Delivery is not about the technical perfection of a message. Spammers get the technical right all the time. No, instead, delivery is about sending messages the user wants. While looking for something on the blog I found an old post from 2011 that’s still relevant today. In fact, I’d say it’s even more relevant today than it was when I wrote it 5 years ago.

Email is a fluid and ever changing landscape of things to do and not do.
Over the years my clients have frequently asked me to look at their technical setup and make sure that how they send mail complies with best practices. Previously, this was a good way to improve delivery. Spamware was pretty sloppy and blocking for somewhat minor technical problems was a great way to block a lot of spam.
More recently filter maintainers have been able to look at more than simple technical issues. They can identify how a recipient interacts with the mail. They can look at broad patterns, including scanning the webpages an email links to.
In short, email filters are very sophisticated and really do measure “wanted” versus “unwanted” down to the individual subscriber levels.
I will happily do technology audits for clients. But getting the technology right isn’t sufficient to get good delivery. What you really need to consider is: am I sending email that the recipient wants? You can absolutely get away with sloppy technology and have great inbox delivery as long as you are actually sending mail your recipients want to receive.
The perfect email is no longer measured in how perfectly correct the technology is. The perfect email is now measured by how perfect it is for the recipient.

June 2016: The Month in Email

We’re officially halfway through 2016, and looking forward to a slightly less hectic month around here. I hope you’re enjoying your summer (or winter, for those of you in the Southern Hemisphere).

Pete and Repeat

Pete and Repeat were on a boat. Pete fell out, who was left?

I was searching the blog for some resources today and these were the first two posts that showed up on the search results. I often feel like I’m repeating myself, but sometimes I am.

Things to read: March 9, 2016

It’s sometimes hard for me to keep up with what other people are saying and discussing about email marketing. I’ve been trying to be more active on LinkedIn, but there are just so many good marketing and delivery blogs out there I can’t keep up with all of them.

Here are a couple interesting things I’ve read in the last week.
Five Steps to Stay Out of the Spam Folder. Conceptually easy, sometimes hard to pull off in practice, these recommendations mirror many things I say here and tell my clients about delivery. The audience is in charge and your recipients are the best ally you can have when it comes to getting into the inbox.
Which states are the biggest sources of spam?. California and New York top the list, but the next two states are a little surprising. Over on Spamresource, Al points out the two next states have some unique laws that may affect the data. I just remember back in the day there were a lot of spammers in Michigan, I’m surprised there’s still a significant volume from there.
CASL didn’t destroy Canadian email. Despite concerns that CASL would destroy the Canadian email marketing industry, the industry is going strong and expanding. In fact, spending on email marketing in Canada was up more than 14% in 2015 and is on track to be up another 10% this year. Additionally, according to eMarketer lists are performing better because they’re cleaner.
A brief history of email. Part of the Guardian’s tribute to Ray Tomlinson, the person who sent the first email. Ray’s work literally changed lives. I know my life would be significantly different if there wasn’t email. Can you imagine trying to be a deliverability consultant without email? 🙂

ISPs speak at the EEC conference

Massimo Arrigoni has a great blog post up summarizing the final session of the EEC conference with representatives from major mailbox providers. This session a number of representatives from major mailbox providers spoke about what it takes to get to the inbox. They discussed what engagement really was, why you need to warmup and what the mailbox providers are measuring.
The short version is delivery is becoming more and more personalized. It’s not about if a mailbox provider thinks mail is spam, it’s about if a mailbox provider thinks this recipient thinks the mail will be spam. It’s all about connecting with each individual recipient.
None of this should be news to any of our regular readers. We’ve long talked about how ISPs measure things differently than senders. We’ve also talked about personalized delivery and how IP reputation isn’t the most important part of delivery.
Know your recipients. Make them want your mail and it will end up in the inbox. ‘
Edit: EmailCopilot also has more information on the ISP session at EEC.

Spam is not a moral judgement

Mention an email is spam to some senders and watch them dance around trying to explain all the ways they aren’t spammers. At some point, calling an email spam seems to have gone from a statement of fact into some sort of moral judgement on the sender. But calling an email spam is not a moral judgement. It’s just a statement of what a particular recipient thinks of an email.
There are lots of reasons mail can be blocked and not all those reasons are spam related. Sometimes it’s a policy based rejection. Mailbox providers publishing a DMARC record with a reject policy caused a lot of mail to bounce, but none of that was because that user (or that mailing list) was sending spam. Most cable companies prohibit customers from running mail servers on their cable connection and mail from those companies is widely rejected, but that doesn’t mean the mail is spam.
Sometimes a block is because some of the mail is being sent to people who didn’t ask for it or are complaining about it. This doesn’t make the sender a bad person. It doesn’t make the sending company bad. It just means that there is some issue with a part of the marketing program that need to be addressed.
The biggest problem I see is some senders get so invested in convincing receivers, delivery experts and filtering companies that they’re not spammers, that they miss actually fixing the problem. They are so worried that someone might think they’re spammers, they don’t actually listen to what’s being said by the blocking organization, or by their ISP or by their ESP.
Calling email spam isn’t a moral judgement. But, if too many people call a particular email spam, it’s going to be challenging to get that mail to the inbox. Instead of arguing with those people, and the filters that listen to them, a better use of time and energy is fixing the reasons people aren’t liking your email.

Tech Talk Podcast

Last week I had the pleasure of sitting down and talking delivery and email with W. Jeffery Rice of Brickstreet software. He’s posted a review and the recordings at Brickstreet and the UR Business Network.

Does email have a guarantee of delivery?

A client asked me earlier this week what SLAs ISPs provided for email delivery. The short answer is that there isn’t a SLA and that the only guarantee is that the email will get there when it gets there.
But as I was mentioning this to Steve, he pointed out that there was a recent change in the RFCs for email. In both RFC 821/2 and RFC 2821/2 (the original email related RFCs and the update in the early 2000’s) the RFCs stated that once a receiving MTA accepted an email that that MTA was required to either delivery the mail or generate an asynchronous bounce. While this isn’t a standard SLA, it does mean that a 2xy response after DATA meant the email would either be delivered to the user or be sent back to the sender. Despite the RFC requirements some receivers would still drop mail on the floor for various reasons, sometimes intentionally and sometimes not.
RFC 5321/2, the current SMTP standard, still says that once a server accepts the mail it must not lose that mail ‘for frivolous reasons.’ The RFC goes on to admit, though, that in recent years, SMTP servers are under a range of attacks and dropping mail on the floor is not frivolous in those cases.

Delivery is about helping you succeed

I was talking with another delivery person today who’s dealing with a customer struggling with some issues. As most of these discussions go, we get to the part where we have to tell the customer that what they’re doing looks problematic from the outside. And then the customer gets all upset and angry and starts complaining to account reps or managers or executives.
The challenge of delivery is working with clients who don’t want to hear they have to change what they’re doing. Some senders deflect better than a 3 year old caught with her hand in the cookie jar.
I think all of us in the delivery space, or at least most of us, want our customers and clients to succeed in their email goals. We want you to have a great mailing program. But when your delivery is having problems, getting to a great mailing program means doing something differently.
These changes can be hard, both in terms of thinking differently about email and how it works and about business models. Some business models make it extremely difficult to use emails. We understand that. We don’t make the rules, we just explain them.
We want your mail to work.

Spamhaus answers questions

Lost in all of the DOS attack news this week is that the first installment of Spamhaus answering questions from marketers in Ken Magill’s newsletter.
It’s well worth a read for anyone who is interested in hearing directly from Spamhaus.
One quote stood out for me, and it really sums up how I try to work with clients and their email programs.

Hear me talk about improving delivery

Tom Sather (ReturnPath) and I will be speaking on Thursday at a webinar hosted by the Association of Business and Media Companies.
How to Boost Email Marketing Response Rates

Delivery emergencies and the holidays

There is a lot of contention between ISPs and senders at the best of times. As we move into the holiday season, retailers are increasing their email marketing, sometimes quite significantly. This causes more delivery issues as recipients and MTAs react to the increased volume.
At many non retail companies, however, the pace of work slows down. There are distractions and office parties and people taking long lunches to finish their holiday shopping. Non-critical departments are not staffed for official holidays like Thanksgiving and Christmas Day.
This means that delivery issues may not be responded to as quickly as senders might like. Just this morning I got a call from someone who wants his delivery issues to be fixed by tomorrow. I’m sorry, even if I were to treat this as an emergency, there is work and investigation that needs to be done at the ISP end, and they’re not necessarily going to have a staffed delivery desk on Thanksgiving day. And even if they do have a staffed desk, it’s possible the staff won’t be focused and issues won’t be handled as fast as they might otherwise be.
I’d love to help, but there’s a limit to what I can do. Filtering decisions are made by the ISP, or their filter vendor, and sometimes they don’t happen as fast as we’d like. It’s frustrating for senders to have to deal with, but these are the realities of email delivery.

Emails that make you smile

This summer’s non-work project for me has been training for a 5K run with Fleet Feet in Menlo Park. As part of the training programs we get weekly emails from the store on Monday. As I was reading through today’s email, I found myself smiling and happy. Lisa, who is one of the store owners and writes the emails, is just so happy and bouncy and thrilled to share her love of running and that comes through in the newsletter.
Our group’s primary coach is the other store owner. During runs we often talk about random stuff, and when I tell people I do email delivery, they always start talking about their experience with email and spam. One night I was running with Jim, and we were talking about Jim’s experiences with sending email. He mentioned their ESP and talked about how convenient it was. But then he mentioned he wasn’t sure that they were sending enough mail (which made me laugh hard enough I almost tripped on a curb).
I realized I am not just a delivery expert when I started thinking about all the ways they could increase the amount of email they send, while still maintaining the quality and the friendly feel of their bulk emails. What could they offer local runners that would increase the value of the store to them? The first very obvious thing was a race calendar. There are dozens of local races every week, telling folks about upcoming races and entry deadlines would be a way to contact folks regularly without it always being a “buy stuff from us!!”
What commercial emails have you gotten recently that have made you smile?

Deliverability of Facebook.com email addresses

Christopher Penn at What Counts did some testing to see what delivery to Facebook.com addresses looks like. It looks pretty grim.

Delivery and marketing part 2

A while ago I wrote some thoughts about the conflicting requirements of delivery and marketing. I posted something similar over on the Only Influencers list, too. My thoughts generated a very interesting discussion, one that helped me clarify some of my somewhat random thoughts from earlier.
Marketing is about finding mindshare. One way you get mindshare is repetition. But people tune out repetition pretty quickly. Sending the same offers, the same copy over and over again means recipients start to tune things out. When recipients start tuning out mail, they may not bother opening it, they just read the subject line. If too many recipients start relying on the subject line then delivery can suffer.
Effective marketing relies on getting mail in front of the target audience. That’s the delivery component. Without inbox delivery, even the best marketing will not work.
No one will see marketing if it is in the spamfolder.
I don’t think you can cleanly separate delivery strategy from marketing strategy, but it’s important to realize they have different constraints and different pressures. When I talk about delivery with a client, I’m talking about getting mail into the inbox. And, most of the time, they’ve come to me because they’re not getting into the inbox and they have to make changes. The genius of their marketing is irrelevant, because no customers see it.
But once mail is in the inbox you can’t just ignore delivery, either. Sure, it becomes less of a pressure on the copy and the marketing strategy, until such time as the mail isn’t getting into the inbox any longer. Then it’s back to working on delivery and maybe having to implement some aggressive data hygiene. Back in the inbox and you can be aggressive on the marketing again.
Successful email marketing requires balancing the constraints of good delivery against the constraints of good marketing.

Inbox rates and conversion rates

Jeanne Jennings published an interesting bit of research on open rates and inbox rates at ClickZ recently. Essentially she looked at two different industry studies and compared their results.
The first study was the Return Path Global Delivery Survey and the second was the Epsilon North American Trend Results. What Jeanne found is that while Return Path shows a decrease in inbox placement, Epsilon is seeing an increase in average open rate.

There are any number of reasons this could be happening, including simply different ways the numbers are calculated. I am not sure it’s just a numbers issue, though. Many of Epsilon’s clients are very big companies with a very experienced marketing team. The Return Path data is across their whole user base, which is a much broader range of marketers at different levels of sophistication.
I expect that the Epsilon data is a subset of the Return Path data, and a subset at the high end at that. It does hint, though, that when the inbox is less cluttered, recipients are more likely to open the commercial mail that does get in there.

Only Influencers blog talk radio

I had the privilege to talk with a bunch of experts on the Only Influencers Blog Talk Radio show this morning. The discussion centered around the perceived conflict between Marketing and Delivery.
The conversation was a good one, with a lot of different perspectives aired. I strongly recommend people who are interested in hearing multiple industry experts talking about email marketing and delivery listen to the podcast.
Once I get back from MAAWG I plan to talk a little more about delivery managers as fire fighters and why that is such a good metaphor for delivery.

Dear Email Address Occupant

There’s a great post over on CircleID from John Levine and his experience with a marketer sending mail to a spam trap.
Apparently, some time back in 2002 someone opted in an address that didn’t belong to them to a marketing database. It may have been a hard to read scribble that was misread when the data was scanned (or typed) into the database. It could be that the person didn’t actually know their email address. There are a lot of ways spamtraps can end up on lists that don’t involve malice on the part of the sender.
But I can’t help thinking that mailing an address for 10 years, where the person has never ever responded might be a sign that the address isn’t valid. Or that the recipient might not want what you’re selling or, is not actually a potential customer.
I wrote a few weeks back about the difference between delivery and marketing. That has sparked conversations, including one where I discovered there are a lot of marketers out there that loathe and despise delivery people. But it’s delivery people who understand that not every email address is a potential purchaser. Our job is to make sure that mail to non-existent “customers” doesn’t stop mail from actually getting to actual potential customers.
Email doesn’t have an equivalent of “occupant” or “resident.” Email marketers need to pay attention to their data quality and hygiene. In the snail mail world, that isn’t true. My parents still get marketing mail addressed to me, and I’ve not lived in that house for 20+ years. Sure, it’s possible an 18 year old interested in virginia slims might move into that house at some point, and maybe that 20 years of marketing will pay off. It only costs a few cents to keep that address on their list and the potential return is there.
In email, though, sending mail to addresses that don’t have a real recipient there has the potential to hurt delivery to all other recipients on your list. Is one or two bad addresses going to be the difference between blocked and inbox? No, but the more abandoned addresses and non-existent recipients on a list there are on a list, the more likely filters will decide the mail isn’t really important or wanted.
The cost of keeping that address, one that will never, ever convert on a list may mean losing access to the inbox of actual, real, converting customers.

Experian CheetahMail believes that opt-out email appending is no longer an acceptable practice, and that marketers should no longer use of this practice to acquire customer email addresses. EmailResponsibly
Read More

Delivery versus marketing

I’ve been thinking lately that sometimes that what works for marketing doesn’t always work for delivery.
For instance in many areas of marketing repetition is key. Repeat a slogan and forge an association between the slogan and the product in the mind of the consumer. More repetition is better. Marketers can even go so far as using the same ad to drive consumer action. Television advertising is a prime example of this. Companies don’t create new content for every advertising slot, they create one or a few ads and then replay them over and over. The advertiser doesn’t even really care if the consumer consciously ignores the ads. The unconscious connection is still being made.
In the world of email delivery, though, having many or most recipients ignore advertising is the kiss of death. Too many unengaged users and filters decide that mail shouldn’t go into the inbox. These don’t even have to be ISP level filters, but Bayesian filters built into desktop mail clients.
Sending repetitive ads over email may be an effective marketing strategy, but may not be an effective delivery strategy.
Am I off base here and missing something? Tell me I’m wrong in the comments.

Looking towards the future

I had the opportunity to go to a seminar and networking event hosted by Return Path yesterday evening. The topic was “Email trends in 2012” and it was presented by Tom Sather.
If any of you get the opportunity to go to a talk presented by any of the Return Path folks I encourage you to do so. They know their stuff and their presentations are full of good information.
One of the trends mentioned is the increase in reliance on domain reputation. It’s something I’ve been thinking about more and more recently. I wrote a little bit about it recently, but have focused more on the whole realm of content filtering rather than just domain reputation.
Domain reputation is where delivery is going. And I think a lot of senders are going to struggle with delivery as they find that IP reputation is not enough to get into the inbox.

Listen to me talk about filtering, blocklists and delivery

I did an interview with Practical eCommerce a few weeks ago. The podcast and transcript are now available.
I want to thank Kerry and the rest of the staff there for the opportunity to talk email and filtering with their readers.
Happy Thanksgiving everyone in the US.

Having the same conversation

This morning I was reading a blog post about the failure of the congressional super committee. The author commented

What's the best ESP?

I often get clients and potential clients asking me to tell them what the absolute best ESP is.
“You’re an expert in the field, which ESP will give me the best inbox delivery?”
The thing is, there isn’t an answer to that question.
ESPs have expertise in sending large amounts of mail. All have staff that manage and monitor MTAs. Most have staff that provide advice on delivery issues. Many have staff that handle abuse complaints, FBLs and blocks.
What they don’t have is magic delivery fairies or bat phones into postmaster desks.
Simply moving mail to an ESP won’t give you delivery. For the most part, delivery is the responsibility of the sender, whether they send mail through an in house system or through an ESP.
Delivery is primarily about how recipients react to a particular mail stream. Send mail recipients want, interact with and relate to and you usually see good delivery. The IP addresses or infrastructure contribute but do not dominate the equation. Sending from an ESP won’t fix poor content, irrelevant mail or unengaged recipients.
I can hear everyone now shouting at their screen “What about shared IPs!!!?!?!” Yes, yes, if you use an ESP with shared IP addresses and the ESP gets a bad customer you may see poor delivery for a time because one of their other customers was bad. It’s a fact, it happens. Plus, if you use an ESP with dedicated IPs and the ESP gets a bad customer you may see poor delivery for a time because one of the other customers was bad and their IP is near yours.
So clearly the answer is to bring email in house. That way no other company can affect your delivery, right? Yes. Kinda.
Are you willing to invest money in hiring email and DNS savvy sysadmins? Invest money in a MTA designed to handle bulk mail? Invest in an expert who not only understands bounce handling, but can explain to your developers what a good bounce handling system must do? Invest in someone who can manage authentication like DKIM? Who can handle delivery issues and understands how to talk to ISPs? Invest in development to write a FBL processor?
For some companies, the internal investment is the right answer, and bringing mail in house makes business sense.
For a lot of companies, though, they just want to use email to communicate with customers. They don’t want to have to invest in multiple staff members (as it’s very rare to find a single person with all the various skill sets needed) to just send a weekly newsletter, or daily sales email. They need a tool that works, they don’t need to know how to sign up for a FBL, they don’t need to know how to handle bounces. They can outsource that work and focus on the communication value.
Finding the best ESP starts with finding out how you want to use email.
Question 1: What role does email play in my business?

Just go read here…

I wrote earlier this week about bad ways to evaluate and choose an ESP. It was all going to end today in an insightful and profound post telling all of you exactly how to find the best ESP.
Then Smartinsights published an insightful and useful article on choosing an ESP yesterday.
So, yeah, just go read what Jordie has to say. I have a couple other things to add, but I’ll drop those in another post.

You can't always get what you want

It’s a problem anyone who has done any delivery work has faced. There’s a client who is having blocklist problems or ISP delivery problems and they won’t pay any attention to what you say. They insist that you talk to the blocklist or the ISP or hand over contacts directly so they can “dialog with” someone internally. They don’t like what they’re hearing, and they hope that the answer will be different if they find a new person to talk to.
The reality is many of the people at ISPs and blocklists don’t want to talk to these types of senders. They may answer a friendly question from someone they know and trust, but sometimes not even then.
Some very large ISPs and major blocklists don’t even take sender questions. They won’t communicate with anyone about any delivery issues.
I’ve had to tell more than a few clients recently that various ISPs and blocklists weren’t interested in helping those clients with their delivery problems. There are two classes of reactions I get from clients. Some clients focus on moving forward. “OK, now what? How can we identify the issue, what data do we have and how can we figure out what the problem is?”
Other clients continue to look for ways to talk to whomever is blocking their mail. They’re convinced if they can just “explain their business model” or be told what they’re doing wrong, that all their delivery problems will magically disappear.
Needless to say those clients who focus on moving forward and looking at the information they do have have much better success resolving their delivery problems. What many senders don’t understand is the wealth of data they have that will help them resolve the issue. And even if they know it’s buried in their files, they don’t always know where to start looking or even what they’re looking for.
But that is, of course, why you hire someone like me who understands spamfiltering and email. I help senders understand how email filters work and identify what parts of their programs are likely to be responsible for delivery issues. I often find the most valuable service I provide to clients is a fresh set of eyes that can see the forest. With my help, they manage to stop obsessing unproductively about one particular symptom and focus on the underlying problems.
Senders who think the holy grail of problem resolution is speaking to the right person at an ISP or blocklist generally are disappointed, even when they hire someone who knows all the right people at the ISPs. They can’t always get what they want. But I can often help them get what they need.

Letters to the abuse desk

Ben over at Mailchimp has shared some of the mail that comes into the mailchimp abuse desk. It’s a post well worth a read.
One of the things that leaped out at me during that post is that the positive emails highlight how much the Mailchimp delivery and compliance people help their users get good delivery. They’re not just saying “you can’t do that” because they’re mean or they want to make life more difficult for their users. They are saying no because what the user wants to do is a bad idea.
I also appreciated the letter from the customer who had to tell Mailchimp that management had decided to not take Mailchimp’s advice. This is something that happens to me sometimes. Clients agree with my recommendations but management decides that they’re not going to implement them. It can be difficult to watch, particularly when I then see how much that company is struggling with blocks or see them show up on some of the big spam lists. But, it’s also part and parcel of the job. Not everyone, no matter how effectively I make my cases, will take my advice.

Thank you, Fred!

I am honored and humbled to be called out as a Goddess of Email Deliverability by Fred Tabsharani in his recent deliverability.com post. He has named and lauded people I am proud to call colleagues and friends. Thank you, Fred.

Guaranteed email delivery

Ben over at Mailchimp has a good post about his response (and his support staff’s more professional and helpful response) to inquires asking if Mailchimp can guarantee an improvement in delivery.
I sympathize with Ben, and commend his staff. I often get potential clients asking me if I can guarantee I can get their mail to the inbox or get them off a public or private blocklist. And, the answer really is no, I can’t guarantee anything. Much of delivery is solely in the hands of the actual sender. Sure, ESPs can enforce a certain standard of behaviour and they can do all the technical things right. And consultants like me can tell you how ISP spam filters work and explain how some of your choices and processes affect delivery. But none of us can guarantee inbox delivery.
Only one company has tried to guarantee inbox delivery, and they shut down earlier this year because they were non-viable and couldn’t get enough of a recipient userbase to attract customers.
For the rest of us, though, the best we can do is give senders the tools and information they need to succeed in getting mail delivered to the inbox.

Facebook Postmaster page

There’s still quite a bit of concern and worry about how the Facebook messaging platform is going to affect marketing. One thing that may help is the Facebook postmaster page. There’s all sorts of good information on those pages, reflecting the years of experience that their messaging team has in running large platforms.
Some points worth mentioning.

Zombie email: Part 2

In zombie email: part 1 I talked about how email addresses were tightly tied to internet access in the very early years of the internet. We didn’t have to worry about zombie email addresses because when an account was shut down, or ignored for a long time then mail would start bouncing and a sender could stop sending to that account.
There were two major changes to email accounts in the early 2000’s that led to the rise of zombie emails.
People started decoupling their internet access from their email addresses. Free addresses were easy to get and could be checked from everywhere. No longer did they have to dial in to get email, they could access it from outside the office and outside the home. Mobile devices, including the first generation of smart phones and laptops, helped drive people to use email addresses that they could access from any network. The easy access to free mail accounts and the permanence led people to adopt those addresses as their primary address.
When people changed addresses, for whatever reason, they didn’t have to stop paying. There was no way to tell the free ISPs to stop accepting mail for that address. Free mail providers would let addresses linger for months or years after the user had stopped logging in. Sometimes those addresses would fill up and start bouncing email, but they were not often turned off by the ISPs.
The lack of purging of abandoned addresses was the start of dead addresses accumulating on mailing lists. But there weren’t that many addresses in this state, and eventually they would fill up with mail. When they were full the ISP would stop accepting new mail for that account, and the address would bounce off a mailing list.
Everything changed with the entrance of Gmail onto the scene. When Gmail launched in 2004 they were providing a whole GB of storage for email accounts a totally unheard of storage capacity. Within a year they were providing multiple gigabytes of storage. Other freemail systems followed Gmail’s lead and now all free accounts have nearly unlimited storage. Plus, any mail in the spam folder was purged after a few weeks and bulk mail doesn’t count against the users’ storage quota. Now, an abandoned email account will almost never fill up thus senders can’t use over quota bounces to identify abandoned accounts.
Now we’re stuck in a situation where SMTP replies can’t be used to identify that there is no one home inside a particular email account. Senders can’t distinguish between a quiet subscriber and an abandoned address. ISPs, however, can and are using zombie addresses as a measure of a senders reputation.
On Monday we’ll talk about why and how zombie addresses can affect delivery. (Zombie emails: part 3)
Tuesday, we’ll talk about strategies to protect your list from being taken over by zombies. (Zombie Apocalypse)

Guide to resolving ISP issues

I often get a chuckle out of watching some people, who are normally on the blocking end of the delivery equation, struggle through their own blocking issues. A recent situation came up on a mailing list where someone who has very vehement opinions about how to approach her particular blocklist for delisting and that the lists policies are immutable. The company she works for is having some delivery issues and she’s looking for a contact to resolve the issues.
While digging through my blog posts to see if there was any help I could provide, I realized I don’t have a guide to resolving blocking issues at ISPs. Much of the troubleshooting can be done without ever contacting the ISPs or the blocklists.
Identify the issue.
There are a number of techniques that ISPs use to protect their users from malicious or problematic mail, from rate-liming incoming mail, putting mail in the bulk folder, or blocking specific IP addresses. Step one to resolving any delivery problem is to identify what is happening to the mail. In order to resolve the issue, you have to know what the issue is.
All too often, the description of a delivery problem is: My mail isn’t getting delivered. But that isn’t very clear as to what the actual problem is. Are you being temp failed? Is mail being blocked? Is mail going to the bulk folder? Is this something affecting just you or is it a widespread problem?
Troubleshoot your side.
Collect as much data about the problem as you can. Dig through logs and get copies of any rejection messages. Follow any URLs that are present in the bounce messages. Try sending a bare bones email to yourself at that ISP with just URLs, is it still blocked? What if you send from a different IP, does the same thing happen?
There is a lot of troubleshooting a sender can do without having to contact an ISP, and the information can lead to resolution that doesn’t involve having to contact the ISP. Also, many current ISP blocks are dynamic, they come up and go down without any human intervention. Those blocks that require contact to get them resolved have clear instructions in the bounce message.
Fix your stuff.
Whether it’s a reputation issue or a minor technical issue, fix the problem on your end. Just moving IP addresses or changing a URL isn’t a sustainable fix. There is a reason mail is being blocked or filtered and if you don’t fix that issue, the blocks are just going to come back. After you do fix your stuff, expect to see changes in a few days or a week. The ISP filters are generally quite responsive to sender improvements so if you’ve fixed the stuff you should see changes pretty quickly. Expect unblocking or filtering to take a little longer than the block was in place.
If you can’t figure out what the problem is, hire a consultant. Here at Word to the Wise we can often quickly identify a problem and provide a path to resolution. Sometimes the problem isn’t even the ISPs, we’ve had multiple cases where our clients were using custom software and their software wasn’t SMTP compliant and we were able to identify the problem and get their mail working again. There are a host of other independent consultants out there that can also help you identify and resolve blocking problems.
Contact the ISPs.
If there is a hard block or after fixing what you think the underlying problem is, you’ll have to contact the ISP. Many ISPs provide self service websites and contact forms to facilitate this process. Generally, though, most issues aren’t going to require contact.

Don't be the tomcat

Our tomcat, Grover, wants to go outside. He wants to go out the side door, so he’s been sitting in front of it, looking at me, then staring at the door. He’s been doing this for about half an hour, intermixed with occasional sad yowling.
The back door is open, and he can get from outside the back door to outside the side door in a minute, tops. But he wants to go out the side door.
(I’d get up and open the side door for him anyway, but I have another cat asleep on my lap and it’s not fair to move her just because Grover is being difficult).
I’ve been having a discussion with him:
Grover, the back door is open.
But I want to go out the side door.
You could go out the back door and then go round to the side door.
But I want to go out the side door.

Content based filtering

A spam filter looks at many things when it’s deciding whether or not to deliver a message to the recipients inbox, usually divided into two broad categories – the behaviour of the sender and the content of the message.
When we talk about sender behaviour we’ll often dive headfirst into the technical details of how that’s monitored and tracked – history of mail from the same IP address, SPF records, good reverse DNS, send rates and ramping, polite SMTP level behaviour, DKIM and domain-based reputation and so on. If all of those are OK and the mail still doesn’t get delivered then you might throw up your hands, fall back on “it’s content-based filtering” and not leave it at that.
There’s just as much detail and scope for diagnosis in content-based filtering, though, it’s just a bit more complex, so some delivery folks tend to gloss over it. If you’re sending mail that people want to receive, you’re sure you’re sending the mail technically correctly and you have a decent reputation as a sender then it’s time to look at the content.
You want your mail to look just like wanted mail from reputable, competent senders and to look different to unwanted mail, viruses, phishing emails, botnet spoor and so on. And not just to mechanical spam filters – if a postmaster looks at your email, you want it to look clean, honest and competently put together to them too.
Some of the distinctive content differences between wanted and unwanted email are due to the content as written by the sender, some of them are due to senders of unwanted email trying to hide their identity or their content, but many of them are due to the different quality software used to send each sort of mail. Mail clients used by individuals, and content composition software used by high quality ESPs tends to be well written and complies with both the email and MIME RFCs, and the unwritten best common practices for email composition. The software used by spammers, botnets, viruses and low quality ESPs tends not to do so well.
Here’s a (partial) list of some of the things to consider:

Email marketing is hard

I’ve watched a couple discussions around the email and anti-spam community recently with a bit of awe. It seems many email marketers are admitting they are powerless to actually implement all the good advice they give to others.
They are admitting they can’t persuade, cajole, influence or pressure their companies to actually follow best practices. Some of the comments public and private comments I’ve heard from various industry leaders:

Don't forget to check out the forest

I have the #emailmarketing feed on twitter scrolling live across my screen while I’m working. It’s been an interesting experience as many of the people who tweet #emailmarketing aren’t part of my social network.
Over the last week or so there’s been a lot of tweeting going on about Ben and Jerry’s GIVING UP EMAIL MARKETING!!! Only, come to find out, that’s not what they’re doing. Yes, they are moving more into the social networking arena but they will be continuing to connect with subscribers through email. Today many are tweeting that perhaps they “jumped the cow” with their initial reports of email abandonment by B&J.
Watching the ongoing discussions led me to wonder if a lot of email marketers are so focused on the trees that they miss the forest? Are they so disconnected from how people actually use email, and social networks for that matter, that they spend way to much time chasing a response and not enough time thinking about what they’re saying and doing?
Email marketing discussions often focus on a limited number of things, the biggest are how to get mail to the inbox and how to get recipients to engage. Many marketers spend time and money looking for the elusive combination of factors that will get their mail to the inbox and impel the recipient to give the sender money. The focus is on details like color and pre-headers and length and timing and content above and below the fold and the perfect call to action.
The discussions focus almost exclusively on the sender and only mention the subscriber in passing. That is understandable on one level. Senders can only control one end of the equation and figuring out what inputs compel the best response from the other side is what marketing is all about.
But there’s another part of email marketing, and that is that subscribers invite marketers into their inboxes. When someone subscribes to a newsletter or mail from a company they’re offering that company the opportunity to interact with them in their personal space. This is, in fact, the holy grail of marketing having the customer invite contact from a seller.
I suspect this is why the rumors of Ben and Jerry’s abandoning email had people all up in arms. A company abandoning a channel where they had an engaged and interested audience? PREPOSTEROUS! What’s happening to email as marketing?
I’ll be honest, I didn’t pay much attention because it was such a silly idea. Any marketer worth their salt wouldn’t give up a way to interact with customers. Ben and Jerry’s is a company with an almost cult like following. Anyone who was going to subscribe to a B&J newsletter was going to want that mail (new flavors! coupons! new locations! inside information!).
Someone started a rumor, though, that B&J were abandoning email marketing and everyone focusing on the trees grabbed that story and ran with it. They were so focused on the details they didn’t take a step back and think about what they were repeating. Had they taken a step back and thought about the forest they would have realized how silly the idea of B&Js abandoning email as a customer communication channel was.

Delivery consulting: it's all about the credibility

A few months ago I found a great blog post written by an ER doctor about how to convince other doctors to come in and deal with a patient in the middle of the night. There are quite few similarities between his advice and the advice I would give delivery experts, ISP relations folks and ESP representatives when dealing with ISPs and spam filtering companies.

Speaking to executives about deliverability

Exacttarget published a Deliverability whitepaper today. They interviewed a number of people around the email industry and asked them what they would tell C-level executives about email and email marketing.
It’s well worth a read, particularly given there are at least two ISP representatives speaking out about what they think makes a good email marketing program. You’ll see many of the themes we talk about here represented in the various articles.
Good delivery boils down to a few things, the most important of which is sending mail people have asked for and want.

Delivery Jobs

There are a couple companies currently looking for delivery specialists.
e-Dialog: Delivery Specialist
Responsys: Delivery Consultant
ThinData: Delivery & ISP Relations Analyst
ThinData: Privacy Analyst
Know anyone else hiring? Leave links in the comments.

Monitoring Email Deliverability

I did an interview with Direct Mag recently about what I recommend mailers do to monitor email deliverability.

Delivery Monitor Closing Down

Delivery Monitor by Aweber is one of the inbox monitoring services available for senders. Aweber has been in the process of winding down Delivery Monitor for the last few months and they will be turning the service off completely tomorrow.
A lot of folks have asked me about replacements for Delivery Monitor. There are, of course, Return Path and Pivotal Veracity, but many of the smaller mailers I talk to can’t justify the expenditure for either service.
Enter Green Arrow Monitor, a service provided by Green Arrow. This is a new seed list service aimed at marketers that need some delivery monitoring at commercial US ISPs. They’re reaching for the middle of the market. As a bonus, they’re offering special pricing for former Delivery Monitor customers.
While they don’t offer all the bells and whistles of other seedbox services, for the small to mid-size company that wants to know what their delivery is like at the major commercial ISPs this is a worthwhile service to investigate.
Full disclosure – I worked with GreenArrow to look at what parts of the market were being missed by other monitoring services and provide delivery consulting for some of their customers.

Delivery problems are not all spam related

Not every delivery failure is due to poor reputation or spam. Sometimes ISPs just have problems on their mailservers and so mail doesn’t get through. It’s often hard for delivery experts (and their bosses and their customers and their clients) to watch email delays or rejections without being able to do anything about it.
Sometimes, though, there is nothing to do. The rejections are because something broke at the ISP and they have to sort through it. Just this week there’s been a lot of twitter traffic about problems at a major cable company. They are rate limiting senders with very good reputations. They have admitted there is a problem, but they don’t have a fix or an ETA. From what I’ve heard it they’re working with their hardware vendor to fix the problem.
Hardware breaks and backhoes eat fiber. Yes, ISPs should (and all of the large ones do) have backups and redundancies. But those backups and redundancies can’t always handle the firehose worth of mail coming to the ISPs. As a result, the ISPs start rejecting some percentage of mail from everyone. Yahoo even has a specific error message to distinguish between “we’re blocking just you” from “we’re shedding load and temp failing everyone.”

Delivery case study

Mailchimp published a case study looking at a customer moving from sending in house to using Mailchimp.

The good, the typical and the ugly

In the theme of the ongoing discussions about ESPs and their role in the email ecosystem, I thought I’d present some examples of how different ESPs work.
The good ESPs are those that set and enforce higher standards than the ISPs. They invest money and time in both proactive and reactive policy enforcement. On Monday I’ll talk about these standards, and the benefits of implementing these policies.
The typical ESPs are those that have standards equivalent to those of the ISPs. They suspend or disconnect customers when the customers generate problems at the ISPs. They have some proactive policy enforcement, but most of their enforcement is reactive. On Tuesday I’ll talk about these standards and how they’re perceived by the ISPs and spam filtering companies.
The ugly ESPs are those that have low standards and few enforcement policies. They let customers send mail without permission. Some of the ugly ESPs even abuse other ESPs to send some of their mail, thus sharing their bad reputations across the industry. On Wednesday I’ll look at some of their practices and discuss how they affect other players in the industry.

Rescuing reputation

One of the more challenging things I do is work with companies who have poor reputations that they’re trying to repair. These companies have been getting by with poor practices for a while, but finally the daily delivery falls below their pain threshold and they decide they need to fix things.
That’s when they call me in, usually asking me if I can go to the ISPs and tell the ISPs that they’re not spammers, they’re doing everything right and will the ISP please stop unfairly blocking them. Usually I will agree to talk to the ISPs, if fixing the underlying problems doesn’t improve their delivery on its own. But before we can talk to the ISPs, we have to try to fix things and at least have some visible changes in behavior to take to them. Once they have externally visible changes, then we can ask the ISPs for a little slack.
With these clients there isn’t just one thing they’ve done to create their bad reputation. Often nothing they’re doing is really evil, it’s just a combination of sorta-bad practices that makes their overall reputation really bad. The struggle is fixing the reputation requires more than one change and no single change is going to necessarily make an immediate improvement on their reputation.
This is a struggle for the customer, because they have to start thinking about email differently. Things have to be done differently from how they’ve always been done. This is a struggle for me because I can’t guarantee if they do this one thing that it will have improved delivery. I can’t guarantee that any one thing will fix their delivery, because ISPs measure and weight dozens of things as part of their delivery making decisions. But what I can guarantee is that if they make the small improvements I recommend then their overall reputation and delivery will improve.
What small improvement have you made today?

Overusing ISP contacts

I’ve written frequently about personal contacts at ISPs and how the vast majority of delivery problems can be solved without picking up the “Bat Phone” and having someone at the ISP do something. Al touches on the same subject today, blogging about his recent experiences having to contact “Barry” multiple times for many different issues.
Al resolves

Delivery emergencies

There is no such thing as a delivery emergency. They just do not happen.
Delivery is fluid, delivery is changing, delivery is complex.
But when delivery goes bad it is not an emergency. There is no need to call up an ISP person at home on a Saturday afternoon and ask them to remove the filters. (And, BTW, experience indicates if you do this that you may have future delivery issues at that ISP.)
I’m sure that people will provide me with examples of delivery emergencies. And, in some cases I might even concede that the receivers will be happy to receive email immediately when it was sent. However, email as a protocol was designed for store and forward. It was not designed to transmit messages instantaneously from sender to receiver. Sure, it works that way much of the time these days. On the whole the Internet is fairly reliable and major servers are connected 24/7 (which wasn’t always the case).
Among many people, particularly recipients and ISP employees, there isn’t the expectation that bulk email is instantaneous. This leads to the belief that delivery problems are not an emergency. Everyone faces them, they get dealt with, life goes on. Demanding an escalation to deal with a “delivery emergency” may backfire and slow down how long it takes to get a response from an ISP.

Compliance vs. Deliverability

Most people I know handling delivery issues for senders have some version of delivery or deliverability in their job title. But as I talk to them about what they do on a daily basis, their role is as much policy enforcement and compliance as it is delivery. Sure, what they’re telling customers and clients is how to improve delivery, but that is often in the context of making customers comply with relevant terms and conditions.
Some delivery folks also work the abuse desk, handling complaints and FBLs and actually putting blocks on customer sends.
I think the compliance part of the delivery job description that is often overlooked and severely downplayed. No one likes to be the bad guy. None of us like handling the angry customer on the phone who has had their vital email marketing program shut down by their vendor. None of us like the internal political battles to convince management to adopt stricter customer policies. All of these things, however, are vital to delivery.
Despite the lack of emphasis on compliance and enforcement they are a vital and critical part of the deliverabilty equation.

Sending mail from unread email addresses

Some marketers, even large marketing companies, send mail from email addresses that are unread. Justin Premick posted a list of reasons this is a very, very bad idea. Be sure to read the comments, too.

The Weekend Effect

Sending mail only Monday through Friday can cause reputation and delivery problems at some ISPs, even when senders are doing everything right. This “weekend effect” is a consequence of how ISPs measure reputation over time.
Most ISPs calculating complaint rate use a simple calculation. They measure how many “this is spam” clicks a source IP generates in a 24 hour period. Then they divide that number by how many emails were delivered to the inbox in the same 24 hour period.
The weekend effect happens when a sender sends on weekdays and not on the weekend thus lowering the number of emails delivered to the to the inbox. Recipients, however, still read mail on the weekend, and they still hit the “this is spam” button on the email. Even if the number of “this is spam” clicks is lower than a normal weekday, with no incoming email the rate of spam complaints goes above ISP thresholds. Even a very well run mailing program may see spikes in complaint rate on the weekends.
Now, when the ISPs are measuring complaint rates over time, they take the average of the average complaint rates. If the rates spike high enough on the weekend (and they can spike to the 1 – 3% range, even for a well run list), that can hurt the senders’ reputation.
The good news is that ISPs are aware of the weekend effect and take this into account when manually looking at complaints. The bad news is that not all of the major ISPs take this into account when programatically calculating reputation.
There isn’t very much senders can do to combat the weekend effect, except be aware this can happen and may be responsible for poor mailing performances on Monday. If you are seeing delivery problems you think may be a result of the weekend effect you can contact the ISPs and ask for manual review of your reputation. Some ISPs can provide manual mitigation for senders with otherwise clean stats. d

How Spamfilters Work

AllSpammedUp has a post describing the primary techniques anti-spam filters use to identify mail as spam or not spam. While is this not sender or delivery focused knowledge, it is important for people sending mail to have a basic understanding of filtering mechanisms. Without that base knowledge, it’s difficult to troubleshoot problems and resolve issues.

Confirmed opt-in

I spent the morning in multiple venues correcting mis-understandings of confirmed opt-in. The misunderstandings weren’t so much that people didn’t understand how COI works, but more they didn’t understand all the implications.
In one venue, the conversation centered around how small a portion of deliverability the initial subscription process affects. Sure, sending unwanted, unexpected email can and does cause reputation problems, but merely using COI as a subscription methodolgy doesn’t automatically give a sender a good reputation or good delivery. Senders using COI as a subscription practice need to also need to send relevant and engaging mail that their recipients expect to receive. They need to handle their bounces well and purge or re-engage inactive subscribers. They need to keep their complaints low and their responses high.
How you manage subscriptions is only one factor in reputation schemes, and even if the subscription method is COI other factors can negate any bonus involved.
The second conversation involved Ken challenging me on the comment I left on his quiz yesterday. I said COI wasn’t foolproof and he challenged me to explain how. I did, and he’ll be following up next week.

Poor delivery is not always about spam

There are days I think we have trained people too well to believe every delivery problem is a misplaced spam block. We also have people trained to expect near 100% immediate delivery from send to inbox.
The problem is, email isn’t 100% reliable. It’s close. Very close. But sometimes mail just fails. It’s not because the ISPs hate you. It’s sometimes not even because the mail looks like spam.
Sometimes Mail Just Fails.
One of the challenges of working in email delivery is knowing enough to be able to separate out the random delivery failures from real delivery issues.

What Mark Said

Mark Brownlow skewers the arguments from opt-out proponents. A definite must read.

Deliverability versus delivery

Deliverability is a term so many people use every day, but what do we really mean when we use it? Is there an accepted definition of deliverability? Is the concept different than delivery?
At a recent conference I was running a session talking about email delivery, senders and the roles senders play in the email industry and at that particular organization. The discussion went on for a while, and the subject of deliverability versus delivery came up. J.D. Falk had a comment about the difference that resonated with me. Paraphrased, he said:

Email is store and forward

Many of us are so used to email appearing instantaneous, we forget that the underlying protocol was never designed for instant messaging. When the SMTP protocol was originally proposed it was designed to support servers that may have had intermittent connectivity. The protocol allowed for email to be spooled to disk and then sent when resources were available. In fact, almost everyone who was around more than 10 years ago knows of a case where an email took weeks, months or even years to deliver.
These days we’re spoiled. We expect the email we send to friends and relatives to show up in their mailbox within moments of sending it. We expect that sales receipt or e-ticket to show up in our mailbox within instants of a purchase. We expect that our ISPs will get us email immediately, if not sooner.
But there are a lot of things that can slow down email delivery. At several points in the process an email may be spooled to disk. It stays on the spool until the next part of the delivery process can happen. Other points of slowdown include the various anti-spam, anti-virus and anti-phishing protections that ISPs must implement. Then add in the extreme volume of email (around 10 billion messages a day) and all of a sudden email delivery is slower than many senders and recipients expect it to be. This delay is not ideal, but the system is designed so that mail is not silently discarded.
While individual emails may be delayed, most users will rarely see that delay in the email that they send. Bulk senders, who may be sending thousands or hundreds of thousands of emails a day, may see more delays in a single send than the average user sees in years of sending one-to-one email.
Email is store and forward, not instant. Sometimes that means there is a delay in getting email into the recipients inbox. And, sometimes there isn’t anything anyone can do to speed up delivery, except to adjust expectations of how email works.

AOL talks about reputation

Over at the AOL postmaster blog, Christine posts about reputation and AOL.

We want your mail to succeed

One thing I hear from a lot of delivery folks, both consultants and those who work at the ESPs, is that their customers and clients fight back whenever they say no. A client or a customer proposes this great idea that involves sending irrelevant email to uninterested people. Then, with bated breath, they ask their delivery consultant to agree it is a brilliant idea. Most of the time, their great idea is actually a bad idea. Those of us who have been around a while can even and provide examples and experiences that back up that it is a bad idea.
The result is similar, when told their idea will hurt their delivery they fight tooth and nail. On good days they will argue and decide to listen. On bad days they go off and do what they were warned not to do.
It can be horribly frustrating for all of us in the delivery field. We actually want customers’ mail to succeed. We tell customers no, not because we want to ruin their day or their business or their ideas, but because we want to help their business. Our job is to make their email work, and sometimes that means saying no.
Next time your delivery consultant, or your ESP delivery expert, tells you that an idea may cause delivery problems, give them some credit for their experience and expertise. We really do have your best interests at heart and really do want your email to succeed.

Reputation: part 2

Yesterday, I posted about reputation as a combination of measurable statistics, like bounce rates and complaint rates and spamtrap hits. But some mailers who meet those reputation numbers are still seeing some delivery problems. When they ask places, like AOL, why their mail is being put into the bulk folder or blocked they are told that the issue is their reputation. This leads to confusion on the part of those senders because, to them, their reputation is fine. Their numbers are exactly where they were a few weeks ago when their delivery was fine.
What appears to have changed is how reputation is being calculated. AOL has actually been hinting for a while that they are looking at reputation, and even published a best practices document back in April. Based on what people are saying some of that change has started to become sender visible.
We know that AOL and other ISPs look at engagement, and that they can actually measure engagement a lot more accurately than sender can. Senders rely on clicks and image loading to determine if a user opened an email. ISPs, particularly those who manage the email interface, can measure the user actively opening the email.
We also know that ISPs measure clicks. Not just “this is spam” or “this is not spam” clicks in the interface, but they know when a link in an email has been clicked as well.
I expect that both these measures are now a more formal and important part of the AOL reputation magic.
In addition to the clicks, I would speculate that AOL is now also looking at the number of dead addresses on a list. It is even possible they are doing something tricky like looking at the number of people who have a particular from address in their address book.
All ISPs know what percentage of a list is delivered to inactive accounts. After a long enough period of time of inactivity, mail to those accounts will be rejected. However for some period of time the accounts will be accepting mail. Sending a lot of mail to a lot of dead accounts is a sign of a mailer who is not paying attention to recipient engagement.
All ISPs with bulk folders have to know how many people have the from address in their address book. Otherwise, the mail would get delivered incorrectly. In this way, ISPs can monitor the “generic” recipient’s view of the email. Think of it as a similar to hitting the “this is not spam” button preemptively.
This change in reputation at the ISPs is going to force senders to change how they think of reputation, too. No longer is reputation all about complaints, it is about sending engaging and relevant email. The ISPs are now measuring engagement. They are measuring relevancy. They are measuring better than many senders are.
Senders cannot continue to accrete addresses on lists and continue sending email into the empty hole of an abandoned account while not taking a hit on their reputation. That empty hole is starting to hurt reputation much more than it helps reputation.

New email related blog

Mickey Chandler, of SpamSuite.com has launched a new email delivery specific blog: Spamtacular.com. He moved a number of posts from his other blog, but today has a new post up about how a prior business relationship impacts compliance with CAN SPAM. He concludes with:

Overheard at the airport

Sitting at the gate, waiting for boarding I overhear a conversation. A woman is texting on her blackberry and saying to her traveling partner, “You know that universal sign for ‘not’? He thinks that is why mail is ending up in the junk folder. And if we add it to our mail then it will get delivered.”
This is one of the strangest theories of email delivery I have heard in a long time.