Bounce Handling

Diagnosing Hard Bounces

A very short post about diagnosing hard bounces, because I’ve had to give the same advice to a dozen folks over the past few months.

Your bounce classification is a bit rubbish

When a mailbox provider rejects or defers an email it sends back a message explaining why.

Let’s Talk: Bounce handling

Next Let’s Talk session is June 17 with the topic of bounce handling. As always: 5pm Dublin, noon Eastern, 9am Pacific. Send an email to laura-ddiscuss@ the obvious.

The nice folks over at Postmark shared a new deliverability resource last week. The SMTP Field Manual. This is a collection of SMTP responses they’ve seen in the wild. This is a useful resource. They’re also collecting responses from other senders, meaning we can crowdsource a useful resource for email deliverability folks.

What’s a bounce?

Bounces and bounce handling is one of those topics I’ve avoided writing about for a long time. Part of my avoidance is because there are decades of confusing terminology that hasn’t ever been really defined. Untangling that terminology is the first step to being able to talk sensibly about what to do. Instead of writing a giant long post, I can break it into smaller, more focused posts.

Targets and measures

April 2017: The Month in Email

April was a big travel month for us. I went to Las Vegas for meetings around the Email Innovations Summit and to New Orleans, where Steve spoke on the closing keynote panel for the EEC conference.
I wrote several posts this month about privacy and tracking, both in email and in other online contexts. It’s increasingly a fact of life that our behaviors are tracked, and I wrote about the need for transparency between companies and those they are tracking. More specifically, I talked about the tradeoffs between convenience and security, and how people may not be aware that they are making these tradeoffs when they use popular mailbox tools like unroll.me. The folks over at ReturnPath added a comment on that post about how they handle privacy issues with their mailbox tools.
Steve contributed several posts this month. First up, a due diligence story about how service providers might look more closely at potential customers for their messaging platforms to help curtail spam and other fraudulent activity. He also looked at the history of “/8” IP blocks, and what is happening to them as the internet moves to IPv6. Steve also added a note about his new DMARC Validation tool, which rounds out a suite of free tools we’ve made available on our site. And finally, he showcased a particularly great email subscription experience from Tor.com — have a look!
I highlighted another post about companies doing things right, this one by Len Shneyder over at Marketingland. In other best practices news, I talked about bounce handling again (I mentioned it last month too), and how complicated it can be. Other things that are complicated: responding to abuse complaints. Do you respond? Why or why not?
Our friends at Sendgrid wrote a great post on defining what spammers and other malicious actors do via email, which I think is a must-read for email marketers looking to steer clear of such activity. Speaking of malicious actors, I wrote two posts on the arrest of one of the world’s top email criminals, Peter Levashov, and speculation that he was involved in the Russian hacking activity around the US elections. We’re looking forward to learning more about that story as it unfolds.

March 2017: The Month in Email

It’s that time again… here’s a look at our last month of blog posts. We find it useful to recap each month, both to track trends and issues in email delivery and to provide a handy summary for those who aren’t following along breathlessly every single day. Let us know if you find it useful too!

As always, I wrote about email filters. It’s so important to recognize that filters aren’t arbitrary — they’re detailed instructions that help meet specific user needs, and the more you are cognizant of that, the better you’ll be able to work with them. Additionally, filters aren’t perfect and likely never will be. False positives and false negatives are frustrating, but as long as spam is still a viable business for spammers, they’ll continue to figure out how to work around filters. As such, we can’t expect filters to be 100% accurate in determining what constitutes wanted and unwanted mail.
Part of this, of course, is due to the problem of fraudulent signups. Companies aren’t particularly vigilant about address acquisition and hygiene, and as a result, they’ll claim you “signed up” for their email when you did not. Some people believe that a confirmed opt-in (COI) will solve this problem, but our experience is companies are reluctant to leave revenue on the table, and that they will continue to mail to addresses that have not confirmed.
Address sharing and co-reg is also part of the problem. As we saw in the extensive RCM data breach, many major brands continue to work with third-party senders to send mail in ways that are quite clearly spam. And in more criminal activity, I looked at the rise of botnets and how some of those criminals were brought to justice. In other justice news, there’s been an indictment in the Yahoo breach and another CASL enforcement action.
I wrote a post about bounce handling and “relaying denied” error messages, which are quite rare. It’s useful to have an understanding of these and other error messages, since bounces are sometimes indicative of a larger technical issue, such as when AOL accidentally bounced all messages for a short period last week. Speaking of AOL, we noted that there’s no official timeline for the move from Verizon addresses to AOL addresses following the 2015 acquisition, but it may be worth considering asking your customers to update their addresses.
Spam and filters aren’t the only factors of course. It can be challenging to figure out the multiple factors that make up the black box of delivery. And of course, the most important part of delivery continues to be engagement, engagement, engagement.
I wrote a few posts this month on why I do what I do, and why it’s so important to me. First, I wrote about A Day Without A Woman, and my choice not to participate in offering advice and guidance for that day. The truth is that I enjoy sharing what I know and helping people solve problems. I was honored to be named one of 11 Innovators in Email, and I know that my volunteer work in the industry and my unpaid blogging work is a big part of that. It may sound corny, but I really do believe we are on the front lines of the fight of good vs. evil online, and despite the distractions of politics and world events, we must all continue to do our part.

AOL accidentally hard bounces valid mail

Last night (Mar 29, 2017) between about 8pm Eastern and 9:30pm Eastern AOL suffered a technical issue. Every email sent to them received a “Recipient address rejected” reply. One example of the error message:
Mar 29 20:45:12 p2-lvmail11 lsb1-99-208-250/smtp[22251]: A88DFC2DBE9: to=<redacted@aol.com>, relay=mailin-01.mx.aol.com[64. 12.91.195]:25, delay=0.18, delays=0.01/0/0.14/0.03, dsn=5.1.1, status=bounced (host mailin-01.mx.aol.com[64.12.91. 195] said: 550 5.1.1 <redacted@aol.com>: Recipient address rejected: aol.com (in reply to RCPT TO command))
The issue was brought to AOLs attention and things were fixed rapidly after that. An AOL representative has stated that these were invalid replies and that addresses do not need to be removed from future emails.
Most of the ESPs are aware of this and are working to restore any bounced addresses to their users. At some places this requires manual intervention, so it’s taking some time to get all the addresses restored.
This is one of the reasons that our best bounce handling recommendations are not to remove an address for a single bounce – sometimes the ISPs have technical problems. Like the time a routing failure meant a major ISPs MX machines couldn’t reach their authentication servers to get the list of active users. Or the time all an ISPs MXs were removed from DNS. A lot of the internet is still managed manually, and despite extensive safeguards put in place bad things can, and do, still happen. Usually these problems are resolved quickly and mail starts flowing again.
Morning advice: Do not deactivate addresses that bounced at AOL last night.

Bounce handling is hard

Sometimes I find it hard to find a new topic to write about. I decide I’m going to write about X and then realize I did, often more than once. Other times I think I can blog about some issue only to realize that it’s too complex to handle in a quick post. There are concepts or issues that need background or I have to work a little harder to explain them.
One thing I haven’t blogged about before is bounce handling. That particular topic falls into the other category of posts that take a lot of time to write and need a significant amount of work to make sense. I was even joking with my fellow panel members at EEC a few months ago about how that’s a post that so needs to be written but I’m avoiding it because it’s so hard. There’s so much to be conceptualized and explained and I realize it’s not a blog post but multiple blog posts, or a white paper or even a book.

So let’s start with some simple definitions. Those of you who work at ISPs are probably thinking of bounces in terms of accept than reject, that’s not exactly what I’m talking about here. I’m writing these for senders, who usually call rejects during the SMTP transaction bounces.

April 2015: The Month in Email

We started the month with some conversations about best practices, both generally looking at the sort of best practices people follow (or don’t) as well as some specific practices we wanted to look at in more depth. Three for this month:

What to do when an important email bounces

Some emails are more important than others. I know, I know, all emails are important, but really, some are more important than others.
I’ve recently been decluttering by the simple expedient of enrolling in paperless statements for some of our accounts. We have a 1TB NAS, I’m not going to run out of storage space and I will have so much less paper to deal with. Plus, electronic searches are easier than digging through a file I’ve just shoved statements in for the whole year.
Some companies just let you sign up for statements online and don’t take any extra steps to verify your email address or tell you what happens if your email breaks. But at least one company has gone the extra mile to establish how they handle email bounces.
consentforpaperlessdocs
First, to sign up for paperless notifications I have to give my consent to receive docs. Even better, when I look at the important information it expressly details what happens if my email address bounces.

Three things marketers should do when domains are retired

A few weeks ago I was alerted to a domain change for INGDirect. The ingdirect.com domain is being retired and all users are migrating to the capitalone.com domain. As part of this change usernames are NOT being transferred, so if you have @ingdirect.com addresses on any B2B mailing list, you will need to drop those addresses and find the new contact information for the subscriber.

Yahoo DMARC articles worth reading

There are a bunch of them and they’re all worth reading.
I have more to say about DMARC, both in terms of advice for senders and list managers affected by this, and in terms of the broader implications of this policy decision. But those articles are going to take me a little longer to write.
How widespread is the problem? Andrew Barrett publishes numbers, pulled from his employer, related to the number of senders using @yahoo.com addresses in their commercial emails. Short version: a low percentage but a lot of users and emails in raw numbers.
What can mailing list managers do? Right now the two answers seem to be stop Yahoo.com addresses from posting or fix your mailing list software. Al has posted how he patched his software to cope, and linked to a post by OnlineGroups.net about how they patched their software.
A number of people are recommending adding an Original Authentication Results header as recommended in the DMARC.org FAQ. I’m looking for more information about how that would work.
For commercial mailers, there doesn’t seem to be that much to do except to not use @yahoo.com address as your header-From address. Yes, this may affect delivery while you’re switching to the new From address, but right now your mail isn’t going to any mailbox provider that implements DMARC checking.
One other thing that commercial mailers and ESPs should be aware of. Depending on your bounce handling processes, this may cause other addresses to bounce off the list. Once the issue of the header-From address is settled, you can reactivate addresses that bounced off the list due to authentication failures since April 4.

Thoughts on bounce handling

This week’s Wednesday question comes from D.

What are your thoughts on bounce handling
Read More

8 things that make your mail look like spam

In the comments of last week’s Wednesday question John B. asked

Retrying mail to AOL

I’m working on stuff for MAAWG so I’m really not all that up on what’s happening in the world of email recently. A lot of folks are commenting on my AOL post, and I’m hearing that queues are backing up and emptying as AOL makes changes.
One thing people have been asking me is if they should retry mail to the addresses that are bouncing. I say yes, absolutely. Some of the error messages are related to real filters, but there seems to be quite a bit of slop in the filters these days. I think, though, that the recipients do exist and removing the addresses from future mailings is premature.

Bounces, complaints and metrics

In the email delivery space there are a lot of numbers we talk about including bounce rates, complaint rates, acceptance rates and inbox delivery rates. These are all good numbers to tell us about a particular campaign or mailing list. Usually these metrics all track together. Low bounce rates and low complaint rates correlate with high delivery rates and high inbox placement.

Bounce handling simplified

I am a strong believer that bounce handling should be designed to remove addresses that have no human on the other end while not removing addresses that have a real recipient on the other end.
Bounce handling should be designed to appropriately manage your subscriber base. Delivery problems are the consequence if you don’t do that. They shouldn’t be the reason you bounce handle, though.
Context matters.
My experience tells me that senders that think about the impact of their sends can do things that “break the rules” while still being respectful of their subscribers and still see good delivery.

Don't think bounce handling is important?

James from Cloudmark has his own insight on spamtraps.

Just give it up already

I have a mail system totally separate from my inbox to use when I’m testing signup forms. Some of them are client, some of them are vendors my clients are thinking about using. In any case, it’s mail I’m seriously concerned won’t stop just by me opting out of it.
The server hosting that mail system has been flakey lately, and needs to be hard power cycled to make it come back. We had a major power glitch this morning and so ended up down at the colo and power cycled that box while we were there.
This box was last working February 4th. It’s been off the internet for almost 2 months now. It wasn’t answering on port 25. It was dead. No mail here. And, yet, a bunch of legitimate email marketers are still attempting to send those addresses mail.
Really. Dead for 2 months and the senders keep trying to mail to those addresses. The server came back about 2 1/2 hours ago. I already have 6 emails from two different senders.
Seriously. If you can’t deliver a mail to someone for TWO MONTHS just give it up already. I am sad that even companies that get the best advice I can give them still can’t get the simple things right.
And, really, don’t argue “but it came back! Clearly we should keep trying!” Yes, it came back. But in all the years I’ve had this disposable email system I have not opened a single image. I’ve not purchased a single thing. I’ve never shown any sign of life on any of those addresses. The mailserver has been down for months at a time. There is no value to continuing to send mail to those addresses. And, yet, people still do it.
Why? WHY!?

AOL transmitting 4xx error for user unknown

AOL is currently returning “451 4.3.0 <invaliduser@aol.com>: Temporary lookup failure” in some cases when they really mean “550 user unknown.” This message from AOL should be treated as 5xx failure and the message should not be retried (if at all possible) and the failure should be counted as a hard bounce for list management purposes.
This is something broken at AOL’s end, and the guys with the magic fingers that keep the system running are working to fix it. Right now there doesn’t seem to be an ETA on a fix, though.
Even if you are a sender who is able to stop the retries, you may see some congestion and delays when sending to AOL for the time being. Senders who don’t get the message, or who are unable to stop their MTAs from retrying 4xx mail will continue to attempt delivery of these messages until their servers time out. This may cause congestion for everyone and a noticeable slowdown on the AOL MTAs.
AOL blog post on the issue
HT: Annalivia

20% of email doesn't make it to the inbox

Return Path released their global delivery report for the second half of 2009. To put together the report, they look at mail delivery to the Mailbox Monitor accounts at 131 different ISPs for 600,000+ sends. In the US, 20% of the email sent by Mailbox Monitor customers to Return Path seed accounts doesn’t make it to the inbox. In fact, 16% of the email just disappears.
I’ve blogged in the past about previous Return Path deliverability studies. The recommendations and comments in those previous posts still apply. Senders must pay attention to engagement, permission, complaints and other policy issues. But none of those things really explain why email is missing.
Why is so much mail disappearing? It doesn’t match with the philosophy of the ISPs. Most ISPs do their best to deliver email that they accept and I don’t really expect that ISPs are starting to hard block so many Return Path customers in the middle of a send. The real clue came looking at the Yahoo numbers. Yahoo is one of those ISPs that does not delete mail they have accepted, but does slow down senders. Other ISPs are following Yahoo’s lead and using temporary failures as a way to regulate and limit email sent by senders with poor to inadequate reputations. They aren’t blocking the senders outright, but they are issuing lots of 4xx “come back later” messages.
What is supposed to happen when an ISP issues a 4xx message during the SMTP transaction is that email should be queued and retried. Modern bulk MTAs (MessageSystems, Port25, Strongmail) allow senders to fine tune bounce handling, and designate how many times an email is retried, even allowing no retries on a temporary failure.
What if the missing mail is a result of senders aggressively handling 4xx messages? Some of the companies I’ve consulted for delete email addresses from mailing lists after 2 or 3 4xx responses. Other companies only retry for 12 – 24 hours and then the email is treated as hard bounced.
Return Path is reporting this as a delivery failure, and the tone of discussion I’m seeing seems to be blaming ISPs for overly aggressive spamfiltering. I don’t really think it’s entirely an ISP problem, though. I think it is indicative of poor practices on the part of senders. Not just the obvious permission and engagement issues that many senders deal with, but also poor policy on handling bounces. Perhaps the policy is fine, but the implementation doesn’t reflect the stated policy. Maybe they’re relying on defaults from their MTA vendor.
In any case, this is yet another example of how senders are in control of their delivery problems. Better bounce handling for temporary failures would lower the amount of email that never makes it to the ISP. This isn’t sufficient for 100% inbox placement, but if the email is never handed off to the ISP it is impossible for that email to make it to the inbox.