Asynchronous Bounces

There are three ways that an email can fail to be delivered:

  1. immediate rejection
  2. timeout
  3. asynchronous bounce

Rejection
A rejection is any delivery attempt where the sending smarthost can tell immediately that the mail can’t be delivered.
That will often be when the receiving machine accepts a connection but returns a “hard bounce” or “5xx” error at some point in the transaction, but can also be caused by the email address being malformed, the recipients domain not existing or the domain’s DNS being configured (accidentally or intentionally) to deny any email.
Traditionally this rejection will be recorded by the smarthost generating an asynchronous bounce email locally and sending that back to the local user who sent the message – that’s usually where the bounce message you see when you typo a recipient comes from. That’s an inefficient way of handling rejections at volume, though, so smarthost designed for high-volume email will typically report rejections some other way rather than creating asynchronous bounce.
Timeout
If something fails during an attempt to deliver a message, but the sending smarthost either can tell that it’s just a temporary problem, or it can’t tell for sure that it’s a permanent problem then the smarthost will keep a copy of that outbound message in it’s spool and schedule an attempt to resend it later.
One common cause of this is a “soft bounce” or “4xx” error at some point in the transaction, but broken DNS records, overloaded or unavailable servers, network problems or firewall level filtering can also cause delivery to fail in a potentially “temporary” way.
Some of these “temporary” problems aren’t really temporary, and the scheduled attempts to resend the mail will also fail. Eventually the smarthost will have give up on the delivery attempt and report a failure. That can take a long time – while the smarthost owner can configure how long it should keep trying it will often default to a week or so. Once the smarthost gives up it will report the failure, often by generating an asynchronous bounce email locally.
Asynchronous bounce
If the receiving mailserver accepts the message it is accepting responsibility for delivering it to the final recipient. If it finds out afterwards that it can’t deliver the message it sends a notification to the sender of the message, typically with more information about why delivery failed and a partial or complete copy of the message.
Like this:

Hi. This is the qmail-send program at yahoo.com.
I’m afraid I wasn’t able to deliver your message to the following addresses.
This is a permanent error; I’ve given up. Sorry it didn’t work out.
<cacakande@gmail.comx>:
74.125.142.27 failed after I sent the message.
Remote host said: 550-5.7.1 [98.138.207.123      12] Our system has detected that this message is
550-5.7.1 likely unsolicited mail. To reduce the amount of spam sent to Gmail,
550-5.7.1 this message has been blocked. Please visit
550-5.7.1 http://support.google.com/mail/bin/answer.py?hl=en&answer=188131 for
550 5.7.1 more information. g2si7192949igq.45 – gsmtp
— Below this line is a copy of the message.

Or this:

Delivery to the following recipient failed permanently:
nev.w.goodman@gone.bristol.ac.ukx
Technical details of permanent failure:
Google tried to deliver your message, but it was rejected by the server for the recipient domain gone.bristol.ac.uk byaspmx.l.google.com. [173.194.67.27].
The error that the other server returned was:
550-5.1.1 The email account that you tried to reach does not exist. Please try
550-5.1.1 double-checking the recipient’s email address for typos or
550-5.1.1 unnecessary spaces. Learn more at
550 5.1.1 http://support.google.com/mail/bin/answer.py?answer=6596 k3si5496468wjb.158 – gsmtp
—– Original message —–

That’s an asynchronous bounce – “asynchronous” because it’s not synchronized with the delivery of the original message; it may come at any time after the delivery appeared to succeed, even weeks later.
There are quite a few operational problems with asynchronous bounces. One of the bigger ones is what’s known as backscatter. The mailserver generating the asynchronous bounce will send it to the email address in the bouncing email’s “envelope from” / “bounce address” – and when the mail being bounced is spam that email address is usually fake or an innocent third party. Several hundred backscatter bounces showed up in my inbox this morning, so it’s a bit of a sore point right now.
Because of the backscatter issue it’s considered a best practice to do as much filtering and other delivery decision making as possible while you have the sending mailserver still connected, so that you can safely reject the mail. Only as a last resort when you find out later that the message can’t be delivered should you send an asynchronous bounce.
Some people go further and suggest that any competently run mailserver should never send asynchronous bounces, and that there is never any operational need for them. I think they’re wrong, and judging from the bounces in my inbox, so do Google, Yahoo, Microsoft, Telstra, Eircom, Orange, OCN, t-online.de, Interhost.it, xs4all.nl and many, many other ISPs, companies and universities across the world…

Related Posts

CASL is more privacy law than anti-spam law

Michael Geist, a law professor in Canada, writes about the new CASL law, why it’s necessary and why it’s more about privacy and consumer protection than just about spam.

Read More

July 2014: The month in email

We continue to be busy with really interesting client work. Look for some new posts and white papers to come out of this research over the next few months, but for now blogging has been a bit light while we’re working hard. In parallel with our busy times, we have also been pondering the ways in which the email world illustrates the classic bon mot  “plus ça change, plus c’est la même chose”, and we’ve been revisiting some posts from a few years ago to examine this.
We started July with a nod to a good subscription experience just as CASL, the Canadian Anti-Spam Legislation went into effect on Canada Day. While companies have another 17 months to put these provisions into practice, it’s a good reminder that periodic re-engagement with customers can be very effective in helping you maintain high-quality subscriber lists. We talked a bit more about CASL here and what protections the law intends.
In stark contrast, we posted about an organization that is doing a less-than-stellar job making sure they’re only sending wanted email. The Direct Marketing Association is a terrific resource and member organization for marketers across industries and channels, but their email marketing practices don’t always live up to their mission of “Advancing and Protecting Responsible Data-Driven Marketing”, and we explored some ways in which they might improve this.
Those of you who have been reading this blog for any time at all know that we tend to talk about wanted mail and unwanted mail rather than the more general category of spam. Marketers tend to think their mail can’t possibly be spam if it’s not offering Viagra or phishing for credit card information, but that’s not really the point — if a customer doesn’t want to read your email about new mountain bikes, even if they bought a mountain bike from you three years ago, that’s unwanted email. Here’s a post we revisited about why customers might not want your mail, and a new post about engagement.
One risk of sending unwanted email, of course, is that customers complain, and that will affect your delivery going forward. We revisited a post about feedback loops, and also talked a bit about addressing delivery problems as they come up rather than waiting for them to resolve on their own (mostly, they won’t!)
I also proposed a bit of a thought experiment around monetizing the complaint stream, and followed up with a second post. There are some good points in the comments of those posts, but mostly I think it’s an interesting solution to addressing risk and abuse at ESPs.
Finally, Steve wrote a short post about our new mail servers and how quickly spammers descended as we set those up. It’s a constant battle!

Read More

Ever changing filtering

One of the ongoing challenges sending email, and managing a high volume outbound mail server is dealing with the ongoing changes in filtering. Filters are not static, nor can they be. As ISPs and filtering companies identify new ways to separate out wanted email from unwanted email, spammers find new ways to make their mail look more like wanted mail.
This is one reason traps are useful to filtering companies. With traps there is no discussion about whether or not the mail was requested. No one with any connection to the email address opted in to receive mail. The mail was never requested. While it is possible for trap addresses to get on any list monitoring mail to spam traps is a way to monitor which senders don’t have good practices.
New filtering techniques are always evolving. I mentioned yesterday that Gmail was making filtering changes, and that this was causing a lot of delivery issues for senders. The other major challenge for Gmail is the personalized delivery they are doing. It’s harder and harder for senders to monitor their inbox delivery because almost every inbox is different at Gmail. I’ve seen different delivery in some of my own mailboxes at Gmail.
All of this makes email delivery an ongoing challenge.

Read More