BLOG

Why is bounce handling so hard

It should be easy, right? Except it’s not. So why is it so hard?

With one-on-one or one-to-few email it’s pretty simple. The rejections typically go back to a human who reads the text part of the rejection message and adapt and makes the decision about future messages. The software handles what to do with the undeliverable message based on the SMTP response code.

In the case of a 5xy response the server stops attempting delivery and alerts the original sender the mail failed. One example from helping a client troubleshoot a delivery problem recently.

There’s useful information in the text portion of this email from my mail server. It says there was a permanent failure (550) and that my message won’t be delivered. It also says the email is quarantined in reply to the end of DATA. That’s actually a critical piece of information. It means Barracuda saw the entire message before deciding to reject it. It’s likely a problem with the content of the email and so I need to look at links in the message.

This type of plain text explanation is great for a human to read and act on. But it’s not that simple for list handling software to identify the relevant information in the text message and act on future emails to that recipient. Different MTA vendors and ESPs have done a lot of work to try and correctly parse bounce messages to pull out relevant information.

ISPs have tried to help the situation by giving more descriptive rejection messages. They’re still using the SMTP required 3 digit numbers, but they include short, parseable codes in the text portion of the message. In many cases they also include URLs and links that open up webpages explaining the meaning of the code. They even post a list of the most common codes on their postmaster webpages.

All of these things make it somewhat easier to handle bounces automatically. Kinda.

I’ve been working on some bounce handling recommendations for a client using a few different ESPs. I spent a good few days digging into the bounces returned by their different ESPs. It was an interesting exercise as it demonstrated how very differently ESPs handle bounces. But it also clarified for me that there are a lot of different kinds of bounces.

  1. Address related bounces: user unknown, mailbox unavailable, invalid address, bad address syntax.
  2. Network related bounces: DNS failures, server too busy, temporarily unavailable, network error, connection timed out.
  3. Reputation related bounces: poor IP reputation, IP blocked, content blocked, too many complaints, prohibited attachment.
  4. Bad sending configuration: relaying denied, message too large,
  5. Authentication failure: DMARC

There’s also an overlap of Network and Reputation bounces: too many connections and too many messages are network methods to slow down sending of problematic mail.

None of these are always hard or always soft bounces. There are multiple ESPs that have also introduced the concept of “spam” bounce, which are an attempt to separate out some of the bounces due to problematic content from bounces labeled “hard” or “soft.”

I still hold out hope that someone, somewhere will come up with the grand unified theory of bounce handling. I suspect I’ll get a pony before that happens, though. I’m starting to get a better handle, though, on how to explain bounce handing – both in helping people visualize the different categories and clarifying what should be done with different messages.

 

8 comments

  1. Richard says

    I’ve sometimes thought that the bounce message should include an SNMP-style OID identifying the cause in machine-friendly format (with relevant strings inserted into the OID). It would also be possible to convert that to localized text.

  2. Daniel K says

    What’s also frustrating is that some ESPs miscategorize their bounces. Hard bounces are often treated as soft and vice versa. I’ve found multiple instances this of this with Sendgrid and Mandrill.

    1. laura says

      I’m not sure they miscategorize them as much as there aren’t clearly defined categories. I have a image I put together for a client grouping the bounces from the different ESPs they use. I’m working on cleaning it up and sharing it more formally.

      This isn’t a space where there are any specific right answers, there’s only the answers we think are right.

  3. Keith Kouzmanoff says

    To be honest, I really don’t think technically there are “soft” and “hard” bounces. Either a message gets delivered or it doesn’t. If it does get delivered, and the email server bounces the message back (back scatter) then that is a bounce. Neither hard nor soft. I think a bunch of marketing noobs made up this “soft” and “hard” reference.

  4. Huey says

    It’s not made up. The specification defines 4xx failures as temporary (soft) and 5xx failures as permanent (hard). “Mailbox full” is temporary, “user does not exist” is permanent (kinda). The problem is that some soft bounces are actually pretty hard, and some hard bounces are soft. The short answer, as it is in so many other places, is “it depends”.

  5. Keith Kouzmanoff says

    a 4xx doesn’t “bounce” anything. That is during the smtp conversation. Nothing got bounced. Deferred, yes. No mail got delivered to either party.

    Read the Wikipedia for non technical email delivery on bounces. That too never mentions anything about hard or soft. https://en.wikipedia.org/wiki/Bounce_message

  6. Huey says

    I don’t need a lesson in ‘non technical email delivery’. I was postmaster@ a site with a quarter of a million users, and I know a few things about email already,

    One of the things that I know is that, in the context of email senders talking about hard and soft bounces, they are talking about 5xx and 4xx SMTP response codes. If you lack this context, I can see how you’d misunderstand.

  7. Keith Kouzmanoff says

    I understand people call bounces hard/soft. I sleep okay knowing that. In your explanation you mention response codes. I am suggestion that they are just that, “response codes” or “non-delivery notification messages” and those response codes describe generally; Deferrals, Failures, and Successes. Only a success can become a bounce. As I was taught, when a mail system accepts the email with a response code (non-delivery notification message) of a 2xx and then at a later time creates a new SMTP connection session following the MAIL FROM from the original SMTP conversation, then that is a “bounce” neither hard nor soft. In the email engineering classes I took I had some pretty smart/nutty “Don Quixote” type teachers. In these cases we don’t have soft or hard bounces. We just have bounces. I didn’t mean to imply any lesson nor wish for you to take this the wrong way, which would be completely inappropriate.

Comment:

Your email address will not be published. Required fields are marked *

  • OTA joins the ISOC

    The Online Trust Alliance (OTA) announced today they were joining forces with the Internet Society (ISOC). Starting in May, they will operate as an initiative under the ISOC umbrella. “The Internet Society and OTA share the belief that trust is the key issue in defining the future value of the Internet,” said Internet Society President and CEO, Kathryn Brown. “Now is the right time for these two organizations to come together to help build user trust in the Internet. At a time when cyber-attacks and identity theft are on the rise, this partnership will help improve security and data privacy for users,” added Brown.No Comments


  • Friday blogging... or lack of it

    It seems the last few Friday's I've been lax on posting. Some of that is just by Friday I'm frantically trying to complete all my client deliverables before the weekend. The rest of it is by Friday I'm just tired. Today had the added complication of watching the Trumpcare debate and following how (and how soon) it would affect my company if it passed. That's been a bit distracting, along with the other stuff I posted about yesterday. I wish everyone a great weekend.1 Comment


  • Indictments in Yahoo data breach

    Today the US government unsealed an indictment against 2 Russian agents and 2 hackers for breaking into Yahoo's servers and stealing personal information. The information gathered during the hack was used to target government officials, security employees and private individuals. Email is so central to our online identity. Compromise an email account and you can get access to social media, and other accounts. Email is the key to the kingdom.No Comments


Archives