It should be easy, right? Except it’s not. So why is it so hard?
With one-on-one or one-to-few email it’s pretty simple. The rejections typically go back to a human who reads the text part of the rejection message and adapt and makes the decision about future messages. The software handles what to do with the undeliverable message based on the SMTP response code.
In the case of a 5xy response the server stops attempting delivery and alerts the original sender the mail failed. One example from helping a client troubleshoot a delivery problem recently.
There’s useful information in the text portion of this email from my mail server. It says there was a permanent failure (550) and that my message won’t be delivered. It also says the email is quarantined in reply to the end of DATA. That’s actually a critical piece of information. It means Barracuda saw the entire message before deciding to reject it. It’s likely a problem with the content of the email and so I need to look at links in the message.
This type of plain text explanation is great for a human to read and act on. But it’s not that simple for list handling software to identify the relevant information in the text message and act on future emails to that recipient. Different MTA vendors and ESPs have done a lot of work to try and correctly parse bounce messages to pull out relevant information.
ISPs have tried to help the situation by giving more descriptive rejection messages. They’re still using the SMTP required 3 digit numbers, but they include short, parseable codes in the text portion of the message. In many cases they also include URLs and links that open up webpages explaining the meaning of the code. They even post a list of the most common codes on their postmaster webpages.
All of these things make it somewhat easier to handle bounces automatically. Kinda.
I’ve been working on some bounce handling recommendations for a client using a few different ESPs. I spent a good few days digging into the bounces returned by their different ESPs. It was an interesting exercise as it demonstrated how very differently ESPs handle bounces. But it also clarified for me that there are a lot of different kinds of bounces.
- Address related bounces: user unknown, mailbox unavailable, invalid address, bad address syntax.
- Network related bounces: DNS failures, server too busy, temporarily unavailable, network error, connection timed out.
- Reputation related bounces: poor IP reputation, IP blocked, content blocked, too many complaints, prohibited attachment.
- Bad sending configuration: relaying denied, message too large,
- Authentication failure: DMARC
There’s also an overlap of Network and Reputation bounces: too many connections and too many messages are network methods to slow down sending of problematic mail.
None of these are always hard or always soft bounces. There are multiple ESPs that have also introduced the concept of “spam” bounce, which are an attempt to separate out some of the bounces due to problematic content from bounces labeled “hard” or “soft.”
I still hold out hope that someone, somewhere will come up with the grand unified theory of bounce handling. I suspect I’ll get a pony before that happens, though. I’m starting to get a better handle, though, on how to explain bounce handing – both in helping people visualize the different categories and clarifying what should be done with different messages.
RSS - Posts
I’ve sometimes thought that the bounce message should include an SNMP-style OID identifying the cause in machine-friendly format (with relevant strings inserted into the OID). It would also be possible to convert that to localized text.
What’s also frustrating is that some ESPs miscategorize their bounces. Hard bounces are often treated as soft and vice versa. I’ve found multiple instances this of this with Sendgrid and Mandrill.
I’m not sure they miscategorize them as much as there aren’t clearly defined categories. I have a image I put together for a client grouping the bounces from the different ESPs they use. I’m working on cleaning it up and sharing it more formally.
This isn’t a space where there are any specific right answers, there’s only the answers we think are right.
To be honest, I really don’t think technically there are “soft” and “hard” bounces. Either a message gets delivered or it doesn’t. If it does get delivered, and the email server bounces the message back (back scatter) then that is a bounce. Neither hard nor soft. I think a bunch of marketing noobs made up this “soft” and “hard” reference.
It’s not made up. The specification defines 4xx failures as temporary (soft) and 5xx failures as permanent (hard). “Mailbox full” is temporary, “user does not exist” is permanent (kinda). The problem is that some soft bounces are actually pretty hard, and some hard bounces are soft. The short answer, as it is in so many other places, is “it depends”.
a 4xx doesn’t “bounce” anything. That is during the smtp conversation. Nothing got bounced. Deferred, yes. No mail got delivered to either party.
Read the Wikipedia for non technical email delivery on bounces. That too never mentions anything about hard or soft. https://en.wikipedia.org/wiki/Bounce_message
I don’t need a lesson in ‘non technical email delivery’. I was postmaster@ a site with a quarter of a million users, and I know a few things about email already,
One of the things that I know is that, in the context of email senders talking about hard and soft bounces, they are talking about 5xx and 4xx SMTP response codes. If you lack this context, I can see how you’d misunderstand.
I understand people call bounces hard/soft. I sleep okay knowing that. In your explanation you mention response codes. I am suggestion that they are just that, “response codes” or “non-delivery notification messages” and those response codes describe generally; Deferrals, Failures, and Successes. Only a success can become a bounce. As I was taught, when a mail system accepts the email with a response code (non-delivery notification message) of a 2xx and then at a later time creates a new SMTP connection session following the MAIL FROM from the original SMTP conversation, then that is a “bounce” neither hard nor soft. In the email engineering classes I took I had some pretty smart/nutty “Don Quixote” type teachers. In these cases we don’t have soft or hard bounces. We just have bounces. I didn’t mean to imply any lesson nor wish for you to take this the wrong way, which would be completely inappropriate.