While discussing a draft of a Deliverability BCP document the issue came up of what rejections at different phases of the email delivery transaction can mean. That’s quite a big subject, but here’s a quick cheat sheet.
At initial connection
Dropped or failed connection:
- your reputation with the receiver is so bad that they don’t want to see any email from you, ever
- their mail server is badly overloaded and can’t accept any connections right now
- the server you’re connecting to isn’t a mail server – this is common with bad email addresses
Temporary (4xx) failure:
- your reputation with the receiver is poor or unknown and they’re waiting for more information before they’ll accept your mail
- the mail server is overloaded and can’t accept any new mail right now
- your reputation is “grey”, the mail server is overloaded and it’s accepting mail preferentially from other senders right now
- there is something technically wrong with your mailserver, such as non-existent reverse DNS, and your mail won’t be accepted until you fix that
Permanent (5xx) failure:
- a 554 response can mean “there’s no mailserver here”. If someone has set up a mailserver specifically to reject connections this way, it’s probably been done for a domain that has a lot of bad delivery attempts. That might mean that the recipient email address is one that’s commonly used for fake signups, or it may be a common thinko address (e.g. something.com vs something.net)
- they’ve explicitly blocked all mail from your mailserver due to a poor reputation – this is a common way of implementing DNS blacklist based blocking
- when a mailserver is shut down, it may drop existing sessions with a 421 response
- if you violate SMTP – send a bad command, bad parameters, lines too long – you might get 5xx responses at any time
After HELO or EHLO
- a syntactically bad or obviously forged (e.g. yahoo.com) HELO hostname – unless you’re a spammer or have a badly misconfigured server, this probably shouldn’t happen
After MAIL FROM
- mail from a domain (or, in theory, return path) might be blocked here, but it’s more likely to be rejected at RCPT TO time
After RCPT TO
This is a favorite point to apply reputation based filtering, for several reasons. It’s the first point at which you know who the intended recipient is, and that’s a very useful thing to record to help with diagnosing spam filtering false positives. So even the mailserver has decided it’s going to reject a message, based on the IP address it’s being sent from, waiting until the RCPT TO to reject if will give better diagnostics. There may be special recipient addresses that are exempt from global filtering (such as role addresses), so that filtering can’t be applied until this step. Also, some mailing list managers historically don’t handle rejections before RCPT TO gracefully – they’ll keep retrying and won’t unsubscribe the user forever.
If a mailserver waits until after this point it either has to accept the entire email or unceremoniously crash the connection (which violates SMTP and will provoke a redelivery attempt, so is rarely seen).
Temporary (4xx) failure:
- the server is out of resources, and just needs you to come back later
- the server is low on resources and mail from you isn’t as important as mail from other senders right now,
- the overall volume of mail you’re sending has exceeded short term limits the receiver is applying to you (based on reputation) and it’s deferring mail so as to throttle your traffic
- there’s a temporary problem (DNS blacklist resolution, perhaps) in looking up your reputation
- your reputation is “grey”, the mailserver hasn’t decided whether your mail is likely to be wanted or not, and it’s letting a small fraction of your mail through, to see how recipients respond
- your reputation is “grey” or there isn’t enough information about you to make a delivery decision, so your mail is being deferred in the hope that there’ll be a clearer reputation decision when you try to redeliver it
- some combination of all the above
Permanent (5xx) failure:
- the recipient doesn’t exist – sometimes a bad address is just a bad address
- the source (IP address or domain) of the email has a poor reputation and mail is blocked
- there’s a per-recipient block on this email
- the content of the message “looks like spam” – either because it looks similar to recently received unwanted messages or based on heuristics such as those used by spamassassin
- URLs in the message have a poor reputation (or resolve to IP addresses with poor reputation)
- DKIM signatures will be checked at this point, so if the mail would have been delivered due to DKIM-based whitelisting and the DKIM signature doesn’t validate that may cause rejection here
- DMARC failures
- virus or malicious content filters
- any of the reasons given for RCPT TO – some filters may defer any rejection until after data, either so as to be able to analyze the content of rejected messages, or because they use a simple architecture that does all it’s filtering at once,
- score-based filtering – based on combining everything the mailserver knows about this message, including the content
There’s probably something I’ve missed – call it out in the comments – but these are some of the things I think about when working out why a message has been rejected with an unhelpful error message.
Are you aware of any cases where the drop is *during* DATA?
Responding to the DATA command with a 503 or 554 permanent error is valid SMTP, but it’s not a sensible place to do spam filtering (except in some theoretical cases where you’re making decisions based on all the attempted recipients) as you don’t have access to any additional information above that you had at RCPT TO. If you do see a rejection there it’s likely because you ignored rejections in response to RCPT TO.
The connection can unceremoniously drop at any time – network problems, server being rebooted, things like that. But when that happens the clients will treat that as a temporary failure and attempt redelivery. It’s outside the SMTP spec, so it doesn’t allow you to give additional information to the client, nor to influence what it’ll do next.
Some people have suggested using content-filtering to abruptly drop a connection during delivery of the message payload, and then 554-ing at connection for future connections from the same IP address – but it’s not a great idea, and I don’t know of it being deployed in production.
Network level filtering pays little attention to the protocol it’s carrying, of course. If you’re getting your connections blocked or throttle at a firewall or router level, that’s going to just look like you lost connection.
So … sure, I’ve seen it happen, but it’s not something I would expect to be intentional filtering; rather I’d assume it to be just a system or network outage. SMTP is a very robust protocol, and handles that sort of problem all the time without anyone noticing unless they’re paying close attention.
[…] bounce può avvenire immediatamente durante la transazione, in diversi punti come spiegato qua, oppure successivamente (e in questo caso si parla di bounce asincrono) rendendo ancora più […]