Undelivered mail without a bounce

A few weeks ago I wrote a blog post focusing on one small part of bounce handling. Today I want to talk about delivery failures that aren’t bounces. This is really the biggest issue for companies who have written their own bulk sending servers. Modern bulk MTA appliances and ESPs correctly handle these types of bounces. However, when you’re troubleshooting it’s important to know that sometimes there won’t be a SMTP level bounce.

When we talk about bounce handling, we’re usually talking about what happens during (or after) the SMTP transaction. That’s built into our terminology. User unknown, 5xy, 550, 4xx is all shorthand we’ve taken from the email protocol documents. There are some types of delivery failures that happen before a SMTP transaction happen. And, like the SMTP failures, they can be permanent or temporary.

Domain not found in DNS

The first thing an outgoing mail server does to send mail is look up the mail exchange (MX) record for the receiving domain. There are a few different responses a DNS server can return. The expected response is a domain name that is actively configured to receive mail for that system. The sending server can move on and attempt to deliver the mail.

When there is no MX configured there are a couple different responses returned. All of them contain a blank MX record, but they also include a status message. If a domain exists, but just doesn’t have a MX configured the DNS server returns a NOERROR status. If a domain or subdomain doesn’t exist in DNS the DNS server returns NXDOMAIN. For NXDOMAIN the email address should be marked as undeliverable and removed from future sends.

What to do in the case of NOERROR isn’t as clear cut. It is a perfectly valid configuration to have no MX published in DNS but to accept mail. What happens is a DNS lookup is performed for the A (or AAAA) record of the domain. If there is an A record then the mail server attempts to deliver to port 25 at that IP address. In the vast majority of cases there is a MX record and any domain publishing a blank MX is not accepting mail. This is even more true when we’re looking at the consumer email domains. However, it’s not 100% guaranteed that a domain will publish a MX record. Overall, I think that if there is a blank MX then mail to that domain should be suppressed as a domain with no email users. This recommendation may result in a very, very tiny fraction of email to hobby and personal domains being suppressed. But I think that overall, it’s better to suppress than attempt to deliver to domains that may not actually want mail.

Sometimes a DNS server will return a SERVFAIL response. In this case, current delivery will fail but the address can be retried in the future.

DotMX or localhost in DNS

There are a couple things that a domain owner does to identify a domain that does not accept email. One of those is using a single . (yes, it’s really just a dot) in the MX record (RFC7570). This is a clear statement by the domain owner that this domain is never used for email. Some domain owners put in a record of 127.0.0.1 or localhost. In both cases, these responses mean mail will never deliver and all future mail to that address should be suppressed.

No server at the MX

Sometimes the domain does respond with a MX but there is no mail server there. Behind the scenes, the sending mail server attempts to connect on port 25 but the connection attempt times out because there is no answer. This situation isn’t as clear cut as a NXDOMAIN or a dotmx. There are cases where a server might not answer temporarily. Ideally, the MTA will requeue the mail and attempt to send it over a few days. If the failure continues over a long period of time then it’s likely this is a dead domain and future mailings to the domain should be suppressed.

Overall, these bounces are quietly handled by competent ESPs and modern bulk MTAs. But they do happen. It’s not a spam issue, but it is a data hygiene issue. Excessive numbers of people submitting addresses with no MX, or a dotMX or no server at that domain mean that they’re also likely submitting addresses that belong to unsuspecting third parties. Understanding what about the collection process is encouraging forgeries should result in cleaner deliverable data.

 

 

Related Posts

The Internet is hard.

There are so many things that need to happen to make the Internet work. DNS entries need to be right. MXs need to be set up. Web servers need to be configured. And, let’s be honest, anyone who has ever run their own services on the Internet has flubbed a configuration.
We don’t think about it, because most of the time the configurations are handled by scripts and they do things right. But at some point someone needs to type in something and there’s a risk it will go horribly wrong. I’ve been digging into domain data for a client of mine today. I think I’m going cross-eyed over it. But I have found so many weird things that just mean someone isn’t paying attention to what they’re doing.
Like the domain that has a MX record that says:
nullmx
 
I’m pretty sure the intention of the domain owner is to publish a null MX. But they added an extraneous “0” in there and ended up publishing something really weird. Even worse, the MTA that this client is using is listing this as a “delivered” email. I’m pretty sure that mail to that domain never left the MTA.
I’ve found horribly typoed MX domains for popular spam filters. I’ve found domains that have invalid characters in them. I’ve found domains that are totally a mess.
The vast majority of us have some story or other of the time we really broke things by accident. Like the time a very large ISP deleted their MX records. Or when a different ISP changed their internal forwarding and broke SPF authentication for everyone mailing that domain. Or when another ISP accidentally blocked every IP beginning with 6.
Sometimes I’m amazed that the Internet ever works. No matter how big it gets, there are actual people writing actual code and configurations. The number of things that have to happen to get packets from A to B is pretty impressive. We rarely ever notice the breakages, the people who run things are really good at their jobs. But sometimes poking in the grotty corners reminds me how easy it is to break things. It’s sometimes a wonder things actually work.
 

Read More

TLS certificates and CAA records

Transport Layer Security (TLS) is what gives you the little padlock in your browser bar. Some people still call it SSL, but TLS has been around for 18 years –  it’s time to move on.
TLS provides two things. One is encryption of traffic as it goes across the wire, the other is a cryptographic proof that you’re talking to the domain you think you’re talking to.
The second bit is important, as if you can’t prove you’re talking to, for example, your bank you could really be talking to a malicious third party who has convinced your browser to talk to their server instead of your bank which makes the encryption of the traffic much less useful. They could even act as a man-in-the-middle and pass your traffic through to your bank, so that you wouldn’t notice anything wrong.

When your browser connects to a website over TLS it, as part of setting up the connection, fetches a “TLS certificate” from the server. That certificate includes the hostname of the server, so the browser can be sure that it’s talking to the server it thinks it is.
How does the browser know to trust the certificate, though? There’s not really a great way to do that, yet. There’s a protocol called DANE that stores information in DNS to validate the certificate, much the same as we do with DKIM. It’s a promising approach, but not widely supported.
What we have today are “Certificate Authorities” (CAs). These are companies that will confirm that you own a domain, issue you a certificate for that domain where they vouch for it’s authenticity (and usually charge you for the privilege). Anyone can set themselves up as a CA (really – it’s pretty trivial, and you can download scripts to do the hard stuff), but web browsers keep a list of “trusted” CAs, and only certificates from those authorities count. Checking my mac, I see 169 trusted root certificate authorities in the pre-installed list. Many of those root certificates “cross-sign” with other certificate authorities, so the actual number of companies who are trusted to issue TLS Certificates is much, much higher.
If any of those trusted CAs issue a certificate for your domain name to someone, they can pretend to be you, secure connection padlock and all.
Some of those trusted CAs are trustworthy, honest and competent. Others aren’t. If a CA is persistently, provably dishonest enough then they may, eventually, be removed from the list of trusted Certificate Authorities, as StartCom and WoSign were last year. More often, they don’t: Trustwave, MCS Holdings/CNNIC, ANSSI, National Informatics Center of India (who are currently operating a large spam operation, so …).
In 2011 attackers compromised a Dutch CA, DigiNotar, and issued themselves TLS Certificates for over 500 high-profile domains – Skype, Mozilla, Microsoft, Gmail, … – and used them as part of man-in-the-middle attacks to compromise hundreds of thousands of users in Iran. Coverage at the time blamed it on “DigiNotar’s shocking ineptness“.
In 2015 Symantec/Thawte issued 30,000 certificates without authorization of the domain owners, and even when they issued extended validation (“green bar”) certificates for “google.com” they weren’t removed from the trusted list.
So many CAs are incompetent, many are dishonest, and any of them can issue a certificate for your domain. Even if you choose to use a competent, reputable CA – something that’s not trivial in itself – that doesn’t stop an attacker getting certificates for your domain from somewhere else.
This is where CAA DNS records come in. They’re really simple and easy to explain, with no fancy crypto needed to set them up. If I publish this DNS record …

Read More

About that DMARC "exploit"

A security researcher has identified a rendering flaw that allows for “perfect” phishing emails. From his website:

Read More