Undelivered mail without a bounce

laura
Faq
February 4, 2019

A few weeks ago I wrote a blog post focusing on one small part of bounce handling. Today I want to talk about delivery failures that aren’t bounces. This is really the biggest issue for companies who have written their own bulk sending servers. Modern bulk MTA appliances and ESPs correctly handle these types of bounces. However, when you’re troubleshooting it’s important to know that sometimes there won’t be a SMTP level bounce.

When we talk about bounce handling, we’re usually talking about what happens during (or after) the SMTP transaction. That’s built into our terminology. User unknown, 5xy, 550, 4xx is all shorthand we’ve taken from the email protocol documents. There are some types of delivery failures that happen before a SMTP transaction happen. And, like the SMTP failures, they can be permanent or temporary.

Domain not found in DNS

The first thing an outgoing mail server does to send mail is look up the mail exchange (MX) record for the receiving domain. There are a few different responses a DNS server can return. The expected response is a domain name that is actively configured to receive mail for that system. The sending server can move on and attempt to deliver the mail.

When there is no MX configured there are a couple different responses returned. All of them contain a blank MX record, but they also include a status message. If a domain exists, but just doesn’t have a MX configured the DNS server returns a NOERROR status. If a domain or subdomain doesn’t exist in DNS the DNS server returns NXDOMAIN. For NXDOMAIN the email address should be marked as undeliverable and removed from future sends.

What to do in the case of NOERROR isn’t as clear cut. It is a perfectly valid configuration to have no MX published in DNS but to accept mail. What happens is a DNS lookup is performed for the A (or AAAA) record of the domain. If there is an A record then the mail server attempts to deliver to port 25 at that IP address. In the vast majority of cases there is a MX record and any domain publishing a blank MX is not accepting mail. This is even more true when we’re looking at the consumer email domains. However, it’s not 100% guaranteed that a domain will publish a MX record. Overall, I think that if there is a blank MX then mail to that domain should be suppressed as a domain with no email users. This recommendation may result in a very, very tiny fraction of email to hobby and personal domains being suppressed. But I think that overall, it’s better to suppress than attempt to deliver to domains that may not actually want mail.

Sometimes a DNS server will return a SERVFAIL response. In this case, current delivery will fail but the address can be retried in the future.

DotMX or localhost in DNS

There are a couple things that a domain owner does to identify a domain that does not accept email. One of those is using a single . (yes, it’s really just a dot) in the MX record (RFC7570). This is a clear statement by the domain owner that this domain is never used for email. Some domain owners put in a record of 127.0.0.1 or localhost. In both cases, these responses mean mail will never deliver and all future mail to that address should be suppressed.

No server at the MX

Sometimes the domain does respond with a MX but there is no mail server there. Behind the scenes, the sending mail server attempts to connect on port 25 but the connection attempt times out because there is no answer. This situation isn’t as clear cut as a NXDOMAIN or a dotmx. There are cases where a server might not answer temporarily. Ideally, the MTA will requeue the mail and attempt to send it over a few days. If the failure continues over a long period of time then it’s likely this is a dead domain and future mailings to the domain should be suppressed.

Overall, these bounces are quietly handled by competent ESPs and modern bulk MTAs. But they do happen. It’s not a spam issue, but it is a data hygiene issue. Excessive numbers of people submitting addresses with no MX, or a dotMX or no server at that domain mean that they’re also likely submitting addresses that belong to unsuspecting third parties. Understanding what about the collection process is encouraging forgeries should result in cleaner deliverable data.

Are they using DKIM?

It’s easy to tell if a domain is using SPF – look up the TXT record for the domain and see if any of them begin with “v=spf1”. If one does, they’re using SPF. If none do, they’re not. (If more than one does? They’re publishing invalid SPF.)
AOL are publishing SPF. Geocities aren’t.
For DKIM it’s harder, as a DKIM key isn’t published at a well-known place in DNS. Instead, each signed email includes a “selector” and you look up a record by combining that selector with the fixed string “._domainkey.” and the domain.
If you have DKIM-signed mail from them then you can find the selector (s=) in the DKIM-Signature header and look up the key. For example, Amazon are using a selector of “taugkdi5ljtmsua4uibbmo5mda3r2q3v”, so I can look up TXT records for “taugkdi5ljtmsua4uibbmo5mda3r2q3v._domainkey.amazon.com“, see that there’s a TXT record returned and know there’s a DKIM key.
That’s a particularly obscure selector, probably one they’re using to track DKIM lookups to the user the mail was sent to, but even if a company is using a selector like “jun2016” you’re unlikely to be able to guess it.
But there’s a detail in the DNS spec that says that if a hostname exists, meaning it’s in DNS, then all the hostnames “above” it in the DNS tree also exist (even if there are no DNS records for them). So if anything,_domainkey.example.com exists in DNS, so does _domainkey.example.com. And, conversely, if _domainkey.example.com doesn’t exist, no subdomain of it exists either.
What does it mean for a hostname to exist in DNS? That’s defined by the two most common responses you get to a DNS query.
One is “NOERROR” – it means that the hostname you asked about exists, even if there are no resource records returned for the particular record type you asked about.
The other is “NXDOMAIN” – it means that the hostname you asked about doesn’t exist, for any record type.
So if you look up _domainkey.aol.com you’ll see a “NOERROR” response, and know that AOL have published DKIM public keys and so are probably using DKIM.
(This is where Steve tries to find a domain that isn’t publishing DKIM keys … Ah! Al’s blog!)
If you look up _domainkey.spamresource.com you’ll see an “NXDOMAIN” response, so you know Al isn’t publishing any DKIM public keys, so isn’t sending any DKIM signed mail using that domain.
This isn’t 100% reliable, unfortunately. Some nameservers will (wrongly) return an NXDOMAIN even if there are subdomains, so you might sometimes get an NXDOMAIN even for a domain that is publishing DKIM. shrug
Sometimes you’ll see an actual TXT record in response – e.g. Yahoo or EBay – that’s detritus left over from the days of DomainKeys, a DomainKeys policy record, and it means nothing today.

Microsoft changes

There’s been quite a bit of breakage and delivery failure to various Microsoft domains this month. It started with them changing the MX for hotmail.co.uk, then the MX for hotmail.fr… and both these things seem to have broken mail. I also saw a report this morning that some of the new MXs have TLS certificates that don’t match the hostnames.

The Internet is hard.

There are so many things that need to happen to make the Internet work. DNS entries need to be right. MXs need to be set up. Web servers need to be configured. And, let’s be honest, anyone who has ever run their own services on the Internet has flubbed a configuration.
We don’t think about it, because most of the time the configurations are handled by scripts and they do things right. But at some point someone needs to type in something and there’s a risk it will go horribly wrong. I’ve been digging into domain data for a client of mine today. I think I’m going cross-eyed over it. But I have found so many weird things that just mean someone isn’t paying attention to what they’re doing.
Like the domain that has a MX record that says:

I’m pretty sure the intention of the domain owner is to publish a null MX. But they added an extraneous “0” in there and ended up publishing something really weird. Even worse, the MTA that this client is using is listing this as a “delivered” email. I’m pretty sure that mail to that domain never left the MTA.
I’ve found horribly typoed MX domains for popular spam filters. I’ve found domains that have invalid characters in them. I’ve found domains that are totally a mess.
The vast majority of us have some story or other of the time we really broke things by accident. Like the time a very large ISP deleted their MX records. Or when a different ISP changed their internal forwarding and broke SPF authentication for everyone mailing that domain. Or when another ISP accidentally blocked every IP beginning with 6.
Sometimes I’m amazed that the Internet ever works. No matter how big it gets, there are actual people writing actual code and configurations. The number of things that have to happen to get packets from A to B is pretty impressive. We rarely ever notice the breakages, the people who run things are really good at their jobs. But sometimes poking in the grotty corners reminds me how easy it is to break things. It’s sometimes a wonder things actually work.