Another day, another ESP telling a client to publish a SPF include for the wrong domain. It shouldn’t annoy me, really. It’s mostly harmless and it’s just an extra DNS look up for most companies. Heck, we followed Mailchimp’s advice and added their include to our bare root domain and it’s not really a huge deal for companies with only a couple SaaS providers. Still, it’s an incorrect recommendation and it does cause problems for some senders who are using multiple SaaS providers and Google.
Both Steve and I have written different posts about SPF over the years. In fact, the Authenticating with SPF: -all or ~all post is the most visited post on the blog. I’ve even written almost this same post, pointing out that a lot of ESPs have bad recommendations for SPF records. Steve’s written about the technical ins and outs of SPF records in DNS and how to stay under the 10 lookup limit. There’s also a time a client made a pretty huge mistake in their SPF record and a spammer took advantage and trashed their reputation.
There are also dozens of other good write-ups and discussions on how SPF works done by other deliverability experts, filtering companies, deliverability monitoring companies and the Open SPF folks themselves. Yet still so many places get it wrong.
One of the errors comes because a lot of folks, even a lot of email experts, don’t always know or remember that there are two separate yet equally important From: addresses in an email. One, called the envelope from or return path is defined by RFC 821 (and its successors 2821 and 5321). It’s used during the SMTP transaction and is where all the bounces go. The second is what most people think of when we talk about a From address. This address is defined by RFC 822 (and its successors 2822 and 5322) and is the what we see when we look at a message displayed in our email client.
SPF authenticates the 5321 From address. This is an address no one ever sees unless they go out of their way to look at the headers. It’s a functional header in that it identifies the source of the mail and is where delayed bounces should go. In a bulk mail context every source of email should have its own hostname. 1 So my clients that send through two ESPs use two different subdomains for their mail (esp1.bounce.client.example and esp2.bounce.client.example). Each hostname has its own TXT record with the relevant SPF file. They don’t need to publish either ESP include in client.example because SPF doesn’t look up that domain.
This doesn’t stop many ESPs from recommending the wrong thing.2 Part of the blame for this we can lay at Microsoft’s feet. Back in the early 00’s Microsoft tried to push a standard called SenderID that would apply SPF lookups to the From address inside the email. This wasn’t successful and SenderID was deprecated not long after they started pushing it.3 However, not to be deterred, Microsoft started doing SPF-like lookups on the From address. Some ESPs noticed this and started recommending their clients publish SPF for both the 5321 and the 5322 domains. You can still see some traces of this with TXT records that start with spf2.0/pra.
The long and short of it is that SPF authenticates the domain in the envelope from, and that is not the address displayed in the email client. I’ve been trying to encourage ESPs and in fact anyone that offers mail through their platform to clean up their documentation. With the explosion of SaaS products, many of which handle email, too many senders end up with SPF records with more than 10 lookups.
1 – Last night in the pub we had a discussion about whether or not an ESP should ever use the 5322.from address in this string. I said ESPs should never do this. Steve said weeeelll… if they had good filtering on the message that would handle bounces and then forward any replies you could do it. And he’s technically right, but even he admits that’s going to be really, really fragile and asking for trouble.
2 – This entire post started out as a bit of a rant because a client’s ESP is refusing to turn on custom DKIM signing unless the customer also publishes a SPF include on their bare root domain. The ESP is wrong here. There’s no reason to require a SPF include on the bare root domain in order to use a custom DKIM d=. Even if we accept that sometimes publishing SPF on the 5322.from domain seems to improve Microsoft delivery, then the include should be in the bare root domain regardless of the d= in the DKIM signature.
3 – You will sometimes see TXT records that look like a SPF record but start with spf2.0/pra. Old DNS rarely gets cleaned out.