Another day, another ESP telling a client to publish a SPF include for the wrong domain. It shouldn’t annoy me, really. It’s mostly harmless and it’s just an extra DNS look up for most companies. Heck, we followed Mailchimp’s advice and added their include to our bare root domain and it’s not really a huge deal for companies with only a couple SaaS providers. Still, it’s an incorrect recommendation and it does cause problems for some senders who are using multiple SaaS providers and Google.
Both Steve and I have written different posts about SPF over the years. In fact, the Authenticating with SPF: -all or ~all post is the most visited post on the blog. I’ve even written almost this same post, pointing out that a lot of ESPs have bad recommendations for SPF records. Steve’s written about the technical ins and outs of SPF records in DNS and how to stay under the 10 lookup limit. There’s also a time a client made a pretty huge mistake in their SPF record and a spammer took advantage and trashed their reputation.
There are also dozens of other good write-ups and discussions on how SPF works done by other deliverability experts, filtering companies, deliverability monitoring companies and the Open SPF folks themselves. Yet still so many places get it wrong.
One of the errors comes because a lot of folks, even a lot of email experts, don’t always know or remember that there are two separate yet equally important From: addresses in an email. One, called the envelope from or return path is defined by RFC 821 (and its successors 2821 and 5321). It’s used during the SMTP transaction and is where all the bounces go. The second is what most people think of when we talk about a From address. This address is defined by RFC 822 (and its successors 2822 and 5322) and is the what we see when we look at a message displayed in our email client.
SPF authenticates the 5321 From address. This is an address no one ever sees unless they go out of their way to look at the headers. It’s a functional header in that it identifies the source of the mail and is where delayed bounces should go. In a bulk mail context every source of email should have its own hostname. 1 So my clients that send through two ESPs use two different subdomains for their mail (esp1.bounce.client.example and esp2.bounce.client.example). Each hostname has its own TXT record with the relevant SPF file. They don’t need to publish either ESP include in client.example because SPF doesn’t look up that domain.
This doesn’t stop many ESPs from recommending the wrong thing.2 Part of the blame for this we can lay at Microsoft’s feet. Back in the early 00’s Microsoft tried to push a standard called SenderID that would apply SPF lookups to the From address inside the email. This wasn’t successful and SenderID was deprecated not long after they started pushing it.3 However, not to be deterred, Microsoft started doing SPF-like lookups on the From address. Some ESPs noticed this and started recommending their clients publish SPF for both the 5321 and the 5322 domains. You can still see some traces of this with TXT records that start with spf2.0/pra.
The long and short of it is that SPF authenticates the domain in the envelope from, and that is not the address displayed in the email client. I’ve been trying to encourage ESPs and in fact anyone that offers mail through their platform to clean up their documentation. With the explosion of SaaS products, many of which handle email, too many senders end up with SPF records with more than 10 lookups.
1 – Last night in the pub we had a discussion about whether or not an ESP should ever use the 5322.from address in this string. I said ESPs should never do this. Steve said weeeelll… if they had good filtering on the message that would handle bounces and then forward any replies you could do it. And he’s technically right, but even he admits that’s going to be really, really fragile and asking for trouble.
2 – This entire post started out as a bit of a rant because a client’s ESP is refusing to turn on custom DKIM signing unless the customer also publishes a SPF include on their bare root domain. The ESP is wrong here. There’s no reason to require a SPF include on the bare root domain in order to use a custom DKIM d=. Even if we accept that sometimes publishing SPF on the 5322.from domain seems to improve Microsoft delivery, then the include should be in the bare root domain regardless of the d= in the DKIM signature.
3 – You will sometimes see TXT records that look like a SPF record but start with spf2.0/pra. Old DNS rarely gets cleaned out.
RSS - Posts
One of the things I’ve noticed over the years is that many ESPs and 3rd party senders do/recommend what is easiest and most convenient for them rather than what is best for their customer. “Just use an include” advice falls into this category.
Thanks. I’ve spent years cleaning my ESP’s knowledgebase-articles of this nonsense. But I on a regular basis encounter customer that added us to their rfc5322 from domain because they found something in some kb- or helpdesk-text from 2012 that I missed. …and our parent company had an spf2.0-record until I spotted it a few months ago.
I used to have a rule that for every 20th customer I had to explain the difference between SPF and 5321 from vs DKIM and 5322 from, I would buy a new guitar. I now have way too many guitars, most are stored in the basement.
Regarding that first footnote, I do see people who want to use their top-level domain for everything, and that’s definitely one of those things that requires functionality that isn’t present in all mail software and services. So, even if your current mail service lets you do it, it will probably be a lot of work to set up, and if you later change mail services you’ll have to reimplement it all again, which may or may not even be possible. As you say, using separate subdomains for each ESP is definitely a lot cleaner and more robust!
You can absolutely use your root domain at multiple ESPs in the 5322.from. There’s absolutely nothing wrong with it and it is, to my mind, a standard and even good practice. But the 5322.from has nothing to do with SPF at all.
An ESP that forces customers to publish a SPF record for their root domain, when they’re not using that root domain in the 5321.from is giving bad advice to their customers. I’ve found exactly one SaaS provider that uses the root domain in the 5321.from in the last 5 years. An ESP that forces you to publish a SPF record for the 5322.from in order to use custom DKIM is extremely wrong.
https://xkcd.com/386/
Is this still the case today?
According to Google’s latest Email sender guidelines:
> The SPF record for your domain should reference all email senders for your domain. If third-party senders aren’t included in your SPF record, messages from these senders are more likely to be marked as spam.
My advice is still accurate. SPF authenticates the 5321.from address and the EHLO domain. If you’re publishing an include record for the 5322.from address you’re not doing SPF and you should stop doing it.