Authenticating with SPF: -all or ~all

laura
Industry
June 12, 2014

What is SPF?

Sender policy framework (SPF, RFC 7208) is an authentication process that ties the 5321.from (also known as the mail from, envelope from or return path) to authorized sending IP addresses. This authorization is published in a TXT record in DNS. Receivers can check SPF at the beginning of a SMTP transaction, compare the 5321.from domain to the connecting IP address and determine if that IP is authorized to transmit mail.

What does a SPF record look like?

At its simplest, the SPF TXT record contains a version indicator, allowed IPs and an authorization type.
In the example "v=spf1 ip4:198.51.100.26 -all":

v=spf1 is the version indicator
198.51.100.26 is the allowed sending IP
-all means only this IP is authorized to send mail for the domain.

Of course, there are other ways to define authorized IP addresses. Using "v=spf1 mx -all" authorizes any IP that is also a MX for the sending domain. Other SPF records can be included using the include: command; for instance include:_spf.google.com includes Google’s SPF record. IPs can be in either IPv4 space or IPv6 space by using either the ip4 or ip6 qualifiers: "v=spf1 ip4:198.51.100.26 ip6:2001:db8:8:4::2 -all". SPF records can also contain IP ranges in the form "v=spf1 ip4:198.51.100.128/25 -all".
Domain owners are also allowed to publish different types of authorization.
[table] Statement,Result,Meaning
+all,pass,Allow all mail
-all,fail,”Only allow mail that matches one of the parameters (IPv4, MX, etc) in the record”
~all,softfail,Allow mail whether or not it matches the parameters in the record
?all,neutral,No policy statement
[/table]

What’s the difference between ~all and -all

Given many receivers are not actively bouncing mail based on SPF pass/fail, there isn’t a strong argument for either -all or ~all in SPF records. For a while, Hotmail was advising that senders who published a -all record would have better delivery. This led to -all became a de-facto standard for a lot of ESPs and bulk senders. More recently, there does not seem to be any benefit to publishing -all even at Hotmail (Outlook.com, live.com, etc).

What should I publish?

I generally recommend publishing ~all records for my clients. There’s not a huge benefit to publishing -all and sometimes mail gets forwarded around. The one time I recommend a -all record is when a domain is getting forged into spam. Domain forgery can cause a lot of bounces. The amount of bounces can be bad enough to take down a mail server, particularly those with a small userbase. Many ISPs will check SPF before sending back a bounce and so a -all record can decrease the amount of blowback the domain owner has to deal with.

Do I have to publish SPF records?

No, there is no requirement for publishing SPF in order to send mail. You don’t even need to publish SPF to get inbox delivery. Gmail will even do a “best guess SPF” for domains not publishing SPF and authenticate off that. However, large volume senders should be publishing SPF records on principle.

Want to check your SPF record?

We provide a SPF checker on our Tools page.

Who can you trust?

I’ve been recently dealing with a client who is looking at implementing authentication on their domains. He’s done a lot of background research into the schemes and has a relatively firm grasp on the issue. At this point we’re working out what policies he wants to set and how to correctly implement those policies.
His questions were well informed for the most part. A few of them were completely out of left field, so I asked him for some of his references. One of those references was the EEC Email Authentication Whitepaper.
My client was doing the best he could to inform himself and relies on industry groups like the EEC to provide him with accurate information. In this case, their information was incomplete and incorrect.
We all have our perspectives and biases (yes, even me!) but there are objective facts that can be independently verified. For instance, the EEC Authentication whitepaper claimed that Yahoo requires DKIM signing for access to their whitelist program. This is incorrect, a sender does not have to sign with DKIM in order to apply for the Yahoo whitelist program. A bulk sender does have to sign with DKIM for a Y! FBL, but ISPs are given access to an IP based FBL by Yahoo. I am shocked that none of the experts that contributed to the document caught that error.
Independent verification is one reason I publish the Delivery Wiki. It’s a resource for everyone and a way to share my knowledge and thought processes. But other experts can “check my work” as it were and provide corrections if my information is outdated or faulty. All too often, senders end up blaming delivery problems on evil spirits, or using “dear” in the subject line or using too much pink in the design.
Delivery isn’t that esoteric or difficult if you have a clear understanding of the policy and technical decisions at a range of ESPs and ISPs, the history and reasoning behind those decisions, and enough experience to predict the implications when they collide.
Many senders do face delivery challenges and there is considerable demand for delivery experts to provide delivery facts. That niche has been filled by a mix of people, of all levels of experience, expertise and technical knowledge, leading to the difficult task of working out which of those “experts” are experts, and which of those “facts” are facts.

Too much email on the brain

Last night I was cruising through our local news website. I see the headline New SPF guidelines coming our way.
My first thought was, “Wow, SPF made the paper?” Now, I live in the SF Bay area so there are a lot of technology related stories that hit our paper which might not see the light of day in other areas. But, still: new SPF guidelines hit the local paper before I’ve heard about it? That seems a little strange.
Then I notice that it’s in the “Living” section. That’s even stranger.
Oh, well, if there’s new SPF stuff, I’d better click and see what is going on with SPF. The internal headline is Beauty Tuesday: New SPF guidelines accompanied by a picture of sunscreen. It was only then I realized it wasn’t about sender policy framework but was about sun protection.
A bit of a moment for me.
Happy Friday, everyone.

Hotmail moves to SPF authentication

Hotmail has recently stopped using Sender ID for email authentication and switched to authenticating with SPF. The protocol differences between SenderID and SPF were subtle and most senders who were getting a pass at Hotmail were already publishing SPF records.
From an email in my inbox from September: