Authenticating with SPF: -all or ~all

By laura
46 comments
June 12, 2014
3 Min read
In Industry
A

What is SPF?

Sender policy framework (SPF, RFC 7208) is an authentication process that ties the 5321.from (also known as the mail from, envelope from or return path) to authorized sending IP addresses. This authorization is published in a TXT record in DNS. Receivers can check SPF at the beginning of a SMTP transaction, compare the 5321.from domain to the connecting IP address and determine if that IP is authorized to transmit mail.

What does a SPF record look like?

At its simplest, the SPF TXT record contains a version indicator, allowed IPs and an authorization type.
In the example "v=spf1 ip4:198.51.100.26 -all":

  • v=spf1 is the version indicator
  • 198.51.100.26 is the allowed sending IP
  • -all means only this IP is authorized to send mail for the domain.

Of course, there are other ways to define authorized IP addresses. Using "v=spf1 mx -all" authorizes any IP that is also a MX for the sending domain. Other SPF records can be included using the include: command; for instance include:_spf.google.com includes Google’s SPF record. IPs can be in either IPv4 space or IPv6 space  by using either the ip4 or ip6 qualifiers: "v=spf1 ip4:198.51.100.26 ip6:2001:db8:8:4::2 -all". SPF records can also contain IP ranges in the form "v=spf1 ip4:198.51.100.128/25 -all".
Domain owners are also allowed to publish different types of authorization.
[table] Statement,Result,Meaning
+all,pass,Allow all mail
-all,fail,”Only allow mail that matches one of the parameters (IPv4, MX, etc) in the record”
~all,softfail,Allow mail whether or not it matches the parameters in the record
?all,neutral,No policy statement
[/table]

What’s the difference between ~all and -all

Given many receivers are not actively bouncing mail based on SPF pass/fail, there isn’t a strong argument for either -all or ~all in SPF records. For a while, Hotmail was advising that senders who published a -all record would have better delivery. This led to -all became a de-facto standard for a lot of ESPs and bulk senders. More recently, there does not seem to be any benefit to publishing -all even at Hotmail (Outlook.com, live.com, etc).

What should I publish?

I generally recommend publishing ~all records for my clients. There’s not a huge benefit to publishing -all and sometimes mail gets forwarded around. The one time I recommend a -all record is when a domain is getting forged into spam. Domain forgery can cause a lot of bounces. The amount of bounces can be bad enough to take down a mail server, particularly those with a small userbase. Many ISPs will check SPF before sending back a bounce and so a -all record can decrease the amount of blowback the domain owner has to deal with.

Do I have to publish SPF records?

No, there is no requirement for publishing SPF in order to send mail. You don’t even need to publish SPF to get inbox delivery. Gmail will even do a “best guess SPF” for domains not publishing SPF and authenticate off that. However, large volume senders should be publishing SPF records on principle.

Want to check your SPF record?

We provide a SPF checker on our Tools page.

FacebookLinkedIn

About the author

laura
View all posts

46 comments

This site uses Akismet to reduce spam. Learn how your comment data is processed.

  • What are the tabs at the top of the spf checker page supposed to do? They don’t seem to do anything now.

  • There are still some javascript bugs with the tools.wordtothewise.com pages. Working on ’em.

  • One thing that I’d like to add here. If you keep your SPF record in a Word document, when you copy it out, the dash changes from an “en dash” to an “em dash”. And that just doesn’t work. Tough to see the difference between: -all and –all

  • I used an online tool to make the spf record I published,
    http://www.spfwizard.net/
    but I have some evidence that it’s not *exactly* right.
    Your tool for example says it’s not there as SPF, but it is returned by your dns TXT tool as TXT.
    That’s the problem with trusting something you find online. You have to trust that it gives the correct output. You have to have faith.
    With some checkers saying it’s fine, and some saying it’s not, well, thanks for the article. I didn’t learn enough to fix it for sure, but I learned something.

  • Hi, Im a newbie in SPF, may I ask what IP should I put in the ip4: ? Where will I get that IP…?
    Thanks in advance.

  • In my opinion there is a huge difference between ~all and -all.
    ~all is “softfail” which has the intended action “accept”, in my world this makes SPF useless since no one will take any action on a “softfail”. In that case you might as well not configure a SPF-record at all.
    -all is “fail” which has the intended action “reject”. If you have control over the servers sending emails for your domain I see no reason not to use -all.
    If all domain owner used SPF with -all there would be a lot less spam- and phishing-emails.

  • I generally recommend publishing ~all records for my clients.
    I would never ever recommend that. SPF’s point is to block all not matching IP’s so why would i ever use that except for testing?

  • I see it this way: The only reason to not use -all is if you are unsure that you have properly listed all the sending IPs associated with your domain. If your IT dept can’t keep track of all the legitimate senders at your domain, then I don’t trust your incompetent IT dept. Since SPF is for the recipient and the recipient is the one that chooses whether to trust the incoming email, I treat ~all as -all. I use -all for my own domains. If the sending IP is not listed or included in the record, the mail is coming from an illegitimate source.
    ~all is for testing and nullifies the purpose of SPF. Don’t use it.

  • ” If your IT dept can’t keep track of all the legitimate senders at your domain, then I don’t trust your incompetent IT dept.”
    If your IT dept CAN keep track of all of the legitimate senders at your domain, then you obviously work for a much smaller company than than a lot of other people do.

  • question can incldude both ?all and -all in one SPF entry? I have two domains i need to list in record:
    example below
    v=spf1 include:spf.protection.outlook.com -all ip4:xx.xxx.xxx.xx include:mail.somecompany.com ?all

  • That is an invalid SPF record, you can only have one “all” statement in the record. You seem to be trying to say “mail can only come from protection.outlook.com” unless it comes from these other places. You can use one -all or you can use ~all (never use ?).

  • so only following entry is correct?
    v=spf1 include:spf.protection.outlook.com ip4:xx.xx.xxx.xx include:mail.othercompany.com -all

  • Hi Laura,
    i have been working at this for over a month and i still get email spoof. i finally learned how to make a spf record. and it works when you test it. our domain is optimasystems.ca but we still get the same amount of email spoofing. it never stopped even a tiny bit. even though the spf records passes the test of being set up properly, is there something i am missing that is allowing these spoof emails to come through? here is my record Host record: @ TXT Value: v=spf1 mx a ip4:MYSHAREDIPADDRESSHERE?all

  • Actually i just noticed something i used your SPF Checker and entered my domain and it said this:
    This is an approximate list of the IP addresses that the domain optimasystems.ca allows email to be sent from according to their SPF record.
    then it showed a complete list of IPS. all of those IPS are where all the spoof emails are coming from according to the email headers of those spoofed emails (mostly from netherlands and Germany)? how is that possible that it says “ALLOWS” where can i typically erase those IPS so they are NOT allowed? thank you so much for helping me out in advance!

  • If you go to the DNS tab on the authentication page, it shows you what all the lookups are. Most of the IPs in the record belong to “mx.spamexperts.com.”
    If you’re only sending from one IP address, take out the “MX” and “A” from your SPF record. That should remove all the unexpected IPs from your SPF record.

  • Could someone let me know if I include: an SPF record has soft-fail (~all) to my SPF record that has hard-fail (-all) what will happen? Will it soft-fail the included servers only, and hard our own?
    Thanks in advance.

  • The fact that major mail handlers like google use ~all IS one of the reasons spf is a total and absolute utter disaster. This is a debug setting and should only be used during testing and initial configuration setup. to recommend it is the same as saying don’t bother using spf.
    The reason why google does do it by the way is because it sends a large amount of mail on behalf of others and those others are unable or do not understand how to create a suitable spf record.
    Personally I don’t think that’s a valid excuse.
    The other reason is, I am seeing a huge increase in spf validated email from all the new tld’s.
    I am now considering filtering all email from the new tld’s as spam. or at least giving them a higher spam score.

  • can i use mx record of my server in the +include option????? in the SPF , i have included my ip address , but i’m using a cnetralized mx record for all of my servers , so i think i have to include those too.
    +include:mx_record_name

  • I am currently in the process of working through all the 3rd parties that are sending as my domain. I have created a sub domain email.domain.com. The plan is to use a separate SPF record, some of the 3rd parties have advised to add their spf record to our own spf by a way of an Include. Whilst I am happy to do that within reason, I noticed their SPF record in some instances ends in a “~all” is that a problem?
    Regarding the Include record. Some instances suggest to add a ? in front of it:
    ?include:spf.3rdparty.com
    I have seen some instances with:
    +include: ?include:spf.3rdparty.com
    Can anyone explain what is the best approach to this?

  • include records only pull the IP addresses, they don’t pull the recommendation, so it doesn’t matter if it’s – or ~. I don’t think the ? in front of the include record is necessary or useful. So I’d suggest leaving it out.

  • SPF is a failing system as long as people are allowed to use ~all. This goes, especially, for large organisations who complain about having no control over their list of sending IPs. This is not an acceptable argument. If the big players can’t “up their game” and sort their sender IP management out SPF will, in time, die and spam will have won, yet again.

  • The reason for google using the soft fail is because of the following record:
    _dmarc.google.com. 300 IN TXT “v=DMARC1\; p=reject\; rua=mailto:mailauth-reports@google.com”
    SPF is no longer good enough security by itself. If you are serious about protecting your brand from spoofing, then you MUST implement DMARC as well.

  • To Steevo, SPF DNS records used to come in two forms: As both SPF records and as a TXT records. This article only goes over the TXT version. The old SPF DNS record type has been deprecated, but is still used by some older systems. Any modern tool creating SPF records will format the record in TXT record format rather than SPF record format so that’s what the query tool will return…TXT records.

  • Laura says to never use ?all. Could someone explain why?
    Wouldn’t a neutral rating be better than a soft-fail or fail? Or does the soft-fail / fail / neutral means what happens when a non-authorized server is trying to send e-mail? For example, if I use -all, I’ll get the fail rating from the SPF checkers I’ve used. That makes me think I’ve gotten it configured wrong. I came here looking for the answer and am still a little bit confused.
    Thanks

  • Soft-fail means that you expect most mail to come from these servers, but some valid mail might come from other servers. ?all means absolutely nothing.
    From http://www.openspf.org/SPF_Record_Syntax
    SoftFail (~all) means the SPF record has designated the host as NOT being allowed to send but is in transition. Suggested action is accept but mark.
    Neutral (?all) means the SPF record specifies explicitly that nothing can be said about validity. Suggested action is accept.
    In practice, there’s not a lot of difference between how ISPs are handling ~all and -all records if the IP address is in them and is correct. To the best of my knowledge, if SPF passes and you’re using ~all there’s no “downgrade” in how the message is handled. If you’re using ~all and SPF fails, the mail is still accepted. If you’re using -all and the message fails, then usually the mail is still accepted but may be filtered or bulked and trigger a DMARC failure report if there is a DMARC record configured.
    If you’re using -all and authentication is failing, then there is some problem.

  • Laura,
    Thank you for explaining that to me. So SoftFail doesn’t mean I’m failing the SPF test that the SPF testers are performing then, right? This was what was confusing me. A +all or all (which is the same thing) shows as a Pass on these tests, but ~all shows as a SoftFail and -all shows as a Fail. I knew what the +all / all, -all, ~all, and ?all meant. From what I’ve been reading, it sounded like -all was the correct choice, once the SPF records were properly configured, but from your article, it seems ~all might be the more common option. I’ll probably stick with ~all.
    Now that I have the SPF and DKIM setup and working properly, now I have to look into this DMARC. I’ve seen a little about it.
    Thanks!

  • +all means that every IP is allowed to send mail for your IP address. So every message will pass. It’s not recommended that you use +all – some of the ISPs will treat that as a problem and actually downgrade your message.
    If the records are right, then you should see a pass no matter what you’re using in front of all. If some testers are telling you there is a failure, then there’s something broken about the record.
    One of the absolute easiest ways to check a record is to send mail to Gmail then click on “view original” to see the headers. They are providing SPF and DKIM reports in an easy to read fashion. For mail I send from wordtothewise.com to gmail.com I see:
    SPF: PASS with IP 184.105.179.154
    DKIM: PASS with domain wordtothewise.com
    The nice part about this is it tells you what IP they’re checking – and so you can then use that to troubleshoot the SPF error. If you want to drop me an email (use the contact address) from the system showing the fail I will take a look. My initial guess is that there’s an issue with the 5321.from and the 5322.from and one of them isn’t right. But there are other issues that will cause a SPF fail, too. But I need to see the message from the problematic system in order to diagnose it.

  • Laura, great forum, thank you. Having digested this (probably too much) I see no reason whatsoever to ever use the ? qualifier. Can you determine a reason why anyone would use it?

  • ~all is originally intended to use as testing/logging purposes only before moving to production mode.
    When activating SPF-record for production it should in my opinion be with -all or nothing at all. Whats the use of SPF-record if its not effective 😉
    -all requires a lot of background work for data gathering but when its done its pretty good anti-forgery(not antispam) tool.

  • Hi Laura,
    thanks for this interesting post and for sharing your knowledge.
    I performed some searches to approach the question from a statistic point of view.
    From this (meaningful but) very short list:
    us.ibm.com, cisco.com, google.com, dell.com and oracle.com use ~all
    apple.com, microsoft.com use -all
    it seems that ~all is the favorite 🙂

  • Hi,
    I have my own vps.
    I am using zoho for my mail (v=spf1 include:zoho.eu ~all)
    I also use aweber for my email marketing (v=spf1 mx include:send.aweber.com -all)
    How can I combine these 2 spf values to ensure email delivery to inbox not spam?
    Kind Regards,
    Thank you

  • a by itself means all A record IPs can potentially send email?
    ptr by itself is deprecated?
    mx by itself unnecessary if all the email sending “IPs”/servers for the domain are in the SPF?

  • > There’s not a huge benefit to publishing -all and sometimes mail gets forwarded around.

    Some people set up automatic email forwarders from their custom domain to a webmail account (e.g. Gmail), and these emails will obviously fail SPF, but the emails are still legitimate. This is why no major email provider will outright reject emails that fail SPF checks. SPF is treated as nothing more than an additional indicator, among many, of either legitimacy or forgery.

    However beware that if you set up a DMARC policy to reject emails that fail SPF (i.e. setting “fo” to “1” or “s”), these email providers will be much more likely to honor the SPF fail, which could prevent emails you send from being automatically forwarded.

Read more

By laura June 12, 2014
46 comments

Recent Posts

Archives

Follow Us