Bad SPF can hurt your reputation

Can a bad SPF record ruin your delivery, even though all your mail still passes SPF?
Yes, it can.
One of our clients had issues with poor delivery rates to the inbox at gmail and came to us with the theory that it was due to other people using their domain to send spam to gmail. This theory was based on ReturnPath instrumentation showing mail “from” their domain coming from other IP addresses, and a plausible looking correlation between that mail being sent and their problems at gmail.
Checking their bounce handler, we see a lot of bounces coming in suggesting that someone is sending poor quality mail using their bounce domain from quite a few IP addresses, including a suspicious number scattered in small blocks across 69.64.0.0/8.
Their question they had was whether they should publish DMARC records to fix the problem, and whether they should use a DMARC policy of p=reject or p=none. They’re a good candidate for DMARC – their domains are used purely for bulk or transactional mail, they have a tightly controlled mail infrastructure for their marketing domains, and they’re already publishing SPF records and signing all the mail they send with DKIM.
I was half way through writing up my normal answer about DMARC deployment for customers with this sort of mail infrastructure – “It won’t help with delivery problems directly, but publishing with p=none and analyzing the reports you get back will give you insight into your mail flows, and provide the data you need to decide whether using DMARC p=reject is appropriate for your business model and mail flows.” – when I realized that something just didn’t make sense.
Gmail, perhaps more than most other mailbox providers, base their delivery decisions on data they gather mechanically from all their mailboxes. And they really understand domain-based reputation and the difference between authenticated and non-authenticated email. Why on earth would non-authenticated email from an unrelated IP address be damaging the domain reputation, and hence the delivery of authenticated legitimate email? That makes no sense.
Meanwhile, over in our slack channel, Josh was double-checking their infrastructure…
 
slack
Oops. They have a small block of 8 IP addresses from which they send most of their email. When setting up their SPF records they inadvertently used ip4:69.20.20.48/8 instead of ip4:69.20.20.48/29 for that block of addresses. A /8 isn’t eight IP addresses – it’s every one of the 16,777,216 IP addresses that begins with “69.”.
Suddenly everything makes sense.
The SPF thinko means that all mail claiming to be from the client domain that’s sent from any IP address beginning with “69.” passes SPF – including the deluge of spam coming from the snowshoe spammers in 69.64.*.
Gmail (and other ISPs) don’t see a difference between the legitimate email and the SPF authenticated spam – they’re just seeing a high volume of authenticated email from the client domain, a large fraction of which is spam. That’s damaged the reputation of the client domain, causing their legitimate email to end up in the spam folder.
(The reality of filtering is more than just domain reputation, of course, but a terrible domain reputation is definitely going to cause you problems.)
The immediate action to take is simple – fix the SPF record so only legitimate mail will be authenticated. That’ll take effect within a couple of hours, as the old SPF record has a short TTL, and ISPs will start seeing the correct SPF record and begin rejigging their reputation.
We’ll keep monitoring delivery rates, check how long ISPs take to notice reputation changes, potentially reach out to some ISPs to see if it’s appropriate for them to do a one-time reputation reset for the affected domains, but we’re hoping things will begin to improve in the next few days.
What can you do to avoid or mitigate this sort of problem?

  1. Check your SPF records are sane. We have an SPF checker that might be useful.
  2. Have a reasonable TTL on your SPF DNS records. Too low and you’ll cause too much DNS traffic and risk timeouts. Too high and it could take days for an SPF fix to propagate. An hour or two is reasonable.
  3. Monitor traffic to your bounce domain, even if they’re not valid bounces. Anyone spamming using your domain will be sending to at least some recipients who’ll cause asynchronous bounces, and a spike there can be a sign that someone is misusing your domain.

Something else to think about: did the spammer pick our client’s domain at random, or did they search for a domain with a good reputation and an SPF bug they could take advantage of?

Related Posts

Spam, Phish or Malware?

Some mornings I check mail from my phone. This showed up this morning.
PizzaHutMail
My first thought was “oh, no, Pizza Hut is spamming, wonder who sold them my address.”
Then I remembered that iOS is horrible and won’t show you anything other than the Friendly From and maybe it was some weird phishing scheme.
When I got to my real mail client I checked headers, and sure enough, it wasn’t really from Pizza Hut. I’m guessing actually malware, but I don’t have a forensics machine to click the link and I’m not doing it on anything I can’t wipe (and have isolated from the rest of my network).
The frustrating thing for me is that this is an authenticated email. It not from Pizza Hut, the address belongs to some company in France. Apparently, that company has had their systems cracked and malware sent through them. Fully authenticated malware, pretending to be Pizza Hut, and passing authentication on various devices.
Pizza Hut isn’t currently publishing a DMARC record, but in this case, a DMARC record for Pizza Hut wouldn’t matter. None of the email addresses in the headers point to Pizza Hut.
I spent last week listening to a lot of people discussing DMARC and authentication and protecting people from scams and headers. But those all the protocols in the world won’t protect against this kind of thing. Phishing and malware can’t be fixed by technology alone. Even if every domain on the planet published a p=reject policy, mail like this would still get through.
 
 
 

Read More

Authenticating with SPF: -all or ~all

What is SPF?

Sender policy framework (SPF, RFC 7208) is an authentication process that ties the 5321.from (also known as the mail from, envelope from or return path) to authorized sending IP addresses. This authorization is published in a TXT record in DNS. Receivers can check SPF at the beginning of a SMTP transaction, compare the 5321.from domain to the connecting IP address and determine if that IP is authorized to transmit mail.

Read More

Email Authentication in a nutshell

There are 3 types of authentication currently in use for email.

Read More