Recent Posts

Optimize your SPF records

I talked on Monday about the SPF rule of ten and how it made it difficult for companies to use multiple services that send email on their behalf.
Today I’m going to look at how to fix things, by shrinking bloated SPF records. This is mostly aimed at those services who send email on their customers behalf and ask their customers to include an SPF record as that’s the biggest pain point, but some of it is also useful to people publishing their own SPF records.

Get rid of costly SPF directives
First, rethink using the “mx” directive. It’s often used in example SPF records, because it makes them look simpler. But an MX directive always triggers a DNS lookup that counts against your limit of ten, and it’ll also trigger a DNS lookup for each hostname in your MX record – they don’t count against the SPF limit but may increase the latency of your delivery a little. Better than using “mx” is to use explicit “ip4” and “ip6” records to list the addresses your smarthost and MX send mail from. Even though this makes your SPF record look longer it’ll actually make it smaller, as measured by DNS queries, as a single “mx” directive costs more than 20 “ip4” directives.
Similarly, avoid the “a” directive. It’s much less commonly seen but again can usually be replaced with “ip4” or “ip6” directives.
Don’t use “ptr” directives. They’re deprecated by the current SPF RFC.
Check the address ranges
If you have many “ip4” and “ip6” directives, make sure they’re not redundant. Are there any address ranges that you’re not using any more? Are there any adjacent address ranges that can be merged? For example, “ip4:x.y.z.4/24” and “ip4:x.y.z.5/24” can be replaced with “ip4:x.y.z.4/23” (note that you can’t always replace adjacent address blocks of the same size – read up on CIDR notation).
If you’ve generated your CIDR blocks from address ranges you can sometimes have very inefficient representations. The address range 10.11.12.1-10.11.12.254 needs 14 “ip4” directives to represent precisely. Instead you can use the single directive “ip4:10.11.12.0/24”, even if you’re not sending any email from the .0 or .255 addresses.
You don’t need a “~all” or “-all” at the end of a TXT record that is only included in another SPF record, not used directly. It won’t do any harm but it wastes a few characters.
Once you’ve got your list of SPF directives cleaned up the next thing to do is to pack them into one or more DNS TXT records.
Use as few TXT records as you can
Some SPF tutorials say that you can’t put more than 255 characters of SPF data into each TXT record. That’s not quite true, though.
A TXT record contains one or more strings of text and each string can contain no more than 255 characters. But an SPF checker will take all of the strings in a TXT record and concatenate them together in order before it starts looking at the content. So you can have more than 255 characters of SPF data in a TXT record by splitting it into more than one string. (Some low-end DNS management web front ends don’t really understand TXT records and won’t let you include multiple strings – you should check that your DNS management system does before relying on this).
How much more than 255? That’s where you have to get a little familiar with the DNS protocol, as the real limitation is that you don’t want your DNS packets to be more than 512 bytes long. (Why 512 bytes? That’s a long story of protocol changes and incompatibility, but 512 bytes are about as big as you can reliably use over UDP. Just trust me.)
The DNS overhead for a reply that contains a single TXT record with two strings is about 34 bytes, plus the length of the hostname that’s being queries (e.g. “spf.example.com” is 15 bytes). So to keep within the 512 byte limit you need to break your SPF into chunks of no more than 478 minus the length of the hostname. Then you need to break that SPF data into two strings (remembering that they’ll be concatenated with no white space added, so if you break it at a space you need to include the space at the end of the first string or the beginning of the second).
That’ll give you a TXT record that looks something like this:

Email nightmare for some FSU students

I mentioned yesterday that sometimes people and software screw up in ways that cause problems. Today I saw an article demonstrating just how bad these issues can be. Florida State University Housing Department sent detailed and confidential violation reports to tens of thousands of students.

The Internet is hard.

There are so many things that need to happen to make the Internet work. DNS entries need to be right. MXs need to be set up. Web servers need to be configured. And, let’s be honest, anyone who has ever run their own services on the Internet has flubbed a configuration.
We don’t think about it, because most of the time the configurations are handled by scripts and they do things right. But at some point someone needs to type in something and there’s a risk it will go horribly wrong. I’ve been digging into domain data for a client of mine today. I think I’m going cross-eyed over it. But I have found so many weird things that just mean someone isn’t paying attention to what they’re doing.
Like the domain that has a MX record that says:

I’m pretty sure the intention of the domain owner is to publish a null MX. But they added an extraneous “0” in there and ended up publishing something really weird. Even worse, the MTA that this client is using is listing this as a “delivered” email. I’m pretty sure that mail to that domain never left the MTA.
I’ve found horribly typoed MX domains for popular spam filters. I’ve found domains that have invalid characters in them. I’ve found domains that are totally a mess.
The vast majority of us have some story or other of the time we really broke things by accident. Like the time a very large ISP deleted their MX records. Or when a different ISP changed their internal forwarding and broke SPF authentication for everyone mailing that domain. Or when another ISP accidentally blocked every IP beginning with 6.
Sometimes I’m amazed that the Internet ever works. No matter how big it gets, there are actual people writing actual code and configurations. The number of things that have to happen to get packets from A to B is pretty impressive. We rarely ever notice the breakages, the people who run things are really good at their jobs. But sometimes poking in the grotty corners reminds me how easy it is to break things. It’s sometimes a wonder things actually work.

The 10 worst …

Spamhaus gave a bunch of us a preview of their new “Top 10 worst” (or should that be bottom 10?) lists at M³AAWG. These lists have now been released to the public.

The categories they’re measuring are:

Some mechanisms and modifiers (collectively, “terms”) cause DNS queries at the time of evaluation, and some do not. The following terms cause DNS queries: the “include”, “a”, “mx”, “ptr”, and “exists” mechanisms, and the “redirect” modifier. SPF implementations MUST limit the total number of those terms to 10 during SPF evaluation, to avoid unreasonable load on the DNS. If this limit is exceeded, the implementation MUST return “permerror”.
Read More

Ask Laura: Does changing ESPs hurt deliverability?

Mutt: Mailbox power tool

Milestone_1_6_–_Mutt

“All mail clients suck. This one just sucks less.”

Things to read: March 9, 2016

It’s sometimes hard for me to keep up with what other people are saying and discussing about email marketing. I’ve been trying to be more active on LinkedIn, but there are just so many good marketing and delivery blogs out there I can’t keep up with all of them.

Here are a couple interesting things I’ve read in the last week.
Five Steps to Stay Out of the Spam Folder. Conceptually easy, sometimes hard to pull off in practice, these recommendations mirror many things I say here and tell my clients about delivery. The audience is in charge and your recipients are the best ally you can have when it comes to getting into the inbox.
Which states are the biggest sources of spam?. California and New York top the list, but the next two states are a little surprising. Over on Spamresource, Al points out the two next states have some unique laws that may affect the data. I just remember back in the day there were a lot of spammers in Michigan, I’m surprised there’s still a significant volume from there.
CASL didn’t destroy Canadian email. Despite concerns that CASL would destroy the Canadian email marketing industry, the industry is going strong and expanding. In fact, spending on email marketing in Canada was up more than 14% in 2015 and is on track to be up another 10% this year. Additionally, according to eMarketer lists are performing better because they’re cleaner.
A brief history of email. Part of the Guardian’s tribute to Ray Tomlinson, the person who sent the first email. Ray’s work literally changed lives. I know my life would be significantly different if there wasn’t email. Can you imagine trying to be a deliverability consultant without email? 🙂

What really are the best practices for email?

A year ago I wrote a post about best practices and how most of my best practices were different from what other people recommend. I don’t talk about rules for frequency or subject line length. I don’t focus on best practices for bounce processing or content length.
BestPracticesBall My best practice recommendations are really about process.

65.0.0.0/8 DNS issues

If you’re sending email from any address beginning with a 65 – in 65.0.0.0/8 – it’s possible you’ll see some delivery problems.
Something appears to be broken with dnssec signatures for the reverse DNS zone, leading queries for reverse DNS to fail for anyone using a dnssec aware DNS resolver (which is almost everyone).