Authentication

Google and Alignment Update

Earlier this month, I published a post about some changes with how Google is displaying information related to authentication in their “View Original” page. There’s one condition I apparently didn’t report and it brought up a question earlier today.

Google, Alignment and DMARC

Google has been making a number of changes to their systems over the last few weeks. Folks are seeing a lot of changes in Google postmaster tools and they’re seeing changes in how Google is displaying headers in the “show original” tab.

A series of red arrows pointed left and one green arrow pointing right.

DMARC is the newest of the authentication protocols. It compares the domain in the From: address to the domains authenticated by SPF and DKIM. If either SPF or DKIM pass and they are in the same organizational domain as the domain in the From: address then the email is authenticated with DMARC.

The gentleman with the excellent hat is Иван IV Васильевич, The Great Sovereign, Tsar and Grand Prince of all Russia, Vladimir, Moscow, Novgorod, Tsar of Kazan, Tsar of Astrakhan, Sovereign of Pskov, Grand Prince of Smolensk, Tver, Yugorsk, Perm, Vyatka, Bolgar and others, Sovereign and Grand Prince of Novgorod of the Lower Land, Chernigov, Ryazan, Polotsk, Rostov, Yaroslavl, Beloozero, Livonia, Udoria, Obdoria, Kondia and Master of all the Siberian Lands and Northern Countries.

A common complaint about the advice or answers any deliverability person gives is that the generic answer to questions is: It Depends. This is frustrating for a lot of folks because they think they’re asking a simple question and so, clearly, there should be one, simple, clear answer.

DNS for white label authentication with SproutDNS

I wrote last year about using “stunt” nameservers for customer subdomain authentication – i.e. dynamically generating all the authentication records needed in DNS for each customer as needed.

Answers to your questions about the new Yahoo and Google technical requirements

On January 9th at 6pm GMT, 1pm EST and 10am PST I’ll be speaking with Nout Boctor-Smith of Nine Lives Digital about the new Yahoo and Google technical requirements.

Yahoogle Requirements Update

Since I wrote about it last month the requirements for bulk senders to Yahoo and Google have changed a little.

Customer subdomain authentication

EDIT: Now with a production-ready implementation I talk about more here.

Wildcards and DKIM and DMARC, oh my!

If you’re an ESP with small customers you may have looked at the recent Google / Yahoo requirements around DMARC-style alignment for authentication and panicked a bit.

New Requirements for Bulk Senders

UPDATE: You need to authenticate with both DKIM and SPF.

Stop with the incorrect SPF advice

Another day, another ESP telling a client to publish a SPF include for the wrong domain. It shouldn’t annoy me, really. It’s mostly harmless and it’s just an extra DNS look up for most companies. Heck, we followed Mailchimp’s advice and added their include to our bare root domain and it’s not really a huge deal for companies with only a couple SaaS providers. Still, it’s an incorrect recommendation and it does cause problems for some senders who are using multiple SaaS providers and Google.

Authentication at Office365

This is a followup from a post a few weeks ago about authentication changes at Office365. We have some more clarity on what is going on there. This is all best information we have right now.

Some Microsoft thoughts

Right at the end of January, Microsoft appears to have made couple of changes to how they’re handling authentication. The interesting piece of this is that, in both cases, Microsoft is taking authentication protocols and using them in ways that are slightly outside the spec, but are logical extensions of the spec.

Cost of authentication

At the end of last year, Steve wrote a post about the different types of authentication. I thought I’d build on that and write about the costs associated with each type. While I know a lot of my readers are actually on the sending side, I’m also going to talk about the costs associated with the receiving side and a little bit about the costs for intermediaries such as CRM systems or ESPs.

Authentication

Some notes on some of the different protocols used for authentication and authentication-adjacent things in email. Some of this is oral history, and some of it may be contradicted by later or more public historical revision.

Microsoft and SPF

Many deliverability folks stopped recommending publishing SPF records for the 5322.from address to get delivery to Microsoft. I even remember Microsoft saying they were stopping doing SenderID style checking. A discussion on the emailgeeks slack channel has me rethinking that.

Why is DMARC failing?

Multiple times over the last few weeks folks have posted a screenshot of Google Postmaster tools showing some percentage of mail failing DMARC. They then ask why DMARC is failing. Thanks to how DMARC was designed, they don’t need to ask anyone this, they have all the data they need to work this out themselves.

Should you publish a DMARC policy statement?

DMARC is a protocol that makes it very, very simple to shoot yourself in the foot. Setup is tricky and if you don’t get it exactly right you risk creating deliverability problems. The vast majority of companies SHOULD NOT publish a DMARC policy with p=reject or p=quarantine for their existing domains.

Null sender address

A question came up on the email geeks slack channel about empty from addresses. I asked if they meant the 5321 or 5322 from address which prompted a question about if you could even have a null 5321 from.

DMARC doesn’t fix phishing

Over the last few weeks I’ve had a lot of discussions with folks about DMARC and the very slow adoption. A big upsurge and multiple Facebook discussions were triggered by the ZDNet article DMARCs abysmal adoption explains why email spoofing is still a thing.

There are a lot of reasons DMARC’s adoption has been slow, and I’m working on a more comprehensive discussion. But one of the absolute biggest reasons is that it doesn’t actually fix phishing.

d= for data

A few ISPs use the d= value in the DKIM signature as a way to provide FBL and reputation data to senders. This has some good bits, in that senders can get FBLs and other information regardless of the IP address they’re using and whether or not they have sole access to it.

Delivery is not dependent on authentication

All too often folks come to me with delivery problems and lead off with all of the things they’ve done to send mail right. They assure me they’re using SPF and DKIM and DMARC and they can’t understand why things are bad. There is this pervasive belief that if you do all the technical things right then you will reach the inbox.

AMP and Gmail

Yesterday,Gmail announced they’re rolling out AMP support in their web client, with support for mobil coming soon.

Phishing and authentication

This morning I got a rather suspicious message from a colleague on LinkedIn.

SenderID is dead

A question came up on the email geeks slack channel (Join Here) about SenderID. They recently had a customer ask for SenderID authentication.

Deliverability Help: Information checklist

When asking a for assistance with email delivery, there are some pieces of information that are required before anyone can help. Be prepared with the information so you can get timely assistance. This advice is true whether you’re looking for help from peers or working with paid deliverability consultants.

Cousin domains

When I checked in on Facebook this morning there was a discussion from a couple people frustrated by cousin domains. I share their frustration.

What SPF records should you publish?

When it comes to SPF records there seems to be a lot of confusion. I mean, a decade after I posted it Authenticating SPF is still the most frequently visited post on the site. And, of course, there are hundreds of other pages out there that discuss SPF and what to publish. Still, there are common questions.

Fun with spam filters

I recently had a challenging travel experience in the Netherlands, trying to get from Schipol airport to a conference I was speaking at. As part of my attempt to get out of the airport, I installed UBER on my phone. There were some challenges with getting UBER to authorise my phone number, so I tried linking it to my Gmail account.

Schroedinger’s email

The riskiest email to send is that very first email. It’s a blank slate. Even if you’re sending confirmation messages, you don’t really know anything about how this email is going to affect your reputation.

Brand indicators in email

A number of companies in the email industry have been working on a way to better identify authenticated emails to users. One proposal is Brand Indicators for Message Identification (BIMI). A couple weeks ago, Agari announced a pilot program with some brands and a number of major consumer mail providers. These logos should be available in the Yahoo interface now and will be rolling out at other providers.

October 2017: The Month in Email

October was a busy month. In addition to on boarding multiple new clients, we got new desks, I went to Toronto to see M³AAWG colleagues for a few days, and had oral surgery. Happily, we’re finally getting closer to having the full office setup.

What is an office without a Grover Cat? (he was so pleased he figured out how to get onto it at standing height).

All of this means that blogging was pretty light this month.
One of the most interesting bits of news this month is that the US National Cybersecurity Assessments & Technical Services Team issued a mandate on web and email security, which Steve reviewed here.
In best practices, I made a brief mention about the importance of using subdomains rather than entirely new domain names in links and emails and even DKIM keys.
We’ve talked about engagement-based filters before, but it’s interesting to note how they’re being used in business environments as well as consumer environments.
We also put together a survey looking at how people use Google Postmaster Tools. The survey is now closed, and I’ll be doing a full analysis over the next couple of weeks, as well as talking about next steps. I did a quick preview of some of the highlights earlier this week.
Finally, a lot of industry news this month: Most notably, Mailchimp has changed its default signup process from double opt-in to single opt-in. This caused quite a bit of sturm und drang from all ends of the industry. And, in fact, a few days later they announced the default double-opt-in would stay in place for .eu senders. I didn’t get a chance to blog about that as it happened. In other news, the Road Runner FBL is permanently shuttered, and Edison Software has acquired Return Path’s Consumer Insight division. Also worth noting: Microsoft is rolling out new mail servers, and you’ll likely see some new — and potentially confusing — error codes.
My October themed photo is behind a cut, for those of you who have problems with spiders.

People are the weakest link

All of the technical security in the world won’t fix the biggest security problem: people. Let’s face it, we are the weakest link. Adding more security doesn’t work, it only causes people to figure out ways to get around the security.

Engagement drives deliverability

Return Path released an white paper today offering the Secrets of Successful Senders. I don’t think any of my readers will be surprised that it boils down to identity, reputation, and engagement. Return Path treats these as separate things and I understand why they do. I think however, that the identity and reputation are supporting players to the overarching issue of engagement.

When I’m dealing with clients and troubleshooting deliverability problems and offering solutions, I focus on the root cause. To me the root cause is almost always a data problem. Either there’s a problem with data collection or there’s a problem with data maintenance. These problems result in mail going to people who don’t really want or care about it.
Yes, identity is important. But, realistically, anyone mailing through a decent ESP has SPF and DKIM in place, at least on some level. There may be better ways to authenticate, but the boxes are checked.
Yes, reputation is important. But here’s the thing, reputation just means that the ISP knows how users are going to react to an email. Reputation isn’t some nebulous concept made up by ISPs. It’s an actual measurement. It quantifies the history of an IP or a domain or a mail stream and says we know that this IP sends wanted mail. We know that this domain sends mail our users ignore. It’s a history. Past performance does indicate future results.
Identity says who a sender is. Reputation tells us that sender’s history of sending. Those are the two factors that enable ISPs to make delivery decisions. Mail comes in and the ISP looks at it. They use identity to determine what reputation to assign to a mail. Reputation drives delivery, whether into the inbox or the bulk folder.

Are they using DKIM?

It’s easy to tell if a domain is using SPF – look up the TXT record for the domain and see if any of them begin with “v=spf1”. If one does, they’re using SPF. If none do, they’re not. (If more than one does? They’re publishing invalid SPF.)
AOL are publishing SPF. Geocities aren’t.
For DKIM it’s harder, as a DKIM key isn’t published at a well-known place in DNS. Instead, each signed email includes a “selector” and you look up a record by combining that selector with the fixed string “._domainkey.” and the domain.
If you have DKIM-signed mail from them then you can find the selector (s=) in the DKIM-Signature header and look up the key. For example, Amazon are using a selector of “taugkdi5ljtmsua4uibbmo5mda3r2q3v”, so I can look up TXT records for “taugkdi5ljtmsua4uibbmo5mda3r2q3v._domainkey.amazon.com“, see that there’s a TXT record returned and know there’s a DKIM key.
That’s a particularly obscure selector, probably one they’re using to track DKIM lookups to the user the mail was sent to, but even if a company is using a selector like “jun2016” you’re unlikely to be able to guess it.
But there’s a detail in the DNS spec that says that if a hostname exists, meaning it’s in DNS, then all the hostnames “above” it in the DNS tree also exist (even if there are no DNS records for them). So if anything,_domainkey.example.com exists in DNS, so does _domainkey.example.com. And, conversely, if _domainkey.example.com doesn’t exist, no subdomain of it exists either.
What does it mean for a hostname to exist in DNS? That’s defined by the two most common responses you get to a DNS query.
One is “NOERROR” – it means that the hostname you asked about exists, even if there are no resource records returned for the particular record type you asked about.
The other is “NXDOMAIN” – it means that the hostname you asked about doesn’t exist, for any record type.
So if you look up _domainkey.aol.com you’ll see a “NOERROR” response, and know that AOL have published DKIM public keys and so are probably using DKIM.
(This is where Steve tries to find a domain that isn’t publishing DKIM keys … Ah! Al’s blog!)
If you look up _domainkey.spamresource.com you’ll see an “NXDOMAIN” response, so you know Al isn’t publishing any DKIM public keys, so isn’t sending any DKIM signed mail using that domain.
This isn’t 100% reliable, unfortunately. Some nameservers will (wrongly) return an NXDOMAIN even if there are subdomains, so you might sometimes get an NXDOMAIN even for a domain that is publishing DKIM. shrug
Sometimes you’ll see an actual TXT record in response – e.g. Yahoo or EBay – that’s detritus left over from the days of DomainKeys, a DomainKeys policy record, and it means nothing today.

ARC: Authenticated Received Chain

On Friday I talked a little about DMARC being a negative assertion rather than an authentication method, and also about how and when it could be deployed without causing problems. Today, how DMARC went wrong and a partial fix for it that is coming down the standards pipeline.
What breaks?

DMARC (with p=reject) risks causing problems any time mail with the protected domain in the From: field is either sent from a mailserver that is not under the control of the protected domain, or forwarded by a mailserver not under the control of the protected domain (and modified, however trivially, as it’s forwarded). “Problems” meaning the email is silently discarded.
This table summarizes some of the mail forwarding situations and what they break – but only from the original sender’s perspective. (If forwarding mail from a users mailbox on provider A to their mailbox on provider-Y breaks because of a DMARC policy on provider-A that’s the user’s problem, or maybe provider-A or provider-Y, but not the original sender’s.)

You're kidding me

All the authentication and DMARC in the world can’t save you from stupid.
I just got a survey request from my bank. Or, at least, it claimed to be from my bank.

Fun with opinions

Over the last few weeks I’ve seen a couple people get on mailing lists and make pronouncements about email. It’s great to have opinions and it’s great to share them. But they’re always a little bit right… and a little bit wrong.

Mail Client Improvements

There’s been extensive and ongoing development of email through the years, but much of it has been behind the scenes. We were focused on the technology and safety and robustness of the channel. We’re not done yet, but things are much better than they were.
The good part of that is there is some space to make improvements to the inbox as well. Over the last few months there have been a number of announcements from different mail client providers about how they’re updating their mail client.

August 2016: The Month in Email

August was a busy month for both Word to the Wise and the larger world of email infrastructure.

A significant subscription attack targeted .gov addresses, ESPs and over a hundred other industry targets. I wrote about it as it began, and Spamhaus chief executive Steve Linford weighed in in our comments thread. As it continued, we worked with M³AAWG and other industry leaders to share data and coordinate efforts to help senders recover from the attack.
In the aftermath, we wrote several posts about abuse, blocklists, how the industry handles these attacks currently, and how we might address these issues going forward. And obviously this has been on my mind before this attack — I posted about ongoing problems with internet security, how open subscription forms contribute to the problem, and other ways that companies inadvertently support phishing operations.
I posted about the history of email, and recounted some of my earliest experiences, when I had a .bitnet and a .gov address. Did you use email before SMTP? Before email clients? I’d be curious to hear your stories.
Speaking of email clients, I did two posts about how mail gets displayed to the end user: Gmail is displaying authentication results, which should provide end users with a bit more transparency about how authentication is used to deliver or block messages, and Microsoft is partnering with Litmus to improve some of the display issues people face using Outlook. These are both notable — if this is not your first time reading this blog, you know about my constant refrain that delivery is a function of sending people mail they want to engage with. If the mail is properly formatted and displayed, and people have a high degree of confidence that it’s been sent from someone they want to get mail from, that goes a long way towards improving engagement in the channel.
On that note, I spoke at length with Derek Harding about how marketers might change their thinking on deliverability, and he wrote that up for ClickZ. I also participated in the creation of Adobe’s excellent Teaching the Email Marketer How to Fish document (no, not phish…).
Steve was very busy behind the scenes this month thinking about abuse-related topics in light of the SBL issues, but he wrote up a quick post about the Traffic Light Protocol, which is used to denote sensitive information as it is shared.
Finally, for my Ask Laura column this month, I answered questions about delivery and engagement metrics and about permissions with purchased lists. As always, if you have a general question about email delivery, send it along and I’ll consider it for the column.

Gmail showing authentication results to endusers

A bit of older news, but worth a blog post. Early in August, Gmail announced changes to the inbox on both the web interface and the android client. They will be pushing authentication results into the interface, so end users can see which emails are authenticated.

These are not deliverability changes, the presence or absence of authentication will not affect inbox delivery. And the gmail Gmail support pages clarify that lack of authentication is not a sign that mail is spam.
This isn’t a huge change for most ESPs and most senders. In fact, Gmail has reported more than 95% of their mail is authenticated with either SPF or DKIM. Now, Gmail does a “best guess” SPF – if it looks like an IP should be authorized to send mail for a domain (like the sending IP is the same as the MX) then it’s considered authenticated.
It’s good to see authentication information being passed to the end user.

Beware the oversimplification

Setting up a DMARC record is the easy bit. Anyone can publish a record in DNS that will trigger reports to them. The challenge is what to do with those reports and now to manage them.
DMARC is a complex protocol. It builds on two other protocols, each with their own nuances and implementation issues. I’ve written in the past about what DMARC is, what you need to know to decide if you’re ready for DMARC and walking through whether or not you should publish DMARC. I’ve done talks where it’s taken me 20 minutes and dozens of slides to set up the context for explaining DMARC. Even experienced email folks can have moments where we get confused by some of the nuances.
DMARC is not a passive protocol. DMARC is an active protocol. Even with a p=none record, there is ongoing monitoring and work. Why consume reports if you’re not going to monitor them? The reports are there so that senders can monitor their authentication. If you’re not monitoring, then why waste cycles and bandwidth to receive them? Do you even know if your mail aligns? Can your mail server handle emails with attachments larger than 10MB? Does your mail server block .zip files? All of these things can cause your mail to be rejected and you won’t receive reports.
Postmark has a great post on DMARC and even has some examples of reports.
I know, I know, there’s a lot of fear mongering about how any company not publishing DMARC isn’t going to get to the inbox. We’re not there, yet. We likely won’t be there in the next few years. We may never get there. In any case, it’s much better to actually think about what you’re going to do with DMARC Plus, ISPs are already checking for DMARC style alignment even in the absence of a DMARC record. You don’t have to publish DMARC for this to happen, it already does.
I’ve said it before: publishing a DMARC record is a good idea. But every company needs to take minimal steps to figure out if publishing DMARC, even just to receive aggregate reports, is the right thing for them. It’s not right for every company or domain at the moment.

July 2016: The Month in Email

We got to slow down — and even take a brief vacation — in July, but we still managed to do a bit of blogging here and there, which I’ll recap below in case you missed anything.

At the beginning of the month, I wrote about email address harvesting from LinkedIn. As you might imagine, I’m not a fan. A permissioned relationship on social media does not equate to permission to email. Check out the post for more on mailing social media contacts.
Even people who are collecting addresses responsibly can face challenges. One of the most important challenges to address is paying attention to your existing subscription processes, testing them regularly, evaluating effectiveness and optimizing as needed.
Our most commented-upon post this month was a pointer to a smart writeup about Hillary Clinton’s email server issues. Commenters were pretty evenly split between those who agreed that they see this kind of workaround frequently, and those who felt like regulatory processes do a good job managing against this kind of “shadow IT” behavior. I wrote a followup post on why we see this kind of workaround frequently in email environments, even in regulated industries, and some trends we’re seeing as things improve.
In other election-related email news, we saw the challenges of campaign email being flagged as spam. As I pointed out, this happens to all campaigns, and is nothing unique to the Trump campaign. Still, there are important lessons for marketers here, too, in terms of list management, email content, frequency, and engagement — all of which are inextricably linked to deliverability.
Speaking of spam and engagement, Steve took a look at some clickthrough tracking revealed through a recent spam message I received — and why legitimate marketers should avoid using these sorts of URL referrers.
On the topic of authentication, I wrote a quick post about how seeing ?all in the SPF record tells me one thing: the person managing the record isn’t doing things properly. Need a refresher on authentication? Our most-read blog post of all time can help you out.
And as always, send me your interesting questions and I’ll be happy to consider them as I resume my Ask Laura column in August.

SPF ?all

The most read post on the blog is Authenticating with SPF: -all or ~all. In fact, it’s in the top 5 posts every single day. We still get comments on it, too. Usually from folks who disagree with my recommendations.
I still stand by my recommendations, though. It doesn’t really matter if you choose ~all or -all in your SPF records. Why? No major provider is rejecting mail solely because of a SPF fail. They may bulk the mail, but they won’t reject it. That’s why, in a deliverability context, it doesn’t matter which one you choose.
My one rule for SPF is never use ?all. Just. No. In the spec, ?all is “testing” mode. But it really is a signifier that the person who put the SPF record together doesn’t know what they’re doing. Unless they really are testing, but even then you shouldn’t see ?all on records for weeks or months.
~ or – never ?

Gmail / Apps authentication issues

I’ve seen several reports of unexpected rejections for unauthenticated email to Google over IPv6 today. Unauthenticated mail over IPv6 is a bad idea, but Google usually spam folders it rather than rejecting it.
The Gmail status dashboard is reporting an issue “Some messages sent to consumer Gmail accounts are being rejected due to authentication enforcement” so something isn’t working as intended.

May 2016: The Month in Email

Summer, already? Happy June! Here’s a look at our busy month of May.

I had a wonderful time in Atlanta at the Salesforce Connections 2016 conference, where I spoke on a panel about deliverability. While in Atlanta, I also visited our friends at Mailchimp, and later spoke at the Email Innovations conference in Las Vegas, where I did my best to avoid “explaining all the things”. Since my speaking schedule for 2017 is filling up already, I’m sure I’ll have plenty of opportunity to explain many more of the things over the next year or so. Let me know if there’s an event that might be a good fit for me, either as a keynote speaker or on a panel.
Steve contributed a few technical posts on the blog this month. He mentioned that Google has stopped supporting the obsolete SSLv3 and RC4, and he explored the ARC protocol, which is in development and review, and which will be useful in extending authentication through the email forwarding process.
Meri contributed to the blog this month as well, with a post on the Sanders campaign mailing list signup process. We’ve written about best practices for political campaigns before, and it’s always interesting to see what candidates are doing correctly and incorrectly with gathering addresses and reaching out to supporters.
In other best practices coverage, I pointed to some advice for marketers about authentication that I’d written up for the Only Influencers list, a really valuable community for email marketers. I wrote about purchased lists again (here’s a handy collection of all of my posts on the topic, just in case you need to convince a colleague that this isn’t a great idea). I also wrote about how getting the technical bits right isn’t always sufficient, which is also something I’ve written about previously. I also discussed the myth of using the word “free” in the subject line. As I said in the post, “Single words in the subject line don’t hurt your delivery, despite many, many, many blog posts out there saying they do. Filters just don’t work that way. They maybe, sorta, kinda used to, but we’ve gotten way past that now.”
On a personal note, I reminisced about the early days of mailing list culture and remembered a dear online friend as I explained some of why I care so much about email.
In my Ask Laura column, I covered CAN SPAM and transactional opt-outs. As always, if you have a general question about deliverability that I can answer in the column, please let me know.

Time for Email Innovations!

After a great experience in Atlanta last week, with the Salesforce and Mailchimp folks, I’m heading off again today. This time it’s Las Vegas for the Email Innovations conference hosted by the Only Influencers group.
My talk is coming together nicely. It’s been a bit of a challenge to try and give enough detail to make sense while not overwhelming with technobabble. There were times when I was all

Thankfully I have some great folks around who talked me down and reminded me that there wasn’t a test and I could gloss over some of the details and still make sense. If you want a preview of part of my talk, check out my blogpost from last week at Only Influencers. Understanding the technical: authentication.
Hope to see you there! My talk is in the Education track after lunch on Thursday.

Don't just follow the HOWTO

There are so many moving parts to ensure good email deliverability. Email marketers need to know marketing, they need to know email and they need to know design. The technical bits of email can be a challenge to learn, and many folks who write tutorials and How-Tos write them for a different audience than marketers.
One of the things I’m trying to do is demystify the technical end of email for marketers. Today I talked about authentication in the Only Influencers newsletter. Check it out!
Understanding the technical: Authentication
Authentication in general

A DKIM primer resurrected

I was looking for some references today back in old blog posts. This means I discover some old links are dead, blog posts are gone or moved, and information is lost.
In this case it’s a post by J.D. Falk on deliverability.com. The link is dead (it looks like the whole website is dead), but I found a copy of his post and am reproducing it here. I don’t have permission, because I can’t get permission from him, but the content is extremely useful and I don’t want it lost.

Ask Laura: Can you help me understand no auth / no entry?

Dear Laura,
I’m a little confused by the term “no auth / no entry”. Gmail and other major receivers seem to be moving towards requiring authentication before they’ll even consider delivery.
Does this just mean SPF and DKIM, or does this mean the much more stringent DMARC, as well?
Thanks,
No Shirt, No Shoes, No What Now?

Ask Laura: Do I have to publish DMARC?

February 2016: The Month in Email

Happy March! Here’s a look back at our last month of email adventures.
It was a busy few weeks for us with the M³AAWG meeting in San Francisco. We saw lots of old friends and met many new people — all in all, a success, despite the M³AAWG plague we both contracted. Hot topics at the conference included DMARC, of course, and I took the opportunity to write up a guide to help you determine if you should publish a DMARC policy.
On the subject of advice and guidance, Ask Laura continues to be a popular column — we’ve had lots of interesting questions, and are always looking for more general questions about email delivery. We can’t tackle specifics about your program in this column (get in touch if we can help you with that directly) but we can help with questions like “Will our ESP kick us off for mailing purchasers?” or “Help! I’m confused about authentication.”
Continuing on the authentication front, I noted that Gmail is starting to roll out some UI to indicate authentication status to users. It will be interesting to see if that starts to affect user (or sender) behavior in any way. In other interesting industry news, Microsoft has implemented an Office 365 IP Delisting page. I also wrote a followup post to my 2015 overview of the state of ESPs and purchased lists — it’s worth checking out if this is something your business considers.
I wrote a post about security and backdoors, prompted by both the FBI/Apple controversy and by Kim Zetter’s talk at M³AAWG about Stuxnet. These questions about control and access will only get more complicated as we produce, consume, store, and share more data across more devices.
Speaking of predictions, I also noted my contribution to a great whitepaper from Litmus that explores the state of Email Marketing in 2020.
As always, we looked at some best practices this month. I wrote up some of my thoughts about data hygiene following Mailchimp’s blog post about the value of inactive subscribers. As always, there isn’t one right answer, but there’s a lot of good food for thought. And more food for thought: how best practices are a lot like public health recommendations. As with everything, it comes down to knowing your audience(s) and looking at the relationship(s), which, as you know, is a favorite subject around here.

Catching up from MAAWG SF

Had a great time a M³AAWG last week. So many familiar faces and a lot of new ones, too. I’ve got a lot of interesting stuff that I can share with readers over the next few days.
One of the things I have received permission to share is the new Office 365 IP delisting link. I botched the first time I posted it, so I’m going to try again. Office 365 IP Delisting Page. Many thanks to the Microsoft guys for getting this together for people.
While I’m talking about Microsoft, there is a bit of a problem with folks signing up with their FBL. Some people are finding that the process gets stuck and FBLs aren’t enabled. MS is aware of the issue and they are working on fixing it. As I know more I’ll share.
Unsurprisingly, authentication was a big topic of conversation, both in the hallways and in the sessions. There were some strong opinions stated. I think, though, that we’re pretty clear that we’re going to get to a more authenticated world. But we have some different opinions on how and how fast that’s going to happen.

Should you publish DMARC?

I’ve been hearing a lot lately about DMARC. Being at M³AAWG has increased that. Last night we were at dinner and heard from the next table “And they’re not even publishing DMARC!!!!”
I know DMARC is the future. I know folks are going to have to start publishing DMARC records. I also know that the protocol is the future. I am also not sure that most companies are ready for DMARC.
So lets take a step back and talk about DMARC, what it is and why I’m still a little hesitant to jump on the PUBLISH DMARC NOW!! bandwagon.

Ask Laura: Confused about Authentication

What to expect in 2016

I don’t always do predictions posts, even though they’re popular. Most years I skip them because I don’t see major changes in the email space. And, I’m not the type to just write a prediction post just to post a prediction.
This year, though, I do see changes for everyone in the email space. Most of them center on finally having to deal with the technical debt that’s been accumulating over the past few years. I see ISPs and ESPs spending a lot of development effort to cope with the ongoing evolution authentication requirements.
When people started seriously looking at how to authenticate email, the first goal was getting organizations to implement the protocols. This was a practical concession; in order for a new protocol to be used it needs to be widely implemented. Phase one of authenticating email was simply about publishing protocols and getting organizations to use them.
During phase one, the organization that authenticated a mail hasn’t been important. In fact, the SPF spec almost guarantees that the ESP domain is the authenticated domain. In DKIM, the spec says any domain could sign as long as they could publish a public key in that domain’s domainkeys record.
ESPs took full advantage of this and lowered their own development overhead by taking most of the authentication responsibility on themselves. Their domains were in the 5321.from and they published the SPF records. Domains they control were in the d= and they generated and published the DKIM keys. Mail was authenticated without ESP customers having to do much.
We’ve hit the end of phase one. Most of the major players in the email space are authenticating outbound email. Many of the major players are checking authentication on the inbound. Phase one was a success.
We’re now entering phase two, and that changes thing. In phase two, SPF and DKIM are used as the foundation for user visible authentication. Neither SPF nor DKIM were designed to be user visible protocols. To understand what they’re authenticating you have to understand SMTP and email. Even now there are days when I begin talking about one of them and have to take a step back and think hard about what is being authenticated. And I use these things every day!
DMARC is the first of these end user visible protocols built on SPF and DKIM. It uses the established and widespread authentication to validate the user visible from address. This authentication requires that the d= value or the 5321.from address belong belong to the same domain in the visible from address. While you can pick whether the alignment between the visible from and the authentication is “strict” or “relaxed” you have no choice about the alignment.
Prior to DMARC no one really paid much attention to the domain doing the authentication. Authentication was a yes or a no question. If the answer was yes, then receivers could use the authenticated domain to build a reputation. But they weren’t really checking much in the way of who was doing the authentication.
In the push to deploy authentication, ESPs assumed the responsibility for authentication deployed ESPs took the responsibility and did most of the work. For many or most customers, authentication was as simple as clicking a checkbox during deployment. Some ESPs do currently let customers authenticate the mail themselves, but there’s enough overhead in getting that deployed that they often charged extra to cover the costs.
DMARC is rapidly becoming an expectation or even a full on requirement for inbox delivery. In order to authenticate with DMARC, the authenticating domain must be in the same domain space as the visible from. If senders want to use their own domain in the visible from, DNS records have to be present in that domain space. Whether it’s a SPF TXT record or a domainkeys record the email sender customer needs to publish the correct information in DNS. Even now, if you try to authenticate with DKIM through google apps, they require you to publish DNS records.
ESPs aren’t in a situation where they can effectively manage authentication alignment for all their customers. Hosting companies are in even worse shape when it comes to letting customers authenticate email. Developers are facing the fact they need to go back and rework their authentication code. Businesses are facing the fact they need to change their processes so customers can authenticate with DMARC.
It’s not just the infrastructure providers that are facing challenges with authentication. Senders are going to discover they can no longer hand authentication off to their ESPs and not worry about it. They’re going to have to get DNS records published by their own staff.
Getting DNS updates through some big companies is sometimes more difficult than it should be. I had one client a few years ago where getting rDNS changed to something non-generic took over a month. From an IT standpoint, changing DNS should require approvals and proper channels. Marketers may find this new process challenging.
And, if organizations want to publish reject policies for their domains, then they will have to publish records for every outside provider they use. Some of those providers can’t support DMARC alignment right now.
In 2016 a lot of companies will discover their current infrastructure can’t cope with modern authentication requirements. A lot of effort, both in terms of product development and software development, will need to be spent to meet current needs. This means a lot of user visible features will be displaced while the technical debt is paid.
These changes will improve the security and safety of email for everyone. It won’t be very user visible, which will give the impression this was a slow year for email development. Don’t let that fool you, this will be a pivotal year in email.

Confusing the engineers

We went camping last weekend with a bunch of friends. Had a great time relaxing on the banks of the Tuolumne River, eating way too much and visiting.
On Saturday I was wearing a somewhat geeky t-shirt. It said 554: abort mission. (Thank you MessageSystems). At some point on Saturday every engineer came up to me, read my shirt and then looked at me and said “That’s not HTTP.”
That lead to various discussions about how their junior engineers don’t actually know SMTP at all. Why? Because the SMTP libraries just work. Apparently the HTTP libraries aren’t that great, so folks have to learn more about HTTP to troubleshoot and use them.
I’m sure there’s a joke in there somewhere: A Kindle engineer, an Android engineer and a robot engineer walk into a campsite…
It did leave me thinking, though, about how it’s not that easy to run your own mail server these days. Gone are the days when running your own server was cost effective and easy. These days, there is just too much spam coming in. Crafting filters is a skilled job. It’s not that hard to run good filters. But to run good filters takes time to do well.
There are also a lot of challenges to sending mail. One of the discussions I had at the campsite was how hard it was to configure outbound mail. The engineer was helping a friend set up a website and trying to get the website to send notifications to the friend. But without setting up authentication the mail kept silently failing.
Of course, we do run our own mail server. But it’s our job and, in many ways, it keeps us honest. We don’t run many filters meaning we see what spammers are doing and can use our own experiences to better understand what commercial filters are dealing with.
For most people, though, I really think using a service is the right solution. Find one with filters that meet your needs and just pay them to deal with the headache.

IPv6 and authentication

I just saw a post over on the mailop mailing list where someone had been bitten by some of the IPv6 email issues I discussed a couple of months ago.
They have dual-stack smarthosts – meaning that their smarthosts have both IPv4 and IPv6 addresses, and will choose one or the other to send mail over. Some domains they send to use Office 365 and opted-in to receiving mail over IPv6, so their smarthosts decided to send that mail preferentially over IPv6.
The mail wasn’t authenticated, so it started bouncing. This is probably going to happen more and more over the next year or so as domain owners increasingly accept mail over IPv6.
If your smarthosts are dual stack, make sure that your workflow authenticates all the mail you send to avoid this sort of delivery issue.
One mistake I’ve seen several companies make is to have solid SPF authentication for all the domains they send – but not for their IPv6 address space. Check that all your SPF records include your IPv6 ranges. While you’re doing that keep in mind that having too many DNS records for SPF can cause problems, and try not too bloat the SPF records you have your customers include.

The holiday mailing season

We’re half way through September and it seems way too early to start thinking about the holidays. But for marketers, even email marketers, planning should be starting now. This planning shouldn’t just be about content and targeting and segmentation, but should also cover deliverability.
Most retailers use email marketing to drive traffic to their websites during the holidays. Experian reported that in 2014 email was the second largest driver of traffic, behind search, to the Hitwise Retail 500. In recent years, though, some retailers have run afoul of filters during the holiday season, losing precious opportunities to reach potential buyers due to delivery problems.
Retailers should consider deliverability as a factor in their marketing strategy.
Choices about who, how, how much and when to email can and do significantly affect marketing. The good news is that smart marketers can use their understanding of filters as part of their strategic planning and avoid some of the bigger problems that have plagued retailers in the past.
In December 2012, retailers Gap and Gilt were listed on the Spamhaus Block List. Since then, other retailers have also had delivery and blocking problems during the holiday season, although none have been quite so public.
Delivery problems can have a significant impact on a retailer’s bottom line. Mark Zadon, the chairman of Zulily, blamed his company’s lower profits in Q3 2014 on changes at their unspecified email service provider. After that announcement, Zulily’s stock value dropped 15%. Zulily isn’t the only company to have email delivery problems affect business growth enough to be mentioned in SEC filings. “Various private spam blacklists have in the past reduced, and may in the future reduce, the effectiveness of our solutions and our ability to conduct our business, which may cause demand for our solutions to decline.”
Deliverability rules don’t change.
Some people argue that the increase in blocking during the holiday season is because the folks running the filters are attempting to sabotage retail marketing. The available evidence doesn’t support this conclusion. For webmail providers and consumer ISPs, the overarching rule for filters is to give users email they want and filter email users don’t want. The processes and techniques the ISPs and filter companies use don’t change during the holidays. A few years ago Return Path interviewed people at a number of providers and all agreed that the receivers don’t change during the holidays.
It is true that during the holiday season some retailers see an increase in delivery problems. These are mostly self-inflicted. The good news is that given the changes are happening at the sending end, there are things senders can do to minimize the impact of filters. It’s all in their control.
Mail volume increases for multiple reasons.
The volume of transactional email goes up because brick-and-mortar retailers collect addresses in the store and email receipts to shoppers. This often involves the shopper spelling out the address for a harried sales associate in the middle of a store blasting holiday music. Typos can, and do, happen. Even when shopping online, from the comfort of the couch, there is a risk of a mis-typed email address.
These typos hurt deliverability a few different ways. The receipt can go to the wrong person, causing a complaint and hurting the reputation of the sender. The receipt can go to a non-existent account, causing a bounce and hurting the reputation of the sender. Both of these things happen, and can hurt delivery if they happen in significant enough numbers. Of even more concern is when a receipt goes to a spamtrap. Enough trap hits or complaints and the sender risks blocking and delivery failures at one or more ISPs.
Many of the larger brick-and-mortar retailers have implemented processes to reduce the chance of bad addresses. Some ask the shopper to input their email address right into the credit card pad. Others show the address to the user on the register and have the user confirm it. These things do help lower the risk of problems and incorrect addresses. But they don’t resolve it completely. Verification services can weed out undeliverable addresses, but can’t really do anything to make sure a deliverable address is the right one.
Transactional email isn’t the only reason volume increases during the holiday season. The volume of marketing email goes up as well. Marketers increase their frequency, sometimes to ridiculous amounts. A few years ago, I was on a list for a cooking store. They increased their volume from 2x a week to 3x a day in the 3 weeks leading up to Thanksgiving. This may make perfect sense from their point of view, but some recipients just don’t want that much email.
In addition to increasing volume to current and engaged customers, retailers often look to older, unengaged lists during the holidays. This has a double negative effect. First, addresses that have gone dormant, whether they bounce or not, can drive reputation down. Second, sending to people after a long period of no email can result in increased complaint rates. Increased complaints, increased bounces, and increased email to abandoned addresses all drive reputation down.
Taken together it’s no wonder some retailers see an increase in deliverability problems during the holiday emailing season. The good news is that mailers have the ability to control and manage their deliverability, even as they manage the holiday volume.

Check your tech

One of the things we do for just about every new client coming into WttW is have them send us an email from their bulk mail system. We then check it for technical correctness. This includes things like reviewing all the different From headers, rDNS of the connecting IP, List-Unsubscribe headers and authentication. This is always useful, IMO, because we often find things that were right when they were set up, but due to other changes at the customer they’re not 100% correct any more.
This happens to most of us. Even a company as small as Word to the Wise misses a rDNS update here or a hostname change update there when making infrastructure changes. That’s even when the same people know about email and are responsible for the infrastructure.
One of the most common problems we see is a SPF record that has accumulated include: files from previous providers. There are a couple reasons for this. One is the fact that SPF is set up while still at the old provider in anticipation of moving to the new provider. Once the move is made no one goes back to clean up the SPF record and remove the old entries. The other reason is that a lot of tech folks don’t like to delete things. Deleting things can lead to problems, and there’s no harm in a little extra in the SPF record. Except, eventually, there are so many include files that the lookup fails.
Every mailer should schedule a regular tech audit for their mail. Things change and sometimes in the midst of chance we don’t always catch some of the little details.

DMARC=BestGuessPass

Looking at the headers within the mail received with my Office365 domain I see dmarc=bestguesspass. BestGuessPass? That’s a new.
Authentication Results
A few days after seeing dmarc=bestguesspass, Terry Zink at Microsoft posted an explanation. Exchange Online Protection, the filtering system for Office365, is analyzing the authentication of incoming emails and if the domain is not publishing a DMARC record, EOP attempts to determine what the results would be if they did. If an email is received that is not authenticated with either SPF or DKIM, the dmarc= results show none just as it always had. DMARC=BestGuessPass will appear if the message is authenticated and the matching authenticated domain does not have a DMARC record.
Having this information is helpful to see what the results would be before setting up a DMARC record. If you are seeing dmarc=bestguesspass when your mail is sent to an Office365 address and you are considering DMARC, the next step would be to publish a p=none DMARC policy and begin to document where your mail is being sent from. P=none will not have an impact on your delivery and asks the receiving mail server to take no action if a DMARC check fails. Once you have setup SPF and DKIM for your mail, p=none policy gives you the ability to begin receiving failure reports from receiving mail servers when unauthenticated mail is sent from your domain.

Four things to check before your next mailing

Like many bits of technology, email is often set-and-forget. Everything is checked and rechecked during setup, and then no one goes back and looks at it again. But mail programs are not static, and people make changes. These changes don’t really break things, but over time they can create their own set of problems.
Setting aside some time every quarter or even every year to check and make sure all the bits of mail are configured correctly is a good idea.

April 2015: The Month in Email

We started the month with some conversations about best practices, both generally looking at the sort of best practices people follow (or don’t) as well as some specific practices we wanted to look at in more depth. Three for this month:

Authentication and Repudiation

Email Authentication lets you demonstrate that you sent a particular email.
Email Repudiation is a claim that you didn’t send a particular email.

SPF is only for email authentication¹
DKIM is only for email authentication
DMARC is only for email repudiation

¹ SPF was originally intended to provide repudiation, but it didn’t work reliably enough to be useful. Nobody uses it for that now.

Email Authentication in a nutshell

There are 3 types of authentication currently in use for email.

Office365 checking DMARC on the inbound

According to a recent blog post, Office365 is starting to evaluate incoming messages for DMARC. I talked a little bit about DMARC in April when Yahoo started publishing a p=reject message.

Spam, Phish or Malware?

Some mornings I check mail from my phone. This showed up this morning.

My first thought was “oh, no, Pizza Hut is spamming, wonder who sold them my address.”
Then I remembered that iOS is horrible and won’t show you anything other than the Friendly From and maybe it was some weird phishing scheme.
When I got to my real mail client I checked headers, and sure enough, it wasn’t really from Pizza Hut. I’m guessing actually malware, but I don’t have a forensics machine to click the link and I’m not doing it on anything I can’t wipe (and have isolated from the rest of my network).
The frustrating thing for me is that this is an authenticated email. It not from Pizza Hut, the address belongs to some company in France. Apparently, that company has had their systems cracked and malware sent through them. Fully authenticated malware, pretending to be Pizza Hut, and passing authentication on various devices.
Pizza Hut isn’t currently publishing a DMARC record, but in this case, a DMARC record for Pizza Hut wouldn’t matter. None of the email addresses in the headers point to Pizza Hut.
I spent last week listening to a lot of people discussing DMARC and authentication and protecting people from scams and headers. But those all the protocols in the world won’t protect against this kind of thing. Phishing and malware can’t be fixed by technology alone. Even if every domain on the planet published a p=reject policy, mail like this would still get through.

What is SPF?

Sender policy framework (SPF, RFC 7208) is an authentication process that ties the 5321.from (also known as the mail from, envelope from or return path) to authorized sending IP addresses. This authorization is published in a TXT record in DNS. Receivers can check SPF at the beginning of a SMTP transaction, compare the 5321.from domain to the connecting IP address and determine if that IP is authorized to transmit mail.

Target did indeed do a blast to customers to offer one year of free credit monitoring. The problem is scammers are also on the prowl and are sending out similar emails.
Target even says it has identified and stopped at least 12 scams preying on consumers via email, Facebook and other outlets.CNN: Did you get an email from Target?
Read More

SPF Fail: too many DNS lookups

I’ve had a couple folks come to me recently for help troubleshooting SPF failures. The error messages said the SPF record was invalid, but by all checks it was valid.
Eventually, we tracked the issue down to how many include files were in the SPF record.
The SPF specification specifically limits the number of lookups that can happen during a SPF check.

DMARC: Please Be Careful!

(Cross posted from Spam Resource.)
Every couple of days, somebody new pops up on the DMARC-Discuss mailing list to ask some question or share an observation. It’s great to see people interested and joining the conversation. Clearly, DMARC interest and adoption are growing. What’s really frustrating, though, is that for about a quarter of the new subscribers, their first mailing list message goes to the spam folder in my Gmail account. It has become sort of an intelligence test I apply to new subscribers — I’ve stopped digging those messages out of the spam folder. I’m figuring that if they can’t figure out how to implement a DMARC record, or they don’t understand that it’s not really compatible with mailing lists nor is it meant for hobbyist domains, then I think perhaps they’ve got some things they’ve got to figure out before they’re ready to join the discussion.
To that end, let me take a moment to jot down some recommendations for folks who are considering implementing DMARC.

New player in the DMARC space

Over on the DMARC-Discuss list, Comcast announced they had turned on DMARC validation and companies that publish DMARC records should start receiving reports from Comcast.

Hotmail moves to SPF authentication

Hotmail has recently stopped using Sender ID for email authentication and switched to authenticating with SPF. The protocol differences between SenderID and SPF were subtle and most senders who were getting a pass at Hotmail were already publishing SPF records.
From an email in my inbox from September:

Gmail sending out warnings for 512 bit DKIM keys

As an update to yesterday’s post, Gmail is contacting postmasters at domains signing with 512 bit keys to warn them of the upcoming changes. This message also clarifies “DKIM keys failing.” Messages signed with 512 bit keys or less will be treated as unsigned by Gmail in the next week or so.

Is Google failing DKIM keys shorter than 512 bits?

Today’s Wednesday question comes from Andrew B. and got pushed to Thursday so I could check a few more facts.

Outlook.com in practice

I’ve seen a few people talking about outlook.com and how it’s working. There aren’t many insights here but there are a couple.

Getting rid of the via at Gmail

There was a question submitted today about the verification process at Gmail.

DMARC: an authentication framework

A new email industry group was announced this morning. DMARC is a group of industry participants, including large senders, large receivers and relevant intermediaries working on a framework to reduce the harm from phishing.
DMARC is working on a standard to allow senders to publish sending policies and receivers to act on those policies. Currently, senders who want receivers to not deliver unauthenticated email have to negotiate private agreements with the ISPs to make that happen. This is a way to expand the existing programs. Without a published standard, the overhead in managing individual agreements would quickly become prohibitive.
It is an anti-phishing technique built on top of current authentication processes. This is the “next step” in the process and one that most people involved in the authentication process were anticipating and planning for. I’m glad to see so many big players participating.

Gmail and the via

I was hoping to have a detailed post up today about the conditions where gmail presents the user with a “via” but time seems to have gotten away from me. But I can give you the conclusions.

Authentication Cheat Sheet

There are a several approaches to authenticating email, and the different authentication methods have a lot of different settings to choose from (sometimes because they’re useful, other times just because they were designed by committee). It’s nice that they have that flexibility for the complex situations that might benefit from them, but almost all the time you just want to choose a good, default authentication approach.
So here’s some short prescriptive advice in no particular order for “how to do email authentication at an ESP well” without the long discussions of alternative approaches and justification of each piece of advice.

Gmail shows authentication data to the recipient

Yesterday Gmail rolled out some changes to their interface. One of the changes is that they are now showing end users authentication results in the user screen.
It’s really the next step in email authentication, showing the results to the end user.
So how does Google do this? Google is checking both SPF and DKIM. If mail is authenticated and the authentication matches the from address then they display the email as:

If we click on “details” for that message, we find more specific information.
In this case the mail went through our outgoing mailserver to gmail.
Mailed-by indicates that the message passed SPF and that the IP address is a valid source of mail from wordtothewise.com.
Signed-by shows the domain in the DKIM d=. In this case, we signed with the subdomain dt.wordtothewise.com. That’s what happens when you sign using the domain in the From address (or a subdomain of it).
For a lot of bulk senders, though, their mail is signed using their ESP’s domain instead. In that case Gmail shows who signed the mail as well as the from address.

And when we click on “details” for that message we see:
This is an email from a sender using Madmimi as an ESP. Madmimi is handling both the SPF authentication and the DKIM authentication.
As an aside, this particular sender has a high enough reputation that Gmail is offering me an unsubscribe option in their interface.
Gmail is distinguishing between first party and third party signatures in authentication. If the mail is authenticated, but the authentication appears to be handled by a separate entity, then Gmail is alerting recipients to that fact.
What does this mean for bulk senders?
For senders that are signing with a domain that matches their From: domain, there is no change. Recipients will not see any mention of your ESP in the headers.
However, if you are using an ESP that is signing your mail with a domain they own, then your recipients will see that information displayed in the email interface. If you don’t want this to be displayed by Gmail, then you will need to move to first party signing. Talk to your ESP about this. If they’re unsure of how to manage it, you can point them to DKIM Core for an Email Service Provider.
Gmail blogpost about the changes
Gmail help page about authentication results

Defending against the hackers of 1995

Passwords are convenient for the end user, but it’s too easy to lose control of them. People share them with other people. People write them down, where they can be read. People send them in email, and that email is easily intercepted. People’s web browsers store the passwords, so they can log in automatically. Worst of all, perhaps, people tend to use the same username and password at many different websites. If just one of those websites is compromised (or even run as a password collecting scam) then those passwords can be used to attack accounts at all of the others.
Two factor authentication that uses an uncopyable physical device (such as a cellphone or a security token) as a second factor mitigates most of these threats very effectively. Weaker two factor authentication using digital certificates is a little easier to misuse (as the user can share the certificate with others, or have it copied without them noticing) but still a lot better than a password.
Security problems solved, then?

What is Two Factor Authentication?

Two factor authentication, or the snappy acronym 2FA, is something that you’re going to be hearing a lot about over the next year or so, both for use by ESP employees (in an attempt to reduce the risks of data theft) and by ESP customers (attempting to reduce the chance of an account being misused to send spam). What is Authentication?
In computer security terms authentication is proving who you are – when you enter a username and a password to access your email account you’re authenticating yourself to the system using a password that only you know.
Authentication (“who you are”) is the most visible part of computer access control, but it’s usually combined with two other A’s – authorization (“what you are allowed to do”) and accounting (“who did what”) to form an access control system.
And what are the two factors?
Two factor authentication means using two independent sources of evidence to demonstrate who you are. The idea behind it is that it means an attacker need to steal two quite different bits of information, with different weaknesses and attack vectors, in order to gain access. This makes the attack scenario much more complex and difficult for an attacker to carry out.
It’s important that the different factors are independent – requiring two passwords doesn’t count as 2FA, as an attack that can get the first password can just as easily get the second password. Generally 2FA requires the user to demonstrate their identity via two out of three broad ways:

Authentication and phishing

Yahoo announced today that they are releasing the Yahoo! Mail Anti-Phishing Platform (YMAP) that will help protect their users from phishing. They have a similar project in place for eBay and PayPal mail, but this will extend to a broader range of companies.

Phishing protection

Last week Return Path announced a new service: Domain Assurance. This service allows companies who send only authenticated email to protect their brand from phishing attacks. Participating ISPs will reject unauthenticated email from domains participating in this program.

ESPs, Non-portable Reputation and Vendor Lock-in

I’ve seen some mentions recently of ESPs suggesting that if you use your own domain in the From: of mail you send through an ESP then that ESP can’t “do email authentication” properly unless they require you to edit your domains DNS settings. That’s not really so, but there is a kernel of truth in there.
The real situation is, unsurprisingly, a bit more complicated.
What authentication features should you look for in an ESP?

Domain Assurance by Return Path

As often happens during MAAWG, email companies are announcing new products. One of the interesting ones is the new Domain Assurance product from Return Path.

Who can you trust?

I’ve been recently dealing with a client who is looking at implementing authentication on their domains. He’s done a lot of background research into the schemes and has a relatively firm grasp on the issue. At this point we’re working out what policies he wants to set and how to correctly implement those policies.
His questions were well informed for the most part. A few of them were completely out of left field, so I asked him for some of his references. One of those references was the EEC Email Authentication Whitepaper.
My client was doing the best he could to inform himself and relies on industry groups like the EEC to provide him with accurate information. In this case, their information was incomplete and incorrect.
We all have our perspectives and biases (yes, even me!) but there are objective facts that can be independently verified. For instance, the EEC Authentication whitepaper claimed that Yahoo requires DKIM signing for access to their whitelist program. This is incorrect, a sender does not have to sign with DKIM in order to apply for the Yahoo whitelist program. A bulk sender does have to sign with DKIM for a Y! FBL, but ISPs are given access to an IP based FBL by Yahoo. I am shocked that none of the experts that contributed to the document caught that error.
Independent verification is one reason I publish the Delivery Wiki. It’s a resource for everyone and a way to share my knowledge and thought processes. But other experts can “check my work” as it were and provide corrections if my information is outdated or faulty. All too often, senders end up blaming delivery problems on evil spirits, or using “dear” in the subject line or using too much pink in the design.
Delivery isn’t that esoteric or difficult if you have a clear understanding of the policy and technical decisions at a range of ESPs and ISPs, the history and reasoning behind those decisions, and enough experience to predict the implications when they collide.
Many senders do face delivery challenges and there is considerable demand for delivery experts to provide delivery facts. That niche has been filled by a mix of people, of all levels of experience, expertise and technical knowledge, leading to the difficult task of working out which of those “experts” are experts, and which of those “facts” are facts.

Goodmail sued for patent infringement

Late last week RPost sued Goodmail for infringing two patents. One patent authenticates content and delivery of documents. The second verifies the message was received by the recipient.
Patent #6,182,219: Apparatus and method for authenticating the dispatch and contents of documents.

DKIM implementation survey: prelim results

First off, I want to thank everyone who participated in the DKIM implementation survey. This week has been pretty hectic so far, so I haven’t had a chance to actually dig down into the data from the survey, but I thought I’d post some preliminary results.
The ESP survey had 45 respondents. 30% of those sent more than 15 million emails a month.
Of all the respondents: 40% are signing with Domain Keys, 51.1% are signing with DKIM.
Of all respondents: 79.5% are signing with Domain Keys and 78.8% are signing with DKIM to access services (whitelists or FBLs) provided by the ISPs.
50% of those not signing with Domain Keys are not doing so because customers have not requested it. 61% of those not signing with DKIM are not doing it because of technical difficulties with deployment.
The ISP survey had 16 respondents, with 37.5% handling less than 500,000 mailboxes and 18.8% handling more than 15 million mailboxes. 75% of respondents said they are not checking Domain Keys on inbound mail. 56% said they are not currently checking DKIM on inbound mail.
Only 10 ISPs answered the question if they plan to check either Domain Keys or DKIM.

Authenticating email in a court of law

Venkat has a discussion of authentication needed to present emails to a judge when asking for a summary judgment.

AOL publishes sender recommendations

In a blog post on April 28, AOL pointed to their new Sender Best Practices document. These are not things a sender must do in order to get mail delivered to AOL, but rather things that will help improve your reputation at AOL.
The recommendations are what I have been recommending for a while and there is nothing overly surprising in the recommendations.

Unauthenticated email

A few weeks ago, NetworkWorld posted an interview with Mark Risher of Yahoo. In it, Mark talked about how Yahoo had no plans to outright block or refuse any unauthenticated email. Of course authentication will be a large part of their decision making for incoming emails but they cannot just refuse to accept mail that is unauthenticated, because there are times when unauthenticated email is the most important mail to their recipients.
A lot of marketers often seem to forget that they are competing for time and space with other, non-marketing, types of email. Email from friends and family and discussion lists are both more important to most people that the latest and greatest email advertisement. These are the emails people want to receive, the ones they open, read and respond to.
In terms of authentication, right now the majority of wanted emails are unsigned with DK or DKIM. Sure there are the early adopters who are using DK/DKIM to sign their emails, and a few large ISPs have started signing outgoing email. But until the vast majority of wanted email is actually signed, recipient ISPs are going to have to accept unsigned email.
Looking forward, even if all of the ISPs sign email sent through their SMTP servers, there will still be some fraction of desired email that will be unsigned. Individuals and small businesses who choose to run their own mailservers may not sign email. Even though these servers make up a tiny fraction of total email, they make up a much larger fraction of wanted email. ISPs cannot block this email without angering their customer base.
Marketers should not be concerned about ISPs blocking unauthenticated email, as it is extremely unlikely that any major ISP will do that. Marketers should focus, instead on making their email relevant and wanted by the recipients. I have been recommending clients plan to have all their outgoing emails DK/DKIM signed by the end of 2008.

Email authentication

The great folks over at MailChimp have compiled a list of which authentication methods (DK, DKIM, SPF and SenderID) are in use at which ISPs.
Good stuff and very clear showing who is using what authentication.

Predictions for 2008

I did not have a lot of predictions for what will happen with email at the beginning of the year so I did not do a traditional beginning of the year post. Over the last 3 – 4 weeks, though, I have noticed some things that I think show where the industry is going.
Authentication. In January two announcements happened that lead me to believe most legitimate mail will be DK/DKIM signed by the end of the year. AOTA announced that approximately 50% of all email was currently authenticated. They did not separate out SPF/SenderID authentication from DK/DKIM authentication, but this still suggests email authentication is being widely adopted. AOL announced they will be checking DKIM on their inbound mail. I expect more and more email will be DKIM signed in response to this announcement.
Filtering. The end of 2007 marked a steady uptick in mail being filtered or blocked by recipient domains. I expect this trend to continue throughout 2008. Recipient domains are rolling out new technology to measure complaints, evaluate reputation and monitor unwanted email in ways that tease out the bad actors from the good. This means more bad and borderline email will be blocked. Over the short term, I expect to see more good email blocked, too, but expect this will resolve itself by Q2/Q3.
Sender Improvements. As the ISPs get better at filtering, I expect that many borderline senders will discover they cannot continue to have sloppy subscription practices and still get their mail delivered. Improved authentication and better filtering let ISPs pin-point blocks. Instead of having to block by IP or by domain, they can block only some mail from a domain, or only some mail from an IP. There are a number of senders who are sending mail that users do not want mixed with mail that recipients do want. Right now, if there is more mail that recipients want in that mix, then ISPs let the mail through. This will not continue to happen through 2008. Senders will need to send mail users actively want in order to see good delivery.
Less is more. A lot of other email bloggers have talked about this, and I will echo their predictions. Less email is more. Send relevant mail that your customers want. Target, target, target. Good mailers will not send offers to their entire database, instead they will send mail to a select portion of their database.
Feedback loops. Use of feedback loops by recipient domains will continue to grow.
Mobile email. More recipients will be receiving email on mobile devices.
Suggestions for 2008

DKIM "i=" vs "d=" and Reputation

This really should be part seven of a twelve part series or some such as it deals with an aspect of DKIM that’s really important, but is way down in the details of implementation. (dkim.org is a reasonable place to start for a general overview of DKIM).
There’s an apparently endless thread on the DKIM-SSP spec development mailing list at the moment about the differences between two fields in a DKIM signature that could be used to tie a senders reputation to. Several ESP delivery folks asked me to explain what everyone was talking about, and this post is a first cut at that.
“i=” vs “d=”
There are two possible fields in a DKIM signature that could be used to identify the sender of a message, and so to tie a sender history and reputation record to. They are the so-called “i=” and “d=” field, from the syntax used to include them in the signature.

Yahoo blocks unauthenticated PayPal and eBay Mail

Yahoo announced this morning that over the course of the next few weeks Yahoo would roll out a new feature to their email that blocks any unauthenticated email from eBay and PayPal.
In a blog post Nikki Dugan says: