DMARC is the newest of the authentication protocols. It compares the domain in the From: address to the domains authenticated by SPF and DKIM. If either SPF or DKIM pass and they are in the same organizational domain as the domain in the From: address then the email is authenticated with DMARC.
I wrote A Brief DMARC primer back in 2014 – when Yahoo deployed p=reject late on a Friday and caused an awful lot of deliverability folks to spend a weekend trying to figure out why so much of their customer mail was being rejected. I still think that post is pretty good and explains things like alignment well. Over the years I’ve written other DMARC posts, discussing DMARC and some of the issues and complexities of deploying it.123
There is a lot, a LOT of hype about DMARC. There are folks who loudly proclaim that every domain everywhere should be publishing p=reject. Unfortunately, DMARC has a lot of problems, both in the execution and in concept. These problems are a major reason that so many domains have avoided p=reject.
One of the topics suggested for deliverability week was the title of the post, so I decided to take a shot at it and talk about some of the fundamental problems with DMARC and why so many of us are not on the p=reject will save the world bandwagon.
The Good
There are a couple good things in the DMARC protocol.
Reporting
DMARC provides a way for domain owners to ask for reports about email using their domain name in the From: header. These reports are useful. They allow domain owners to see authentication (SPF and DKIM) as well as the IP addresses sending those messages. These reports are designed to be machine readable. They are sent as XML and require a small amount of processing to present the data in a consumable and actionable format. There are a number of services that will process DMARC reports for prices ranging from free to expensive.
Personally, I find reports aren’t just useful for authentication, but they also help diagnostics when I’m trying to address a client’s delivery problems. Often there is no central person at a company who knows all the email sources. Marketing cares about marketing email, IT cares about corporate email, Ops cares about ops email, but no one really knows where all the different sources are. DMARC reports allows one person to see all the sources of mail using the domain in the From: header. Reports also give information about 3rd parties using domains in some, but not all, cases.4
Alignment
DMARC defined and codified alignment between different domains in the header. Specifically between the authenticated domains (SPF domain and DKIM d=) and the domain in the From: address. I think before DMARC many of us were talking generally about the SPF authentication or the DKIM authentication matching the domain the recipient saw. Due to the complexity of domain names, this is often easy to do manually but challenging to do mechanically. DMARC codified the mechanical methods of determining alignment.
The OK
The other official part of DMARC allows domain owners to make policy statements in DNS. The request statuses are “none” “quarantine” and “reject”. Receiving domains can check these requests at delivery time and use that as part of their delivery decision pathway.
- None means: take no action based on the authentication but still send me reports so I can monitor mail From: my domain.
- Reject means: we want you to reject any mail that comes through if it doesn’t comply with DMARC. This mail, even if we sent it, we don’t want delivered and you should reject it.
- Quarantine means: that mail that fails to authenticate should be treated suspiciously. What that means is up to the receiving system to determine. 5
The underlying justification for p=reject statements is that it prevents unauthorized use of the domain name. Initially it was sold as a way to stop phishing. The reality is that it doesn’t do that except in cases where the phishing directly forges the domain in question. In practice, phishers just moved on to using cousin domains. When that stopped working they started moving on to other ways of phishing as well.6
The Bad
The above are things that DMARC does well or even OK. But there are a lot of bad bits of DMARC and not many people, including myself, have ever really come out publicly with a list of the problems. Here are a few of the bad things about DMARC.
DMARC makes it trivially easy to shoot yourself in the foot.
There are dozens of stories I’ve heard over the years where some company decided to implement DMARC and jumped directly to p=reject without actually doing their homework. One story told at a DMARC deployment workshop a long time ago involved a bank and going p=reject the day before their (unaligned) consumer bank statements went out. I’m sure this has happened more than once.
I have helped clients clean up from DMARC deployment failures. One memorable case was a large kid-focused company that provides creative supplies and runs a number of theme parks throughout the US. Their parent company decreed all their subsidiaries would publish p=reject. Unfortunately, the parent company provided no deployment assistance, no education. They just sent down a DMARC record that was simply “v=DMARC1; p=reject”. My client did as they were told, but some of their mail was not aligned. As a result a number of important emails were rejected, including things like amusement park tickets. The company hired me to help clean up and actually align all their mail.
Then there was the data loss at GitLab a few years ago. This was not a single problem, there were a series of events that contributed to this failure. One of the problems, however, was deploying DMARC p=reject without properly authenticating operational email. When the backup processes were failing, cron was appropriately sending email notifications. However, they were never received because of the p=reject policy. This is a common failure mode. Often internal notifications and other operational emails are mission critical, but are overlooked in DMARC deployment.
DMARC breaks forwarding
By design, DMARC only allows for mail to come from authenticated sources. But SMTP isn’t a point-to-point protocol. In the SMTP spec the receiving mailserver takes responsibility for delivery once the transaction is over. This means they now ‘own’ the message and can forward it, modify it or otherwise change it.
There are a lot of receiving mailservers that change messages. Many companies run email security appliances that modify links or change the subject line of a message or add a banner at the top that says “External mail”. These are valid and legal modifications of the messages. However, these modifications break DKIM. If a mail is forwarded the message then fails DMARC. When the original domain publishes a p=reject, that message then fails – even though the message is legitimate and was correctly authenticated at the time of send.7
One of my clients provides a security notification system for IoT companies. Their system monitors Internet connected devices after they’ve been purchased. The companies manufacturing these systems set up email alerts based on a complex set of criteria. When the criteria are met, then my client sends mail to their customers. Based on DMARC reports, some of these customers appear to be tagging and modifying the messages and then forwarding the mail back out, presumably to their engineering teams.
Digging into the failures, they’re always the same sources: Office365 and an anti-virus platform account for the vast majority of failures. Incidentally, I can see some of these failures in delivery logs so I even know some of the companies involved. These are very large, multi-national brands with products many people have in their homes.
There’s nothing my client can do other than not publish p=reject. Continuing to publish p=reject means certain customers of theirs will not be notified of problems when mail is forwarded out. This could lead to a Gitlab style disaster and I recommended client switch the notification mail to p=quarantine.
DMARC breaks mailing lists
Mailing lists work by accepting email from a list member, and then forwarding out that email to all the other list members. As with forwarding, sometimes the mailing list modifies the message, appending list specific footers and adding subject line tags, before sending the message out to the full list.
List members posting from a domain with p=reject results in other list members, whose mailservers respect the p=reject, being removed from the mailing list due to bouncing email. To cope with this, many mailing lists have started rewriting the From: address of list members. This breaks normal list functionality in some significant ways.
- It’s harder, and sometimes impossible, to reply directly to the sender in many common list configurations. The Reply-To: address is set to the list so just replying to the message doesn’t go to the sender. Some lists don’t maintain any indication of the original poster’s address, so you can’t find out who sent it. There’s even a couple lists where I’m on where the posters don’t sign their emails, so it’s actually impossible to know who made the post due to header rewriting.
- It’s harder, and often impossible, to search for list mail from a particular sender. This sounds trivial but I actually keep list mail and search for mail from particular people regularly. Or, I did, until everyone posting to the list has the exact same address and it’s impossible to find “that post Steve made.”
Mailing lists are a communication tool in both personal and professional spaces. I was recently at a conference where their value was wholly dismissed by panelists promoting p=reject. One of their arguments was “the volume is very small, it only affects a few people.” As someone who came from the science end of things, I can guarantee you that scientists use mailing lists to communicate about things like emerging viruses and pandemics. I am even willing to bet an expensive bottle of whisky that mailing lists were an utterly vital communications channel used to share data about recent viruses and vaccines. There probably were no more than 1000 people on those lists, but their conversations, their data sharing, their existence literally saved peoples lives.
The Ugly
As I said above, DMARC p=reject breaks email in deeply fundamental ways. I could be convinced this breakage was worth it if there were clear indications that DMARC p=reject actually stopped bad behavior. But there is very little data on the ground that p=reject provides more value than it destroys.
There’s also the very ugly truth that many of the folks pushing DMARC, and even those working on the DMARC protocol through the IETF, are directly profiting from pushing DMARC p=reject on the rest of us. The founded or worked for or are C*O of companies that sell monitoring or deployment services. At best it’s a conflict of interest, particularly when those folks shut down any discussions of the down sides or the problems with p=reject.
All of the issues I’ve mentioned here, and a host of others I’ve not included in the interest of length, are considered unimportant by the people profiting off DMARC p=reject. These issues have been raised, by myself and others, time and time again, only to be waved off. Proponents of p=reject have gone so far as to say that mailing lists are “spoofing” mail – and that the true author of a message is the mailing list, not the person who actually created the mail. How does that even make sense? The author is the person that put the thoughts to writing, not the transport mechanism that shares it.
Last October I was at a conference where the CEO of a DMARC company was speaking about DMARC and why p=reject should be best practice for every domain. He cited the massive costs of phishing. After the panel I asked him if he had any data or case studies backing up his position. Could he demonstrate that deploying p=reject would have prevented the billions of dollars in losses due to phishing? He told me this data was very complicated and they didn’t have that data. But, they might have it in a few months (pssst: it’s been more than a few months).8
Maybe they don’t really have the data, but if they don’t it’s only because they’ve not bothered to look. Even looking at the publicly disclosed phishing losses, it should be trivial to say “this much money was lost due to direct domain phishing.” It’s not, and I believe it’s because there’s a lot of cousin domain phishing going on, and none of that will be impacted by deploying p=reject widely.9
I actually think the truth is much uglier than they haven’t bothered to collect the data. They have looked, they have done the comparison, they have seen that widely deployed p=reject won’t have a huge impact on the losses due to phishing. That there is no business reason to deploy p=reject, particularly for domains used by people not machines. But they don’t want to admit that because it will hurt their bottom line.
Is DMARC worth it?
DMARC reporting is good. In fact, I think there should be an expansion of reporting, so senders can see where their SPF and DKIM domains are also being used in mail. Maybe even include links in the reporting scheme. Over the last few years we’ve seen an uptick in domain abuse that aren’t covered by DMARC and addressing that would lead to a better and safer Internet.
I think there is a subset of mail, including bulk marketing mail, where DMARC p=reject is, at worst, mostly harmless. This mail includes highly automated mail, like receipts or such, that comes from a dedicated subdomain with no human users. In fact, when I first heard about DMARC it was designed exactly for that type of mail. Even in those cases, though, it will cause mail that should be delivered to fail.
When applied to domains used by people, DMARC p=reject imposes costs on the overall email ecosystem and most of those costs aren’t borne by the companies that deploy p=reject. Those costs are externalized and other people and companies bear those costs.
The problem I have with thinking the entire internet should go p=reject is that it breaks so many valid use cases for email. It hinders person to person communication in a fundamental way. Email is not primarily a machine to machine communication system, it’s a human communication system. Email is a communication tool and despite what some think, it’s a person to person or company to person communication tool. Deploying DMARC p=reject on consumer domains, or domains used by people, makes it harder for those people to communicate with others.
DMARC as it is currently promoted and implemented sees the internet as a commercial broadcast channel. I don’t share that vision and I think fundamentally transforming email to prioritize commercial interests is detrimental to people.
- Should you Publish DMARC (Feb 2016) ↩︎
- DMARC and Organizations ↩︎
- Beware the oversimplification ↩︎
- There is ongoing domain hijacking in the SPF string and different attacks against the DKIM d= domain. None of these are included in current DMARC reports as the hijacker uses their own domain in the From:. These attacks do affect domain reputation and can harm delivery, however. ↩︎
- RFC 7489. This was written by some of the original DMARC authors, so I trust it as reflecting their viewpoints and intentions. ↩︎
- That’s a whole, very long post about the different ways phishers continue to operate and will continue to operate even in an wholly p=reject environment. There are other people who are better positioned to write it than me. ↩︎
- There are ongoing discussions about how to deal with this, including a protocol called ARC. Currently, however there is no way to ensure that post delivery modifications don’t break DMARC. ↩︎
- Also during that conversation I told him that if he could show me actual data that supported his position I would become a DMARC evangelist. I’d be mad about it because I’m wrong, but I’d change my mind if there was evidence. ↩︎
- When discussing this post, Steve and I came up with at least 2 other methods that we’d use to demonstrate DMARC did what proponents claim it does. The data should be utterly trivial for a company to collect, if they wanted to. ↩︎
RSS - Posts
Laura wrote “Initially it was sold as a way to stop phishing. The reality is that it doesn’t do that except in cases where the phishing directly forges the domain in question.”
Like many, you are incorrect in your assertion(s).
A little bit of history is in order. The origins of what became DMARC trace back to two workshops the Federal Trade Commission (FTC) held in 2003 and 2004 on Spam and email authentication. Following those events I wrote a roughly 50 page document for internal use only, which outlined various problem spaces and potential approaches for mitigating abuse in those spaces, for my then employer, a large greeting card company. One of the areas covered was direct domain abuse. Senior management allowed me to implement some of my suggestions but held back on others. There were also ongoing discussions between a small group of large corporate senders (primarily large financials and a greeting card company, large mailbox providers and a couple intermediaries, on email abuse and potential mitigation approaches.
Fast forward to January 2007 when The Storm Worm (https://en.wikipedia.org/wiki/Storm_Worm) started running rampant. Roughly 3 months after it was identified, The Storm Worm started pretending to be greeting card notifications. It quickly reached a point where there were roughly 10 times as many fake (and dangerous) notifications as all the legitimate notifications from the entire greeting card industry combined.
Senior management agreed that I could re-architect our processes to protect our mail streams from direct domain abuse, including The Storm Worm. This was a non-trivial set of projects. I assembled a cross functional team that included developers, web operations, security, project managers, marketing, customer service and other groups. This was basically re-architecting business processes at some scale for hundreds of domains. In 6 weeks we rolled out the changes to the first (live) domain. Other domains quickly followed.
At the same time I was working closely with several very large mailbox providers. I was telling them that if emails claiming to be from one of our domains (aligned on the RFC822 From email address domain) failed to pass either SPF or DKIM, they should reject the email (not quarantine). It worked but was initially a very manual process to for mailbox providers to validate emails. Also in this time frame I started posting about this approach in various places like IETF (Internet Engineering Task Force) mailing lists such as dkim-ops. I suggested that both senders and receivers could implement this approach for commercial mail flows.
I should point out that at roughly this same time frame, PayPal was doing something similar with Yahoo! but was not being public about it and was not suggesting it as a general solution for anyone and everyone.
At a later point, a group of large senders, large mailbox providers and a couple intermediaries settled on this approach of mitigating direct domain approach. It worked well for this “private club” but the people involved that it was valuable enough that it was worth proposing an open standard. We started working on the standard in 2009 and it was publicly unveiled in 2011. We later turned it over to the IETF as was always our intent. The nature of the emails covered by DMARC (at that time) did not involve mailing lists or many of the other indirect mail flows. DMARC was a point solution to a point problem.
I never profited from the development of DMARC and neither did most of the others involved in the effort. I was personally asked to be a cofounder at various startups or to become an early stage employee. To me, DMARC was simply a tool that needed to be developed to solve a specific problem.
There’s a lot more to the story but this is a good stopping point.
That’s interesting prehistory, Mike – I was aware of most of it but it’s nice to have it narrated. Thanks!
DMARC changed the day Yahoo deployed it, in an attempt to offload their support costs onto the rest of the industry. Prior to that the assumption by most involved was that it would be used for mail streams that were controlled entirely by a single entity, with financial orgs being a common example, as you say.
After that, though, DMARC was sold to the larger community as a way of stopping phishing, that’s the historical fact.
Nobody is suggesting all the DMARC proponents who spend effort to shout down criticism of that cost shifting have their careers attached in some way to continuing DMARC deployment, but many do. That’s been very obvious to me, and I’m sure it’s even clearer to any women who discuss it publicly.