Industry News & Analysis

Jane! Stop this crazy thing!

One of the consequences of moving to Ireland is I’m unsubscribing from most commercial mail, including some lists I’ve been on for a decade or more. Sadly, many of the companies don’t ship to Ireland, or their shipping costs are prohibitively expensive. Even if I wanted to purchase from them, I couldn’t.

This process has made me realise how horrible many company’s unsubscribe processes are. Look, I get it, having people leave your list is no fun. Losing subscribers is not what we’re in this for. But, sometimes, sometimes you just have to let them go. But there are senders out there that continue to mail me after I have unsubscribed.

In one case, every time I unsubscribe I get a note “you’ve been unsubscribed from company <list name>.” There is no option available for me to unsubscribe from all company publications. At this point I’ve submitted at least 3 separate unsubscribe requests, and the company is still mailing me. This is, in fact, a CAN SPAM violation.

In both the original law and the rulemaking from the FTC opt out requests are to the “sender” of the commercial email message. We can rules lawyer about how different divisions of a company may be different senders, or how different messages are coming from different people inside the company. In this case, though, the email address in the From: line of the message is identical. They aren’t different “senders” they’re the exact same sender.

The DMA fought long and hard to make sure CAN SPAM was an opt-out law. They argued that every company should have  the opportunity to try and sell consumers something. “We call it the ‘one bite at the apple’ rule,” [Patricia Faley of the Direct Marketing Association] says. “Give me one chance to show you what I have to offer you, and if you don’t like it, then I won’t contact you again.”(Congress has hard time stomaching e-mail spam).  Unfortunately, all too many companies forget the won’t contact you again piece.

In fact, just yesterday I received email from the DMA of Northern California that was in blatant violation of their one-bite rule and CAN SPAM. They got the address from me because I spoke on a panel at a meeting back in 2002 or 2003. I never actually opted in, but as part of the event they required every attendee to give them a business card. After I got the first message I unsubscribed. Yes, the unsubscribe request was more than 15 years ago. That doesn’t make it invalid.

Worse for the DMA, the address is now a spamtrap. Knowing who was at the meeting with me, that wasn’t the only address turned into a spamtrap.

If someone goes through the trouble to opt out of your mail, listen to them. Respect their no. Senders who don’t create a preference center need to accept that when a recipient opts out of one email, then the recipient has opted out of all emails. Sure, if you have a preference center, they can pick and choose and maybe they will want to stay on the recipe list without staying on the sales list. But lacking that facility unsubscribe means unsubscribe from everything.

Likewise, opt-outs don’t expire! If someone says to stop mailing them and don’t contact them again, you stop mailing them and don’t contact them again. The DMA should know better. They’re supposed to be industry leaders in best practices. Unfortunately, they failed.

Email only works because senders respect recipients. Both of these examples show marketers that haven’t bothered to actually consider their recipients. You can’t respect someone you haven’t even thought about.


Spamhaus DBL

Over the last few months I’ve gotten an increasing number of questions about the Spamhaus DBL. So it’s probably time to do a blog post about it.

Last year I wrote about the DBL:

DBL is the Domain Block List. It lists domains and not IP addresses. I’ll be honest, I don’t have as much experience with the DBL as with other lists, but I have had a few clients on the DBL.

  • DBL is tied into the CSS.
  • You can get on the DBL without the sending IP being on CSS.
  • DBL makes no judgement on the source of the mail, only the content of the mail

With more clients being on the list, I have a little more information about it.

DBL listings are generated both by automated tools and by manual entries from the Spamhaus folks. Automated listings are the ones most closely tied to CSS listings.

From my perspective, the goal of the DBL is to block domains found in spam being sent from many IP addresses in a way that makes it difficult to address with standard IP based blocks. I believe that the automated DBL listings are generated based on domains found in the content of the email rather than domains found in the headers. However, most of the DBL users match against any domain in the message including those in the headers.

The automated DBL listings are usually the root domain, but it is possible some of the manual listings are more specific and list subdomains.

There is an automated delisting process, but there are limits to the number of times you can delist. Too many delistings and you need to send email and be manually delisted. This can take quite more than 24 hours, in some cases. If you are listed on both the CSS and the DBL you need to ask for delisting for both.

If your domain is on the DBL but your IPs are not on the CSS then then I suggest looking at the possibility that someone is putting your links in spam. It could be web server compromise hosting phishing. Or, if you’re an ESP, maybe a customer  grabbed a tracking link and is using it in mail sent through another provider.

As with all listings, identifying the underlying reason for the listing and fixing the problem is crucial to staying off the list. If you’ve not fixed the problem, the listing will come back. And, eventually, you won’t be able to delist automatically.

No Comments

Twilio acquires Sendgrid

Woke up this morning to the news that Sendgrid has been acquired by Twilio in an all stock deal. This fills a gap in Twilio’s platform, they didn’t seem to have any email capability before.

I’m wondering how this will shake out in the broader email industry. Sendgrid is one of the biggest providers of SMTP services and have a significant customer base. I’ve worked with a lot of Sendgrid customers over the years. Some of them are small and medium ESPs themselves, others provide communication platforms but aren’t really ESPs, still others are simply companies that need a powerful and robust SMTP service.

Email sending / deliverability is such a small community, a fact that was really hammered home the last few days. I can’t help but wonder how this is going to drive change in the industry. Some of the major leaders in the email space are at Sendgrid. I hope that the integration goes smoothly and that the Twilio folks take the time to understand the market space they’re moving into.

Congrats to all my friends and colleagues at Sendgrid. This sounds like an exciting opportunity for all of you.

No Comments

Good morning DMARC

I’m thinking I may need to deploy DMARC report automation sooner rather than later.

… and so on, and on, and on for a lot further down the mailbox.


Company responsibility and compliance

I blogged a few times recently about Zoho and their issues with malicious actors abusing their platform. They asked me to post the following statement from their CEO Sridhar Vembu.

Unfortunately phishing has become one of the bad side-effects of Zoho’s rapid growth over the last couple of years, especially the growth of our mail service. Since Zoho Mail offers the most generous free accounts as part of our freemium strategy, this gets exacerbated as more malicious actors take advantage of this massive customer value. But we are clamping down on this heavily and I quickly wanted to share what we have done and will be doing.

The first step is to examine all accounts, especially free ones since this is where most of the abuse appears to be happening. We are now mandating verification using mobile numbers for all accounts, including free ones (which also helps in two-factor authentication for accounts). We are actively looking at suspicious login patterns, and blocking such users, particularly for outgoing SMTP.

The second step is around improving and tightening our policies for all users. We have recently revised and changed our policy around SPF (sender policy framework) and implemented DKIM (domain key identified mail) for our domain. This will result in a solid DMARC policy that we will also publish.

There are other heuristic methods and algorithms we are exploring and testing before we deploy at scale that we will not discuss in any detail, for all the right reasons.

I commend Zoho for the steps they’re taking. I think they are a step in the right direction.

I also want to emphasize that this is not a problem limited to very large companies. Any company, and I do mean any company, providing email services is a target for malicious entities. It doesn’t really matter how big you are or how small your customer base is. It is crucial for anyone providing online services to have plans for what to do in case of abuse.

Does this mean every company needs a full fledged compliance desk? Possibly not. If your customer base is very small, like a few hundred customers and you only advertise to a niche group then you may get away with hiding from the bad guys. But security by obscurity has never been a long term solution to most security problems.

In my experience, if you’re big enough to have a dedicated customer support desk, then you’re big enough to monitor deliverability and abuse. This isn’t as hard as it sounds if you’re using a 3rd party to send mail. Companies like Sendgrid, Sparkpost, Mandrill and many of the SMTP providers, are doing most of the heavy lifting for you. They’re signed up for and collecting feedback loop emails, they’re analyzing bounce data and providing all this information for you in an easy to digest way.

The simplest thing to do is task one of your support people with monitoring deliverability metrics from your upstream and reading the abuse@ mailbox. As your company grows, they become the lead for your compliance team. Whatever you decide, it’s critical that someone have ownership of compliance. Compliance needs to be built into processes from an early stage.

It is, of course, possible to add compliance onto an existing company, that’s how most existing companies have done it. But they’ve mostly done it due to business interrupting events because they ignored abuse and compliance issues for too long. Zoho wasn’t specifically ignoring compliance, our experience was they did cut off phishers, but someone missed the bigger picture that there was other abuse going on.

Whatever your company size, if you’re providing email services you must address abuse issues. The bad guys will find you and abuse you. It’s not a question of if, it’s a question of when. The longer you wait the more likely it is that you’re going to have a business interrupting event like a Spamhaus listing, disconnection by your ESP or even disconnection by your registrar. Planning ahead doesn’t mean you’ll never be abused, it just means you’ll be equipped to deal with it.

No Comments

2018 JD Falk Award … a mailing list

It’s M3AAWG time. Even though we’re not there, I’m getting regular updates from friends and colleagues who are there. Yesterday, was the presentation of the 2018 JD Falk award. The award recognises “a particularly meritorious project undertaken by a dedicated individual or group reflecting the spirit of volunteerism and community building.” In this case, the award went to a group of people on the “BEC mailing list.”

BEC stands for Business Email Compromise (I had to look it up, now you don’t have to). According to  M3AAWG:

The Business Email Compromise List deals with a broad assortment of criminal activity and deceptive emails, often described as “Nigerian” schemes, that use phishing and fake social media activities to attract victims. By sharing information and expertise, they have blocked spoofed emails and malware; tracked real estate, romance, IRS, W2 and lottery schemes; and identified the money “mules” used to transfer illicit funds. BEC fraud accounts for more than $12 billion in losses globally and threatens users in 150 countries, according to the FBI’s IC3 (Internet Crime Complaint Center).

Congratulations to all the participants who work tirelessly to make the internet safer for businesses and consumers.

The group does have a video describing some of what they do.

I’m sure almost every field has these types of small, private, invite only lists that allow diverse groups of experts to collaborate and share information in a (mostly) secure environment. In many cases, this is good. Groups of smart, concerned people step up and collaborate to catch criminals and prevent bad behaviour. They do so because it’s the right thing to do. They’re not looking for praise or public adulation. Participation is often simply because this thing is a problem and they have the knowledge and ability to help solve the problem.

No Comments

Fun with spam filters

I recently had a challenging travel experience in the Netherlands, trying to get from Schipol airport to a conference I was speaking at. As part of my attempt to get out of the airport, I installed UBER on my phone. There were some challenges with getting UBER to authorise my phone number, so I tried linking it to my Gmail account.

I checked Gmail today and noticed there was a message from UBER sitting in my spam folder. Why? Because the language of the email was “a different language than [my] messages usually use.”

Language filters are ones we don’t often talk about. I will sometimes mention them when I’m discussing the scope of what filters look like. But for most senders, they’re never going to send a message that is in a language different than the default language of the user. The confluence of events that caused me to sign up for UBER in a non-English-speaking country was somewhat unique.

At the time I signed up for UBER I was in The Netherlands. So, appropriately, they sent me the terms and conditions that cover riders there. They sent said terms and conditions to me in Dutch, because that’s the default language in the country where I activated my account. I wonder, though, if I had signed up with my Irish (or even US) phone number rather than my Gmail account, if the message would have arrived in English. Of course, being back in an English speaking country, it’s hard to test.

One thing this does tell me is “not in users’ primary language” is a huge negative in Gmail. This is UBER. When it comes to email they do things right.

Authentication is spot on. Gmail KNOWS this message is really from UBER. DKIM checks out, it’s aligned with the From: address, there’s a DMARC record. The content is UBERs terms and conditions, and I’m sure Google knows that, too. As UBERs service continues to grow, I’m sure this content is sent regularly and at a high enough volume to generate a strong positive reputation. Google even knows it was me that signed up for UBER because I used Google’s OAUTH to connect UBER to my gmail address.

Still, none of that was enough to get Google to deliver the mail to my inbox. It’s not my language, so it goes to bulk. I’m sure, though, that all the native Dutch speakers got the exact same message in their inbox.

It’s rare we can see how one single factor can so clearly affect delivery of an otherwise good, high reputation stream. But, as I regularly say: context matters. In this case, a different language matters more than any positive reputation signals, including the fact that I signed up using my Google account.

Filtering at some places is more complex than spam or not spam. It’s about what the user wants and expects in their inbox. Send something the webmail provider thinks is not what the user wants or expects, and mail is likely to end up in the bulk folder.



No Comments

All filters are not equal

Many questions about delivery problems often assume that there is one standard email filter and the rules are the same across all of them. Unfortunately, this isn’t really the case.

The biggest divide is consumer versus business filters. Business filters don’t really care about things like engagement. A sender could have near perfect engagement with a message to a business. But a decision maker inside the company can still decide that mail doesn’t get in. There’s no appealing to permission or wanted mail. Employee mail is provided for the good of the business, not for the good of the individual user or the sender.

There are other less obvious divides between filters as well.

I frequently refer to “webmail providers” (Oath, Microsoft, Gmail). These are companies that control the mail delivery and, for the bulk of their customers, control the mail client as well. They can use engagement filters because they have more data. Other companies, like broadband providers or web hosting services, don’t have the same level of access to customer behaviour, so they can’t heavily use engagement as part of their filtering processes. They may have some access to IMAP folders, depending on their setup, so they can look at some engagement flags.

Filtering companies also have their own type of filters. In many cases, though, they have no access to any engagement filters. They handle mail at a discrete point that starts during SMTP sessions and ends when the mail is handed off to the local delivery agent. These companies cannot use engagement as part of their filtering process all, they simply don’t have access to that data.

Understanding what data filters act on and what data they have access to can inform how to deal with blocks and delivery problems.



DNS Flag Day

There are quite a lot of broken DNS servers out there. I’m sure that’s no surprise to you, but some of them might be yours. And you might not notice that until your domains stop working early next year.

DNS is quite an old protocol, and when it was originally specified there wasn’t really a good way to extend the protocol to add new features. That was fixed about 19 years ago when Extension Mechanisms for DNS (EDNS0) was specified, and solidly standardized in RFC 6891 in 2013. It added a backwards compatible way for a DNS client to ask “Hey! Do you support new features?” and for servers to include as part of their response “Yes! Yes I do!”.

That’s incredibly useful, and critical for extending the DNS to support new features (such as DNSSEC, or support for larger replies). And yet some authoritative DNS servers not only don’t support it, they misbehave when they’re asked if they support it. It’s been the case forever that DNS servers should just ignore (some sorts of) fields in requests if they don’t understand them. So when you send a request that includes an EDNS0 “Do you support new features?” field to a DNS server that doesn’t understand EDNS0 it should return a regular DNS response. Some (broken) nameservers don’t do that – instead they drop the request on the floor and don’t respond (or, even worse, crash). Eventually the recursive resolver will give up on the request.

(DNS servers broken in this way aren’t that rare in 2018 – just last week I had to add code to a DNS library I use so that it didn’t crash when it saw EDNS0 requests.)

Right now most recursive resolvers will see a timeout for a request that included EDNS0 and decide “Maybe it only failed because the remote server has buggy EDNS0 handling”. They’ll retry the request without EDNS0 and get an answer. This workaround means that the DNS will resolve eventually, after five or ten seconds of delay. Not good, but the web page will open or the mail will be delivered eventually.

But it’s a horrible workaround, and the developers of the most widely used recursive resolvers are done with this silliness. As of February 1st next year they’re not going to do it any more. If your DNS server is broken with respect to EDNS0 your hostnames won’t resolve via a large fraction of recursive resolvers. Your webpages won’t load, mail you send won’t have any SPF, DKIM or DMARC information or even any reverse DNS. Lots of things will break in a very visible way.

You can check whether your DNS server is broken or not, and get a bunch more technical details at

No Comments

Zoho, phishing and who’s next?

ZDnet reports that Zoho’s problems with phishing aren’t over. Their report states that Zoho is being used as a pipeline to exfiltrate data from phished accounts.

The software platform’s email address service, on both and domains, is being exploited in 40 percent of phishing campaigns in which email “is the primary exfiltration vehicle.”

That’s some serious problems.

Look, managing abuse and security is hard. Every online service is at risk and companies need to think not only about how they might be attacked but also how they may be a vector for attacks against others. Email is even more vulnerable than most services. Not only is email the key to online identity it’s also the vector for the majority of online attacks.

Companies running email services for customers must have two things.

  1. A security team that monitors infrastructure for attacks from bad actors. These attacks include “customers” attempting to identify vulnerabilities in your system so they can spam or phish through the system.
  2. A compliance team that monitors customers and acts on those “customers:” that managed to sneak through the automated defences.

Every company that provides an email module in their platform is vulnerable. Every one. The big ESPs, the ISPs and the cable companies have pretty good defences these days. They’ve made spamming and phishing through their services hard enough that the bad guys are looking at much smaller companies.

No service is too small for them to look at. In fact, the smaller companies are ideal. Often the smaller companies outsource their infrastructure to a larger company, like SendGrid, Mandrill or Sparkpost. The spammers have been kicked directly off their platforms, but they can still spam through them, by abusing their customers.

The bad guys are getting smarter. They work hard to make themselves look like somewhat confused customers to extend any time on a platform. In every case they know they’re going to get cut off, at some point, they’re just trying to abuse the platform a little longer.

Compliance and security are hard. Being small is no excuse to ignore either.

No Comments