In some ways it’s been really hard to focus on email for the last few months. There are so many more important issues in the world. Terrorism, Brexit, the US elections compromised by a foreign government, nuclear threats from multiple countries, the repeal of ACA, mass deportations and ICE raids here in the US. I find myself thinking about what to blog. Then I glance at the news and wonder if there’s any value in another blog post about deliverability.
Generally I’ve tried to keep politics and world events mostly off the blog. But sometimes events are such that I need to talk about them.
Last October I had the chance to speak at the Email Innovations Summit in London. Steve and I took the chance to spend some time doing tourist things in London – including a photo walk along the Thames.
As an American I’m always a little surprised by the security in London. I grew up a few miles outside of DC. I could talk about prohibited airspace and security measures before I was 10. London is so much more open than even the DC of my youth. The surprise there is that London has been a much bigger target and attacked more than any city in the US.
The last few times we were in London I noticed a bit more visible security. In 2013 it was armed security walking through Tube stations. Last year it was Underground trains that were one long car. They were a bit weird and visually disconcerting. The part that really made me think, though, was this was a way to stop people hiding explosives between cars and to facilitate evacuations if something happened.
Last night Steve and I were talking and I mentioned the attack in London didn’t seem like terrorism to me. And it didn’t, not really. He then pointed out that explosives and guns are difficult to come by in the UK and this was classic terrorism. Oh. Sometimes our cultural differences come out in the strangest places.
Thinking about bigger issues like this make it hard to focus on email. There’s a regularly shared joke in deliverability, “There’s no such thing as a deliverability emergency.” And there isn’t, not really. Yes, even if a whole range of IPs is listed on Spamhaus, it’s still not an emergency and there’s no fast response team to deal with it.
There are abuse issues that are higher stakes than getting to the inbox. Child abuse materials. Harassment. Privacy issues. Terror threats. Every online services company, particularly the social media companies, have to deal with these kinds of things. Many of them are dealing poorly. Others have employees who are doing their best, but lack the tools, support, and training to do it well. Many companies don’t understand why they need to police their customer base.
The reality is, though, that abuse on the net (as opposed to abuse of the net) is a huge issue that needs to be dealt with. These are not small issues. The Internet is global and there’s no internet police. Law enforcement in different jurisdictions have to work together with technology experts to address crime and harassment on the internet.
It may surprise you to hear that the people who create spam filters and try and protect your inbox are the same people who fight crime on the internet. Spam and email are a vital part of online crime, so it falls on the abuse team to work with and educate law enforcement about tracing the source of email. The people you never see in ops, and abuse and support are vital to protecting folks online.
During the closing talk at MAAWG the chair was discussing how we can protect our online spaces. He stated “There is no cavalry; no second wave. It’s us or no one.” That’s a huge thing. My friends and colleagues are the people who stand protecting users online. It feels like a huge burden, but it’s something we can do to make the world a better and safer place.
In the delivery space, stuff comes in cycles. We’re currently in a cycle where people are unhappy with spam filters. There are two reasons they’re unhappy: false positives and false negatives.
False positives are emails that the user doesn’t think is spam but goes into the bulk folder anyway.
Fales negatives are emails that the user does thing is spam but is delivered to the inbox.
I’ve sat on multiple calls over the course of my career, with clients and potential clients, where the question I cannot answer comes up. “Why do I still get spam?”
I have a lot of thoughts about this question and what it means for a discussion, how it should be answered and what the next steps are. But it’s important to understand that I, and most of my deliverability colleagues, hate this question. Yet we get it all the time. ISPs get it, too.
A big part of the answer is because spammers spend inordinate amounts of time and money trying to figure out how to break filters. In fact, back in 2006 the FTC fined a company almost a million dollars for using deceptive techniques to try and get into filters. One of the things this company did would be to have folks manually create emails to test filters. Once they found a piece of text that would get into the inbox, they’d spam until the filters caught up. Then, they’d start testing content again to see what would get past the filters. Repeat.
This wasn’t some fly by night company. They had beautiful offices in San Francisco with conference rooms overlooking Treasure Island. They were profitable. They were spammers. Of course, not long after the FTC fined them, they filed bankruptcy and disappeared.
Other spammers create and cultivate vast networks of IP addresses and domains to be used in snowshowing operations. Still other spammers create criminal acts to hijack reputation of legitimate senders to make it to the inbox.
Why do you still get spam? That’s a bit like asking why people speed or run red lights. You still get spam because spammers invest a lot of money and time into sending you spam. They’re OK with only a small percentage of emails getting through filters, they’ll just make it up in volume.
Spam still exists because spammers still exist.
I know a lot of folks working at ESPs. For those I know well, I will usually send in reports. Sometimes they’re not spam reports per se. Often it’s just “hey, this sender shouldn’t have my address, might want to poke them.”
Sometimes it’s even more specific. A few years ago I spoke at a user conference for an ESP. I stayed at the hotel for one night, and the hotel now has my email address. Not a big deal, they’re on the coast and an easy drive from here. They’ll run specials for the locals, and I like it.
Enter in hotel B. I’ve never stayed at hotel B. I’m not sure who hotel B is. They’re also local and may be the same management. I don’t know. They sent me email to the address I’ve only given to hotel A. Not only that, the message is completely unreadable. Dark blue on brown… not exactly a great design choice.
I wasn’t going to send anything in to the ESP, but then I noticed that at the bottom of the email there is a notice that says “This email was sent to: %%emailaddr%%.” That looks suspiciously like this was an accidental send. The ESP folks there are colleagues, so I sent them an email into abuse@.
Mail sent to an address given to a different hotel (Hotel A, from a conference). Also, they really screwed up their HTML here. I can barely read the dark blue on brown.
Anyway, either someone did something bad, or they’ve been compromised somehow. Might be worth looking into.
Bad email marketing makes me twitch. But I’ve discovered that sending in messages to support@ or even calling their 800 numbers just leads to frustration for everyone involved. Maybe sending something broke messages to abuse@ isn’t the right thing to do. At least I know the right people will see the mail.
I’ve got multiple clients right now looking for insights about bounce handling. This means I’m doing a lot of thought work about bounces and what they mean and how they match up and how different ISPs manage delivery and how different ESPs manage delivery and how it all fits together. One thing I’ve been trying to do is contextualize bounces based on what the reason is.
Despite what people may thing, spam filtering isn’t the only reason an email fails to deliver. There are lots of other reasons, too. There is a whole category of network problems like routing issues, TCP failures, DNS failures and such. There are address issues where a recipient simply doesn’t exist, or is blocking a particular sender. There are spam and authentication issues. The discussion of all these issues is way longer than a blog post, and I’m working on that.
One of the interesting bounces that is so rare most people, including me, never talk about is “Relaying Denied.” This is, however, one of the easier bounces to explain.
Relaying Denied means the mail server you’re talking to does not handle mail for the domain you’re sending to.
Well, OK, but how does that happen?
There are a couple reasons you might get a “Relaying Denied” message, most of them having to do with a misconfiguration somewhere. For whatever reasons, the receiving server doesn’t handle mail for a domain.
DNS records are incorrect. These can be due to a number of things
- Failure to remove a MX record after a server is decommissioned;
- Pointing to a “backup” MX that isn’t configured to act as backup;
- DNS record changes have not yet propagated
In rare other cases, the DNS records are “correct” but there’s a misconfiguration on the receiving server and it doesn’t “know” that it is supposed to be handling mail for a domain. This can be temporary, if someone publishes DNS records before they finish the server configuration, this message may happen.
In these cases the mail won’t be delivered until the receiver fixes their configuration. It may be reasonable to continue mailing to the addresses, or they can be removed from the list completely.
The other case is that there was an error with the sending server. Some servers cache DNS records for longer than they should. This means that DNS is right but the sending server simply isn’t checking DNS before sending. The sender can fix this by making sure their system isn’t incorrectly caching DNS.
Chances are senders will never see a “Relaying Denied” message. But they do happen in some rare circumstances.
Today AWeber published a link to 11 innovators in email marketing. I’m honored to be one of them.
I don’t really think of myself as a marketer, I’m a delivery person. My job, really, is to help clients devise email strategies (and overall digital marketing strategies) that result in inbox delivery. When I started, there were some significant divides between email marketing and deliverability. Often what was good marketing strategy was bad deliverability strategy. That’s not as true as it once was and now good deliverability advice is good marketing advice.
I saw a headline today:
New Research from Return Path Shows Strong Correlation Between Subscriber Engagement and Spam Placement
I have to admit, my first reaction was “Uh, Yeah.” But then I realized that there are some email marketers who do not believe engagement is important for email deliverability. This is exactly the report they need to read. It lays out the factors that ISPs look at to determine if email is wanted by the users. Senders have to deal with vague metrics like opens and clicks, but the ISPs have access to user behavior. ISPs can see if mail is replied to, or forwarded or deleted without reading. They monitor if a user hits “this-is-spam” or moves the message to their junk folder. All of these things are signals about what the users want and don’t want.
Still, there are the folks who will continue to deny engagement is a factor in deliverability. Most of the folks in this group profit based on the number of emails sent. Therefore, any message about decreasing sends hurts their bottom line. These engagement deniers have set out to discredit anyone who suggests that targeting, segmentation or engagement provide for better email delivery and getting emails to the inbox.
There’s another group of deniers who may or may not believe engagement is the key to the inbox, but they don’t care. They have said they will happily suffer with lower inbox delivery if it means they can send more mail. They don’t necessarily want to discredit deliverability, but they really don’t like that deliverability can stop them from sending.
Whether or not you want to believe engagement is a critical factor in reaching your subscribers, it is. Saying it’s not doesn’t change the facts.
There are three things important in deliverability: engagement, engagement, engagement.
Last week the CRTC published a CASL enforcement action wherein they fined an individual $15,000 for 10 violations of the act.
The Commission imposes an administrative monetary penalty of $15,000 on William Rapanos for 10 violations of section 6 of Canada’s Anti-Spam Legislation. Specifically, Mr. Rapanos sent commercial electronic messages (i) that did not identify the sender, (ii) that did not include information that enables the recipient to readily contact the sender, (iii) without prior consent from the recipients, and (iv) that, in certain cases, did not include a functioning unsubscribe mechanism.
I do encourage folks who are concerned about CASL to read through the full article on the CRTC website. They write out how hard they tried to work with the individual in question. They really seem to have tried to do what they could to get compliance with the act without assessing a fine. As the CRTC says, the aim of penalties is to promote compliance, not to publish people who violate the act. We’ve certainly seen other CASL cases involving much more mail, that proportionally smaller penalties assessed.
One thing that I noticed in the article was the description of the individual under investigation. The CRTC walks through the discussions with him and the investigations into sending. The documented behavior is very “spammer” to me, especially the “someone is doing this to frame me” and “someone must have stolen my identity.” No one really believes that someone would steal your identity, break into your house, use your wifi and … only send spam. There’s so much more that can be done with that level of access.
None of his behavior is any surprise to any of us who have worked with spammers.
In my previous career I studied the effect of alcohol on developing embryos. It’s a bit weird I ended up in that field because embryological development always seemed to complex to me. And it was and is complicated. In a lot of ways, though, it was good training for deliverability. We dealt with a lot of processes that seem, on the surface, straightforward.
Fertilization happens, then you get a flat group of cells, those cells fold up into the neural tube, cells migrate around, things happen, limbs form, organs form and 21 days later you have a fluffy little chick.
The details in all those steps, though. They’re a bit more complicated, looking something like this:
There are lots of different things going on inside the embryo to take it from a single cell up to a complex multicellular being. Genes turn on, genes turn off at different times in development, often driven by overlapping concentration gradients. Genes turn each other and themselves on and off. It’s complex, though, and there are things that happen that we don’t quite understand and have to black box. “If I add this protein, or take this gene and that gene away… what happens?”
A lot of that is like what email reputation is these days. There isn’t one factor in reputation, there are hundreds or thousands. They interact with each other, sometimes turning up reputation, sometimes turning down reputation. We figure this out by poking at the black box and seeing what happens. Unlike development, though, delivery rules are not fixed. They are changing along the way.
It’s not simple to explain delivery and how all the moving parts interact with each other. We don’t always know that doing A will lead to X. Because A -> X is not a straight line and there are other things that impact that line. Those other things also impact A, X and each other.
Delivery is a tangled web. On the surface it seems simple, but when you start peeling back the layers you discover the jumble of factors that all interact with each other. It’s what makes this a challenging field for all of us.
Today is International Women’s Day. In recognition of this day, there has been a call for a general woman’s strike. I thought long and hard about how I would participate in this event. Even yesterday I had no clear view of whether or not I would be working today.
As a self-employed woman, me not working today only hurts me and my clients. There’s no one to leave work for, I either do it before or after. It’s got to get done and it’s my responsibility to do it. But at the same time, I recognize the unpaid and underpaid work most women do and fully support the strike.
After much thought, I decided that my contribution to the strike would be to do what I needed to do for work. But that I would remove myself from public conversations about email today. I spend quite a bit of my time doing unpaid work that supports the email industry: standards work, answering questions in various fora, supporting different initiatives, writing documents, blogging about industry events. I won’t be doing any of that work today.
Yes, there are questions I could answer, advice I could give, industry events that I have comments and insight on. But today, today I’m not going to do any of that.