It’s that time again… here’s a look at our last month of blog posts. We find it useful to recap each month, both to track trends and issues in email delivery and to provide a handy summary for those who aren’t following along breathlessly every single day. Let us know if you find it useful too!
As always, I wrote about email filters. It’s so important to recognize that filters aren’t arbitrary — they’re detailed instructions that help meet specific user needs, and the more you are cognizant of that, the better you’ll be able to work with them. Additionally, filters aren’t perfect and likely never will be. False positives and false negatives are frustrating, but as long as spam is still a viable business for spammers, they’ll continue to figure out how to work around filters. As such, we can’t expect filters to be 100% accurate in determining what constitutes wanted and unwanted mail.
Part of this, of course, is due to the problem of fraudulent signups. Companies aren’t particularly vigilant about address acquisition and hygiene, and as a result, they’ll claim you “signed up” for their email when you did not. Some people believe that a confirmed opt-in (COI) will solve this problem, but our experience is companies are reluctant to leave revenue on the table, and that they will continue to mail to addresses that have not confirmed.
Address sharing and co-reg is also part of the problem. As we saw in the extensive RCM data breach, many major brands continue to work with third-party senders to send mail in ways that are quite clearly spam. And in more criminal activity, I looked at the rise of botnets and how some of those criminals were brought to justice. In other justice news, there’s been an indictment in the Yahoo breach and another CASL enforcement action.
I wrote a post about bounce handling and “relaying denied” error messages, which are quite rare. It’s useful to have an understanding of these and other error messages, since bounces are sometimes indicative of a larger technical issue, such as when AOL accidentally bounced all messages for a short period last week. Speaking of AOL, we noted that there’s no official timeline for the move from Verizon addresses to AOL addresses following the 2015 acquisition, but it may be worth considering asking your customers to update their addresses.
Spam and filters aren’t the only factors of course. It can be challenging to figure out the multiple factors that make up the black box of delivery. And of course, the most important part of delivery continues to be engagement, engagement, engagement.
I wrote a few posts this month on why I do what I do, and why it’s so important to me. First, I wrote about A Day Without A Woman, and my choice not to participate in offering advice and guidance for that day. The truth is that I enjoy sharing what I know and helping people solve problems. I was honored to be named one of 11 Innovators in Email, and I know that my volunteer work in the industry and my unpaid blogging work is a big part of that. It may sound corny, but I really do believe we are on the front lines of the fight of good vs. evil online, and despite the distractions of politics and world events, we must all continue to do our part.
Over on the MarketingLand website, Len Shneyder talks about 3 companies (Uber, REI and eBay) that do email right. In there he shows how the companies use email to further their business goals while understanding and meeting the needs of their customers.
Meeting the needs of recipients is the way to get your mail to the inbox. Send email that your users want, and they will tell the ISPs when they don’t get your mail. It’s sometimes hard to convince senders of this. Instead they want to tweak URLs or authentication or IPs or domains. But none of those things are what deliverability is all about. Deliverability is about the recipient. Deliverability is about the relationship between the sender and recipient.
Send to the right people – and the right people are those who have asked for and want your mail – and deliverability problems don’t materialize. Sure, every once in a while something might happen that throws mail into the bulk folder for one reason or another. But fighting to get to the inbox isn’t an every day thing. Instead, senders can focus on knowing their users and sending mail that makes them happy when it shows up in the inbox.
Various things happening in the email space recently that are worth mentioning but don’t have enough to justify a whole blog post.
Verizon announced a new umbrella company for the AOL and Yahoo media properties, including things like Engadget, Huffington Post. Based on the various press articles I’ve seen this doesn’t appear to affect the email handling for either set of domains.
Making deliverability people swear since 2017.
Spamhaus recently discovered an anonymous DNSBL that was taking their data and republishing it. They cut the company off and released a statement: Fraudulent DNSBL uncovered. Based on public discussions it appears the DNSBL was using other sources as well. Multiple people complained about a high false positive rate and lack of ability to delist. I don’t know how widely used it was, it’s a list I hadn’t heard of until a few weeks ago. If you’re looking for a filter / blocking list to use, you’re better off finding one that communicates with listees. That tends to mean it’s well run and responsible.
Experian has sold its cross channel marketing division, including Marketing Suite, CheetahMail, and Mail Publisher and some of its strategic marketing services to a venture capital firm. The chairman of the new company is former Exacttarget founder Peter McCormick. Worth watching the new company develop. I’m wondering how this might change Experian’s approach to permission based email. Right now, they have some rather sloppy data acquisition processes and don’t do a great job at sending to the right person. Just this month I got spam from their UK marketing arm targeting me as a university student looking for an apartment and needing to know my credit score. (Yes, I’ve shared the relevant information with the Experian folks. I expect they’ll investigate and tell me some UK college student wanted a free t-shirt and so handed over my email address. This doesn’t excuse Experian for the spam, they should have better data hygiene in place. And I know their in house experts and internal consultants have told them this, repeatedly.)
The FTC settled a claim with a group of Florida marketers for $500,000 related to CAN SPAM violations. These marketers hired affiliates to send mail on their behalf. Some of this mail falsified headers and lacked opt-out links. The final order is worth a read. (Stipulation final order pdf)
It should be easy, right? Except it’s not. So why is it so hard?
With one-on-one or one-to-few email it’s pretty simple. The rejections typically go back to a human who reads the text part of the rejection message and adapt and makes the decision about future messages. The software handles what to do with the undeliverable message based on the SMTP response code.
In the case of a 5xy response the server stops attempting delivery and alerts the original sender the mail failed. One example from helping a client troubleshoot a delivery problem recently.
There’s useful information in the text portion of this email from my mail server. It says there was a permanent failure (550) and that my message won’t be delivered. It also says the email is quarantined in reply to the end of DATA. That’s actually a critical piece of information. It means Barracuda saw the entire message before deciding to reject it. It’s likely a problem with the content of the email and so I need to look at links in the message.
This type of plain text explanation is great for a human to read and act on. But it’s not that simple for list handling software to identify the relevant information in the text message and act on future emails to that recipient. Different MTA vendors and ESPs have done a lot of work to try and correctly parse bounce messages to pull out relevant information.
ISPs have tried to help the situation by giving more descriptive rejection messages. They’re still using the SMTP required 3 digit numbers, but they include short, parseable codes in the text portion of the message. In many cases they also include URLs and links that open up webpages explaining the meaning of the code. They even post a list of the most common codes on their postmaster webpages.
All of these things make it somewhat easier to handle bounces automatically. Kinda.
I’ve been working on some bounce handling recommendations for a client using a few different ESPs. I spent a good few days digging into the bounces returned by their different ESPs. It was an interesting exercise as it demonstrated how very differently ESPs handle bounces. But it also clarified for me that there are a lot of different kinds of bounces.
- Address related bounces: user unknown, mailbox unavailable, invalid address, bad address syntax.
- Network related bounces: DNS failures, server too busy, temporarily unavailable, network error, connection timed out.
- Reputation related bounces: poor IP reputation, IP blocked, content blocked, too many complaints, prohibited attachment.
- Bad sending configuration: relaying denied, message too large,
- Authentication failure: DMARC
There’s also an overlap of Network and Reputation bounces: too many connections and too many messages are network methods to slow down sending of problematic mail.
None of these are always hard or always soft bounces. There are multiple ESPs that have also introduced the concept of “spam” bounce, which are an attempt to separate out some of the bounces due to problematic content from bounces labeled “hard” or “soft.”
I still hold out hope that someone, somewhere will come up with the grand unified theory of bounce handling. I suspect I’ll get a pony before that happens, though. I’m starting to get a better handle, though, on how to explain bounce handing – both in helping people visualize the different categories and clarifying what should be done with different messages.
This morning I got spam from a major data broker / ESP / credit reporting agency claiming I’d signed up on some college website. In the UK. To check my credit score.
Uh. No. No I didn’t.
Of course, it’s very possible someone did use my email address when signing up for something at a UK university. They probably got a t-shirt or free pizza out of it. But that doesn’t really matter to me. A certain credit agency is spamming me with irrelevant and horribly targeted advertisements for their services and claiming the mail is opt in.
I know that address is widely sold in the UK to “legitimate” marketers. It’s very possible that it was purchased by the spammer in question. Or, I dunno, maybe they’re the ones selling it. As a victim, I don’t really care why a company is spamming me.
Part of a sender’s job to make sure their data is accurate. And they failed.
But for this particular company, that’s par for the course. When I posted about this over on Facebook, I had multiple friends pointing out that this company regularly spams and sells spamming services.
Spammers gonna spam.
Last night (Mar 29, 2017) between about 8pm Eastern and 9:30pm Eastern AOL suffered a technical issue. Every email sent to them received a “Recipient address rejected” reply. One example of the error message:
Mar 29 20:45:12 p2-lvmail11 lsb1-99-208-250/smtp: A88DFC2DBE9: to=<firstname.lastname@example.org>, relay=mailin-01.mx.aol.com[64. 12.91.195]:25, delay=0.18, delays=0.01/0/0.14/0.03, dsn=5.1.1, status=bounced (host mailin-01.mx.aol.com[64.12.91. 195] said: 550 5.1.1 <email@example.com>: Recipient address rejected: aol.com (in reply to RCPT TO command))
The issue was brought to AOLs attention and things were fixed rapidly after that. An AOL representative has stated that these were invalid replies and that addresses do not need to be removed from future emails.
Most of the ESPs are aware of this and are working to restore any bounced addresses to their users. At some places this requires manual intervention, so it’s taking some time to get all the addresses restored.
This is one of the reasons that our best bounce handling recommendations are not to remove an address for a single bounce – sometimes the ISPs have technical problems. Like the time a routing failure meant a major ISPs MX machines couldn’t reach their authentication servers to get the list of active users. Or the time all an ISPs MXs were removed from DNS. A lot of the internet is still managed manually, and despite extensive safeguards put in place bad things can, and do, still happen. Usually these problems are resolved quickly and mail starts flowing again.
Morning advice: Do not deactivate addresses that bounced at AOL last night.
In the email space we talk about filters as if they were sentient beings. “The filters decided…” “The filters said…” This is convenient shorthand, but tends to mask that filters aren’t actually deciding or saying anything. Filters are software processes that follow rules dictated by the people who create and maintain them. The rules flow from the goals set by the mailbox provider. The mailbox provider sets goals based on what their users tell them. Users communicate what they want by how they interact with email.
What we end up with is a model where a set of people make decisions about what mail should be let in. They pass that decision on to the people who write the filters. The people who write the filters create software that evaluates email based on those goals using information collected from many places, including the endusers.
What mail should be let in is an interesting question, with answers that differ depending on the environment the filter is deployed in.
Consumer ISPs typically want to keep their users happy and safe. Their goals are to stop harmful mail like phishing, or mail containing viruses or malware. They also want to deliver mail that makes their users happy. As one ISP employee put it, “We want our users to be delighted with your mail.”
Businesses have a few other goals when it comes to filters. They, too, need filters to protect their network from malicious actors. As businesses are often directly targeted by bad actors, this is even more important. They also want to get business related email, whether that be from customers or vendors. They may want to ensure that certain records are kept and laws are followed.
Governments have another set of goals. Universities and schools have yet another set of goals. And, of course, there are folks who run their own systems for their own use.
Complicating the whole thing is that some groups have different tolerances for mistakes. For instance, many of our customers are folks dealing with being blocked by commercial filters. Therefore, we don’t run commercial filters. That does mean we see a lot of viruses and malware and rely on other strategies to stop a compromise, strategies that wouldn’t be as viable in a different environment.
Filters are built to meet specific user needs. What they do isn’t random, it’s not unknowable. They are designed to accomplished certain goals and generally they’re pretty good at what they do. Understanding the underlying goals of filters can help drive solutions to poor delivery.
Use the shorthand, talk about what filters are doing. But remember that there are people behind the filters. Those filters are constantly maintained in order to keep up with ever changing mail streams. They aren’t static and they aren’t forgotten. They are updated regularly. They are fluid, just like the mail they act on.
Botnets are a huge problem for a number of reasons. Not only are they used to send spam, they’re also used in criminal activities. One of the major challenges in dealing with botnets is finding and stopping the people who create and use them. Why? Because the internet is global and crime tends to be prosecuted within local jurisdictions.
White Collar Crime.
Catching someone running a botnet, or involved in crime online in general, requires cooperation from authorities in multiple jurisdictions. Police, lawyers, and other officials have had to create relationships to work together, all while respecting international law. It’s a involved and complicated process, and that’s before we talk about the challenges in actually figuring out who is running the botnet. Subject matter experts, like operating system manufacturers or anti-virus companies, are also part of the process in most cases. (Read about the Simda botnet takedown at Interpol)
Despite the challenges, botnets do get taken down and criminals do get arrested and brought to justice. Today the Department of Justice announced a guilty plea from a Russian citizen charged with infecting machines with malware.
Senakh and his co-conspirators used the Ebury botnet to generate and redirect internet traffic in furtherance of various click-fraud and spam e-mail schemes, which fraudulently generated millions of dollars in revenue. As part of the plea, Senakh admitted that he supported the criminal enterprise by creating accounts with domain registrars which helped build the Ebury botnet infrastructure and personally profited from traffic generated by the Ebury botnet.
Ebury is kinda interesting because it’s actually a Linux botnet, not a Windows one. It used a SSH exploit to get in, stole user credentials and then smuggled the credentials out in special TCP packets. CERT-BUND has some of the gritty technical details of what they discovered. And WeLiveSecurity also has a writeup on how the infection worked.
Botnets are a problem. Catching people is a long, drawn out challenge. But, it can be done.
“If you want to use another means that violates the law, and every common definition of “spam”, then by all means, go ahead. You can enjoy fines and being added to the ROKSO database,” says a comment on my recent COI blog post. It’s both disconcerting and entirely predictable.
My post was a discussion of what to do with addresses that don’t confirm. Data tells us that there is some value in those addresses – that there are people who won’t confirm for some reason but will end up purchasing from an email. Using COI leaves some fraction of revenue on the table as it were. My post was a short risk analysis of things to think about when making decisions about continuing to mail to people who don’t confirm.
Mentioning COI often brings the only-COI-mail-is-not-spam zealots out of the woodwork, as it did in this case. In this case, we have the commenter first asserting that failure to do COI is a violation of CAN SPAM (it’s not). When this was pointed out, he started arguing with two people who have been actively fighting spam for 20 years (including running a widely used blocklist). Finally, he ends up with the comment asserting that anyone not using COI will end up on ROKSO. It’s as if he thinks that statement will fear other commenters into not having opinions. It can’t because everyone in the discussion, except possibly him, knows that it’s not true.
The worst problem with folks like the commenter is that they think asserting horrible consequences will make people cower. First off, people don’t react well to threats. Secondly, this is a hollow threat and most people reading this blog know it.
There are millions of mailing lists not using COI and have zero risk of ever getting on ROKSO. The only thing hollow threats do is make people not pay attention to what you have to say. Well, OK, and have me write a blog post about how those threats are bad because they’re completely removed from reality.
Exaggerating or lying about consequences is not just wrong, it’s stupid. “Do this or else BAD THING,” is awesome, up until someone decides they’re not going to do this and the bad thing never happens. It makes people less likely or pay any attention to you in the future. It certainly means your opinions and recommendations are not going to be listened to in the future.
I probably go too far the other direction. I can spend too much time contextualizing a recommendation. It’s one of the things I’m trying to get better about. No, client doesn’t need a 4 page discussion of the history of whatever, they just need 2 lines of what they should do. If they need the context, I can provide it later.
In order to effectively modify behavior consequences have to be real. Threats of consequences are meaningless. Any toddler knows this, and can quite accurately model when mom means it and when she’s just threatening.
Risk analysis is not about modifying behavior. It’s about analyzing a particular issue and providing necessary information so the company action understands potential consequences and the chance those risks will happen. The blog post about COI was not intended to modify anyone’s behavior. I know there are companies out there successfully maintaining two mail streams: one COI and one not. I know there are other companies out there successfully mailing only single opt-in mail. I know there are companies with complex strategies to verify identity and address ownership. And I smile every time I walk into a retail store and they ask me if my email address is still X and if I want to make any changes to it.
Lying about consequences does nothing to modify behavior. All it does is diminish the standing and audience of the liar. Be truthful about the consequences of an action or lack of action. Don’t make up threats in order to bully people into doing what you think is right. Sooner or later they’re going to realize that you don’t know what you’re talking about and start to ignore you.