Thoughts on filters

One of the questions we received during the EEC16 closing keynote panel was why isn’t there a single blocklist that everyone uses and why don’t ISPs share data more. It would be so much easier for senders if every ISP handled mail the same as every other. But the world isn’t that simple, and it’s not always clear which mail stream is spam and which is good mail.

12495088_10154028658527410_7653293570464523506_n

There were quite a few answers but they basically boiled down to a few facts.

  • Different blocklists have different data strengths and weaknesses.
  • No blocklist has all a full view of all the bad mail.
  • You may want to have different polices for delisting depending on what kind of mail the blocklist is targeting. For instance, Spamhaus has different polices for different lists: CBL has self serve delisting, SBL requires email, ROSKO requires no traceable spam for 6 months.

The short reason was we use different lists and techniques because it makes the spam filtering better.

When I got home from the conference, I saw In-depth analysis of the lessons we learned while protecting Gmail users post. Among other things, it answered the “why not one blocklist” question. Even more, I think it did a really good job of talking about what email looks like from the receiving end.

Any defense can be defeated – Use defense in depth with multiple layers of protection.
Since no combination of detection systems at a given layer is perfect, there is a need to add multiple layers of defense to make it even harder for attackers.

One thing I’ve been trying to get across to marketers is that email is an a very malicious channel. Many of the bad mails out there, the ones the filters are aiming for, are dangerous and malicious. Those attackers spend a lot of time trying to figure out how to get past the defenses.

Make it hard for attackers to understand your defenses – Use overwhelming force and deploy many countermeasures at once.

It is very important to make probing more difficult for attackers by rolling out multiple changes. That way they are overwhelmed by the number of things to test and can’t easily figure out what changed.

This is why it’s so hard to test “what Gmail changed.” They are going out of their way to release multiple things at once. It’s also why it’s not really useful to test. It’s more useful to look at your mailing practices and see where they might be borderline and driving your reputation down.

The whole article is well worth a read. It gives a good overview of what Gmail is doing and how they think about email, filtering and dangers. It also gives examples of the different challenges they deal with on a regular basis.

Overall, it’s important to realize that filters are an important part of the email ecosystem. They are a big part of why it’s a viable marketing channel. Think of it this way, an unweeded garden is not as productive as a weeded garden. Weeds take nutrients away from the plants and stunt their growth. They also make it harder to find the actual produce at harvest time. Filters are the herbicides and weeding that keep gardens healthy and productive. Without them, no one could effectively use or trust email.

Related Posts

March 2016: The Month In Email

Happy April! I’m just back from the EEC conference in New Orleans, which was terrific. I wrote a quick post about a great session on content marketing, and I’ll have more to add about the rest of the conference over the next week or so. Stay tuned!
March2016_blog
Here’s a look at what caught our attention in March:
On the DMARC front, we noted that both Yahoo and mail.ru are moving forward with p=reject, and Steve offered some advice for ESPs and software developers on methods for handling this gracefully. I also answered an Ask Laura question about making the decision to publish DMARC. Look for more on that in this month’s Ask Laura questions…
Our other Ask Laura question this month was about changing ESPs, which senders do for many reasons. It’s useful to know that there will generally be some shifts in deliverability with any move. Different ESPs measure engagement in different ways, and other issues may arise in the transition, so it’s good to be aware of these if you’re contemplating a change.
In industry news, I wrote a sort of meta-post about how the Internet is hard (related: where do you stand on the great Internet vs. internet debate? Comment below!) and we saw several examples of that this month, including a privacy debacle at Florida State University. Marketing is hard, too. I revisited an old post about a fraud case where a woman sued Toyota over an email marketing “prank”. As always, my best practices recommendation for these sorts of things (and everything else!) really boils down to one thing: send wanted email.
Steve wrote extensively about SPF this month in two must-read posts, where he explained the SPF rule of ten and how to optimize your SPF records. He also wrote about Mutt, the much-loved command line email client, and marked the passing of industry pioneer Ray Tomlinson, who, in addition to his many accomplishments, was by all accounts a very thoughtful and generous man.
Finally, I occasionally like to take a moment and follow the twisty paths that lead to my spam folder. Here’s a look at how Ugg spams my email doppelganger, MRS LAURA CORBISHLEY. In other spam news, there’s a lot of very interesting data in the recent 10 Worst list from Spamhaus. Take a look if you haven’t seen it yet.

Read More

Security vendors and trust.

A big part of my predictions for 2016, that I’ll publish shortly, is that security is going to be a huge issue. I think we’re really going to see receivers expecting senders to have their houses in order when it comes to sending mail.
Of course, some filter companies need to get their houses in order to. Yesterday, a security researcher went public with problems in the TrendMicro anti-virus appliance. These vulnerabilities would let any email sender remotely execute code on the recipients machine with no interaction of the user. They also exposed all the passwords on the machine to the outside world.
Even worse, Trend doesn’t seem to understand the urgency to fix this. They have started releasing patches for the exploits, but there are significant problems with the patched versions as well.
If you’re a Trend user, you may want to consider other vendors for desktop security. I know that no security is perfect and that other vendors have problems, too. But shipping a password manager that exposes all passwords is just incompetence. It seems like a corporate lack of understanding of what their business is and how to actually create security software.
Even worse is that lack of urgency from the Trend folks as the security researchers are explaining the problem. I don’t care if the person receiving the report was the janitor, anything that says security exploit should be escalated to someone who can determine if the report is valid.
Compare Trend’s reaction to this to Juniper’s reaction to discovering a backdoor in their code in December. First off, Juniper found the exploit during a routine code review. That alone tells you Juiper is continually monitoring their code security. Second, Juniper was reasonably open about the issue, with executives posting blogs and security posting advisories talking about the issue. More importantly, they shared how they were going to fix it and prevent it from happening again.
Security is such a large issue right now. We have to be able to trust our vendors to do what they’re selling us. Every vendor is going to make mistakes and have vulnerabilities. No code and no developer is perfect. I do expect, though, that vendors will take exploits seriously and act fast in order to correct the problem. I’m not seeing that sense of urgency with Trend.
 

Read More

January 2016: The Month in Email

Jan2016_blogHappy 2016! We started off the year with a few different “predictions” posts. As always, I don’t expect to be right about everything, but it’s a useful exercise for us to look forward and think about where things are headed.
I joined nine other email experts for a Sparkpost webinar on 2016 predictions, which was a lot of fun (see my wrap up post here), and then I wrote a long post about security and authentication, which I think will be THE major topic in email this year both in policy and in practice (see my post about an exploit involving Trend Micro and another about hijacked Verizon addresses). Expect to hear more about this 2016 continues.
My other exciting January project was the launch of my “Ask Laura” column, which I hope will prove a great resource for people with questions about email. Please let me know if you have any questions you’d like to see me answer for your company or your clients — I’ll obscure any identifying information and generalize the answers to be most widely applicable for our readers.
In other industry news, it’s worth noting that Germany has ruled it illegal to harvest users’ address books (as Facebook and other services do). Why does that make sense? Because we’re seeing more and more phishing and scams that rely on social engineering.
In best practices, I wrote about triggered and transactional emails, how they differ, and what to consider when implementing them as part of your email program. Steve describes an easy-to-implement best practice that marketers often ignore: craft your mails so the most important information is shown as text.
I re-published an older post about SMTP rules that has a configuration checklist you might find useful as you troubleshoot any issues. And a newer issue you might be seeing is port25 blocking, which is important if you are hosting your own email senders or using SMTP to send to your ESP.
Finally, I put together some thoughts about reporting abuse. We work closely with high-volume abuse desks who use our Abacus software, and we know that it’s often not worth the time for an individual to report an incident – but I still think it’s worthwhile to have the infrastructure in place, and I wrote about why that is.

Read More