BLOG

Industry News & Analysis

Global Suppression Lists

Global Suppression List.

Pander File.

Screamers List.

stop

Whatever you call it, it’s the list of email addresses you suppress from every mailing.

If you’re an ESP, this is the list of people who you never, ever want to send email to – and I’m talking about ESP-wide global suppression lists here, not the suppression lists maintained per-customer.

Global suppression lists are a vital tool to have, as it’s the only way you can comply with requests like “Never mail me again.” – and failing to comply with those will lead to, at best, irritation, yelling and blocking, and at worst legal action.

But it’s only the right tool for suppressing mail in a few cases. One obvious one is when someone specifically requests no more mail, ever, through your system. Another is when there’s a technical reason (you never want to send mail to autoresponders, for instance), or a legal reason (pending litigation, or an incompatibility between the mail you send and a specific jurisdiction).

And there are a very few people who just cause way too much support overhead when you send them email – that’s the origin of the term screamer list, I’m sure.

But it’s not what you should be reaching for in response to spam complaints, even heated ones, or feedback loop hits. A spam complaint is a sign that your customer is probably doing something wrong, and that this recipient doesn’t want that customers mail. A feedback loop hit says that this recipient doesn’t want that customers mail (and, statistically may indicate that your customer has a problem).

Neither of them is a sign that the recipient doesn’t want mail from any of your customers. You definitely wouldn’t want one of your customers sending spam to cause mail from all of your customers to be blocked – so why would you let a complaint about one of your customers block mail to that recipient from all your customers?

(We’ve occasionally come across ESPs who have preemptively blocked all mail to addresses @wordtothewise.com, for no clear reason. When our clients discover that their ESPs are silently discarding our attempts to subscribe to their mailing lists it doesn’t do much for that ESPs reputation in our clients’ eyes.)

And whatever you do, don’t respond to a spam complaint telling them you’ve added them to a global suppression list. That says several things, to an already annoyed person. It tells them that you’ve just broken their subscriptions, past or future, to your other customers. And by “fixing” the spam problem for this one recipent in this way it suggests that you’re not actually going to do anything to deal with the customer they’re complaining about. Nothing about this can end well.

Instead, tell them that you’ll make sure they don’t receive any further mail from that customer, and that you’ll talk with the customer and take action that you deem appropriate. (And then do that).

P.S. Does anyone know the origin or etymology of the term “pander file”?

2 Comments

Security issues affect us all

I’ve been talking about security more on the blog. A lot of that is because the security issues are directly affecting many senders. The biggest effect recently has been on companies ending up on the SBL because their signup forms were the target of a subscription attack. But there are other things affecting online spaces that are security related. Right now not much of it is affecting email senders, but it’s good to be aware of.

DDOS attacks

There has been an increase in DDOS attacks against different companies and network. Some of the online game sites have been targeted including EA, Blizzard and others. A group called PoodleCorp is claiming responsibility for those attacks.

Another set of DDOS attacks hit Brian Krebs’ website this week. The site stayed up, but Akamai has told Brian they can no longer host his website. His website is down for now and the foreseeable future.

While this activity doesn’t affect marketers directly, it does tell us that there is active development happening on the less legal side of the internet. The volumes of the recent attacks have sent records. They’re also changing in scope and including new kinds of traffic in an effort to knock sites offline. Even more concerning, they appear to be systematically attempting to discover defenses in order to attack the internet as a whole.

Increase in Spam

Spam has been on the decrease over the last few years. Many of us were treating it as a mostly-solved problem. But a new report from Cisco Talos shows that trend is reversing and spam levels are increasing. Current levels are approaching those last seen more than 5 years ago. Cisco Talos has used a number of different sources of data, all showing an increase in spam directly and indirectly.

CBL Volumes over past 10 years: 

CBL_totalflow-tenyears

Cisco Talos also looks at the number of IP addresses in the Spamcop blocklist as a proxy for the amount of spam sent. Average numbers of listed IPs have doubled over pre-2106 levels.

According to the author, this rise is mostly attributed to the Necurs botnet. This botnet is a little different than most, in that it only uses a small subset of infected machines for each spam run. It sends some mail, and then the bot goes quiet.

While this doesn’t affect marketers directly, it does mean that spam filters will be under even more active development. I’ve actually seen some of this increase in activity myself. For me, the addresses hit hardest are the ones stolen from ESPs and retailers over the years.

ISPs being compromised

This week Yahoo announced that over 500 million accounts were compromised. Account owners are being alerted to update their passwords when they log in. Yahoo also cautions that actual Yahoo mail will have a special badge when viewed in the Yahoo web client and the smartphone applications.

The icon is a small purple Y next to the from address in the inbox:

YahooSecurity4

And in the message itself:

YahooSecurity5

Of concern is that Yahoo has attributed the hack to state sponsored actors. On the surface it’s hard to believe that a government would care about getting into people’s Yahoo mail. But, as Yahoo and other mail providers are used worldwide, they may be looking for access to certain accounts and it’s easier to take all of them or some of them.  Yahoo has set up a website for customers concerned about the compromise and to answer common questions.

For marketers this isn’t necessarily a direct concern. However, companies that tie account access to email addresses need to address the security of those accounts. What happens when the email address is compromised? How easy is it for someone to get into your system if they own someone else’s email address? Can they find credit card numbers and other PII?

What next?

Well, we don’t really have a what’s next. But security is a major issue online and with the active development of new tools everyone online needs to start prioritizing security. What are your defenses? What happens when you’re compromised? What can you do? Who do you call?  These discussions need to happen and they need to happen sooner or later.

 

No Comments

Use the form…

A lot of senders get frustrated with the time it can take to get a response from some ISPs. It’s totally understandable, for a lot of companies delivery problems are all hands on deck level problems. They want them fixed and they want them fixed IMMEDIATELY. They want feedback that their issue is being addressed. They want to know someone at the ISP knows there is a problem.

I’ve talked before about visiting my friend Anna and watching her laptop screen explode with IMs from senders who wanted help with an AOL issue. She’s awesome and conscientious and tried to address all of those issues as fast as she could. She did want senders to feel like their issues were important and that someone inside AOL cared about the mail blocks.

SpecialSnowflake
I was always a strong advocate for following the official pathways for addressing problems. That was the whole point of the 2009 blog post. These days it’s easier to do than it ever was. Many ISPs have forms and process around handling delivery issues. This is good! In the past getting an answer to “why is my mail blocked” required knowing the right people. Now, it’s not about who you know. The ISPs and filtering companies who are open to senders have postmaster pages, unblock forms and official request channels. Those that don’t have those channels have made certain business decisions to not provide support for senders.

Despite the availability of webforms and knowledge bases and detailed information, a lot of people still think that the only way to get attention or get an issue addressed is to get someone on the phone. It’s not, though.

ISPs have their processes. If you want things handled quickly use those processes. Even in the places where very helpful reps are, they can’t (on order of lawyers and executives) help people unless there is a ticket already open.

Always, always use the recommended processes before trying to find “a real person.” Most of the time your issue can be solved faster if you fill out the form than if you hunt around for a person. In the worst case, all that time will be wasted as the person in question will tell you to fill out the form.

 

1 Comment

Upcoming events

Next month I’ll be in London for the Email Innovations Summit. This will be an updated version of what you need to know to talk with technical folks.

In early December I’ll be doing a DMA webinar discussing the subscription bombings. That’s still in the works.

I’m looking at some events for next year. I am planning on being at M3AAWG in San Francisco in February.

I’m looking at others, too. What are your favorite events?

4 Comments

Ongoing subscription form abuse

Last week Spamhaus posted information on the ongoing subscription attacks. They provided a more information about them that was not make public previously, including some information about the volume of mail some targets received.

Today SendGrid also blogged about this, going into a little more detail about why senders should care about this. They also provided a number of suggestions for how to mitigate the risk of being part of an attack.

Many abstract images on the theme of computers, Internet and high technology.

There are a couple of things I think it’s important for folks to realize.

This is the new normal

As Spamhaus states, there is some evidence that this may have been a test run for a new product selling mailbombing as a service. Even if it’s not, although I do agree with their assessment, this is something we need to address. Many online companies are struggling with how to stop being a conduit for abuse and harassment. These issues aren’t easy, but they’re there and we have to address them.

Spamhaus saw a direct attack yesterday and a number of ESPs woke up to new SBL listings this morning.

The damage is ongoing

ESPs and other relevant parties have stepped up to the plate to minimize the effect on victims. Despite this there are many addresses still receiving email at significant volumes. Certainly it’s not the hundreds per minute but addresses are permanently affected by this kind of abuse. Because of the targets, including WordPress installations, much of the mail isn’t coming through traditional ESPs.

This diverse sources make it difficult to block the mail, in the short term and the long term.

This is not about spam

This isn’t just about marketing mail. Again, a lot of the conduits for abuse are WordPress forms. Some of the conduits are online alert services. This is about online services being used as tools for harassment.

We need new tools

The problem with spam is a lot of people suffer a little bit of damage. This means most tools use volume of complaints as a primary metric. But with direct harassment like this, it’s a lot of damage for a small number of people. Until Spamhaus started listing ESPs, no one knew it was happening. This includes the ESP that sent 81,000 confirmation emails to 9 email addresses over the course of 2 weeks.

We need new strategies

COI isn’t a great solution for this. In fact, the 81,000 emails were all COI requests. Captchas are not idea for a number of reasons, including discouraging signups from actual customers. We, as an industry, are going to have to think of ways to fix this. Yes, right now COI and captcha are the only solutions we have. But that doesn’t mean they are the only solutions, they’re just the stop gap. I don’t think it’s a huge secret that I don’t like the subscription validation companies very much, but they have the opportunity here to really stop this kind of abuse. No, their current SMTP tickling and delivery testing isn’t going to catch this (and, in fact, will cause problems for smaller targets), but there are other strategies they can create to address this.

Overall, this is something that needs to be addressed to prevent significant damage to individuals. Subscription forms need to be secured better and high volume senders need to pay attention to their address lists. One thing that was discovered is that this is not new. Some ESPs found a single address on thousands of their lists added over months. Low level abuse was happening, we didn’t see it because we weren’t looking. Now, we know it’s there and we must act to fix it.

3 Comments

Spamhaus and subscription bombing

Spamhaus released a blog post today discussing the recent subscription bombing: Subscription bombing COI captcha and the next generation of mail bombs.

As I mentioned in my initial posts, this abusive behavior goes beyond spamming. This is using email to harass individuals. Spamhaus even mentions a potential service that can be used to do these kinds of mailbombing.

Things folks need to know is that this is not just about ESPs and commercial mail. One of the big targets was WordPress admin forms. As Spamhaus says:

[T]he onus of stopping this kind of attack is not only on ESPs or mailing list owners. It is on everyone that has any sort of web-based signup that results in an email being sent: somebody clearly spent a great deal of time assembling URLs of mailing lists, and of account sign up pages, and has written a script to submit addresses to them at speed. We suspect that this was a test run for a tool that will will soon be offered for sale in the ‘Underground Economy’: Mail-bombing as a Service – MaaS.

With more and more abuse happening, every one who runs a service online needs to be cognizant of the abuse potential. Moreover even paths that have been around and haven’t been exploited may be exploited in the future.

We need to protect ourselves by making services that are difficult, if not impossible, to use as abuse vectors.

No Comments

Mail Client Improvements

There’s been extensive and ongoing development of email through the years, but much of it has been behind the scenes. We were focused on the technology and safety and robustness of the channel. We’re not done yet, but things are much better than they were.

The good part of that is there is some space to make improvements to the inbox as well. Over the last few months there have been a number of announcements from different mail client providers about how they’re updating their mail client.

Unsubscribes handled by the email client

Apple announced they were adding a link to unsubscribe into the mail client for iOS 10. It works much like the links in the Gmail and Hotmail clients, by looking for the list unsubscribe header and then sending a message to that address. Al did a bunch of testing and has a full blog post on how the list unsub link works in iOS, so go check out his post.

The important bits are they’re only using the mailto: link, they are not following any URLs. For those of you who want to support this, you’ll need to provide an address for unsubscribes. One of the absolute easiest ways to do this is use an encoded left hand side of the address so each unsubscribe can be processed based on the email address. Think of it like a VERP string.

It’s worth noting that two large players, Apple and Google, in the email client space have focused on the mailto: link for unsubscribes. There are issues that come up with an email client mediated unsubscribes, but a number of them go away using a http:// level unsubscribe.

It’s also interesting that this innovation has created a discussion among some marketers about where the unsubscribe link should be in an email. Some people feel very strongly that the only right place to put an unsubscribe link is at the bottom of an email. That choice is being somewhat removed from their hands with these changes to the mail client.

Better CSS and HTML support

Last month Microsoft went to the Litmus design conference and announced they were going to be working with Litmus and email senders to improve mail display in Outlook. This week Gmail announced they were supporting more CSS to make responsive design easier.

Displaying security information to end users

Earlier this year, Gmail started showing their users if mail came in over an encrypted connection. Mail sent without using TLS received an open red padlock next to the sender’s name.

This week folks noticed Gmail had quietly rolled out another feature to communicate security status to end users. Now, if you click on “show original” Gmail doesn’t simply show you a raw text version of the file, they show you specific authentication information about that message.

I have a screen shot of what that information looks like.

GoogleShowOriginal

There are multiple features here that make it easier to see what’s going on with email.

  1. How long the delivery took! This is great, because there are so many places email can get caught up. This will tell senders wether the problem is on the sending side or the receiving side. Looking at the headers of this particular message, the time is looking at how long the message took to get from the Gmail MX to the user’s inbox (or, in this case, spam folder).
  2. SPF pass. The learn more link is a little disappointing, as it  mostly talks about how you can implement SPF, not about what it means for recipients. It also says it helps recipients distinguish spam, except this particular message is a classic 419 spam. But it’s a good start.
  3. DMARC pass. Again, there isn’t much information about why a user should care about DMARC passing in the learn more link, but it is a good start.

Overall, these are exciting developments for recipients and senders. It’s really nice to see some work being done at making mail clients more descriptive. Because so much online security revolves around email, it’s a critical security step to show authentication results to end users. I expect some of these changes will be pushed out to the inbox over time, as Gmail wrestles with providing enough but not too much information.

All in all, these are more meaningful changes to email clients than I’ve seen in years.

No Comments

Microsoft deprecating SmartScreen filters

At the beginning of the month Microsoft announced that they were deprecating the SmartScreen filters used by the desktop Microsoft mail clients. These are the filters used in Exchange and various version of Outlook mail. This is yet further consolidation of spam filtering between the Microsoft free webmail domains, Office365 hosted domains and self hosted Exchange servers.  The online services (hotmail.com, outlook.com, Office365, live.com, etc) have been  using these filters for a while. The big change now is that they’re being pushed down to Exchange and Outlook users not hosted on the Microsoft site.

EOP was developed for Outlook.com (and friends) as well as Office365 users. From Microsoft’s description, it sounds like the type of machine learning engine that many providers are moving to.

Microsoft has published quite a bit of information about these filters and how they work on their website. One of the best places to start is the Anti-spam Protection FAQ. Something senders should pay attention to is the final question on that page: “What are a set of best outbound mailing practices that will ensure that my mail is delivered?” Those are all things  deliverability folks recommend for good inbox delivery.

Poking around looking at the links and descriptions, there is a host of great information about spam filtering at Microsoft and how it works.

A page of note is their Exchange Online Protection Overview. This describes the EOP process and how the filters work.

MS_filterProcess

An incoming message initially passes through connection filtering, which checks the sender’s reputation and inspects the message for malware. The majority of spam is stopped at this point and deleted by EOP. Messages continue through policy filtering, where messages are evaluated against custom transport rules that you create or enforce from a template. For example, you can have a rule that sends a notification to a manager when mail arrives from a specific sender. (Data loss prevention checks also occur at this point, if you have that feature; for information about feature availability, see the Exchange Online Protection Service Description.) Next, messages pass through content filtering, where content is checked for terminology or properties common to spam. A message determined to be spam by the content filter can be sent to a user’s Junk Email folder or to the quarantine, among other options, based on your settings. After a message passes all of these protection layers successfully, it is delivered to the recipient.

Well, if you ever wanted to know how Microsoft filters mail, now you do.

I’m also pleased to read that MS is continuing to support SNDS. I know it’s been problematic for folks lately and was somewhat concerned as the person who created SNDS recently moved to a new position. However, their FAQs still recommend SNDS so I think we can expect it to be maintained by new folks.

 

 

2 Comments

The perfect email

More and more I’m moving away from consulting on technical setup issues as the solution to delivery problems. Delivery is not about the technical perfection of a message. Spammers get the technical right all the time. No, instead, delivery is about sending messages the user wants. While looking for something on the blog I found an old post from 2011 that’s still relevant today. In fact, I’d say it’s even more relevant today than it was when I wrote it 5 years ago.

authenticated

Email is a fluid and ever changing landscape of things to do and not do.

Over the years my clients have frequently asked me to look at their technical setup and make sure that how they send mail complies with best practices. Previously, this was a good way to improve delivery. Spamware was pretty sloppy and blocking for somewhat minor technical problems was a great way to block a lot of spam.

More recently filter maintainers have been able to look at more than simple technical issues. They can identify how a recipient interacts with the mail. They can look at broad patterns, including scanning the webpages an email links to.

In short, email filters are very sophisticated and really do measure “wanted” versus “unwanted” down to the individual subscriber levels.

I will happily do technology audits for clients. But getting the technology right isn’t sufficient to get good delivery. What you really need to consider is: am I sending email that the recipient wants? You can absolutely get away with sloppy technology and have great inbox delivery as long as you are actually sending mail your recipients want to receive.

The perfect email is no longer measured in how perfectly correct the technology is. The perfect email is now measured by how perfect it is for the recipient.

No Comments

Archives