Starting next January, Google will be modifying its mobile search results to lower the ranking of sites that use interstitials that interfere with the users experience. In a blog post announcing the change they explain:
Pages that show intrusive interstitials provide a poorer experience to users than other pages where content is immediately accessible. This can be problematic on mobile devices where screens are often smaller. To improve the mobile search experience, after January 10, 2017, pages where content is not easily accessible to a user on the transition from the mobile search results may not rank as highly.
While this doesn’t have any effect on email delivery, I think it’s noteworthy to mention here for 2 reasons.
First, many interstitials are subscription boxes. If subscription boxes are considered an “intrusive interstitial” then websites may suffer lower visitation due to lower Google ranking. This will result in fewer signups from mobile devices. Removing the interstitial will reduce signup rates, another unwelcome consequence to this change. I don’t have a good solution, although it may be as simple as not showing interstitials to users coming directly from Google. Folks who use interstitials for signups should be looking at this issue now.
Second, it clearly demonstrates the priority Google puts on user experience. Many users get frustrated when they go to a site and there is immediately something blocking the information they’re looking for. Google has heard this and is trying to make their results less frustrating for users. This attitude is also a part of their filtering and blocking decisions. Mail that is deemed annoying or frustrating for users may go to the bulk folder, even when they’re lacking overt spam signs. We’ve certainly seen cases where mail gets filtered with no clear reason other than “people have reported mail like this as spam.”
Overall, I think consumers will appreciate the new search ranking algorithm. I think marketers are going to have to adapt in many ways, not the least of which is figuring out how to collect email addresses without compromising search engine rankings.
A few weeks ago, I got a call from a potential client. He was all angry and yelling because his ESP had kicked him off for spamming. “Only one person complained!! Do you know him? His name is Name. And I have signup data for him! He opted in! How can they kick me off for one complaint where I have opt-in data? Now they’re talking Spamhaus listings, Spamhaus can’t list me! I have opt-in data and IP addresses and everything.”
We talked briefly but decided that my involvement in this was not beneficial to either party. Not only do I know the complainant personally, I’ve also consulted with the ESP in question specifically to help them sort out their Spamhaus listings. I also know that if you run an open subscription form you are at risk for being a conduit for abuse.
This abuse is generally low level. A person might sign up someone else’s address in an effort to harass them. This is a problem for the victim, but doesn’t often result in any consequences for the sender. Last week’s SBL listings were a response to subscription abuse happening on a large scale.
We’ve generally accepted that low friction signup forms are a win for business. There aren’t many consequences to the business to maintaining them. That doesn’t mean all signups are low friction. Almost any social networking site will require some sort of confirmation before allowing full access to their platform. Certainly the big platforms – Twitter, Facebook, and LinkedIn to name a few – require new users to click a link to confirm their address. This is standard process that most internet users are familiar with.
Not all “networking” sites require confirmation, though. Over at Spamtacular Mickey talks about the Ashley Madison hack. He’s been reading through the report from the Canadian and Australian governments. He quotes the report:
The level of accuracy required is impacted by the foreseeable consequences of inaccuracy, and should also consider interests of non-users. This investigation looked at ALM’s practice of requiring, but not verifying, email addresses from registrants. While this lack of email address verification could afford individuals the ability to deny association with Ashley Madison’s services, this approach creates unnecessary reputational risks in the lives of non-users — allowing, for instance, the creation of a potentially reputation-damaging fake profile for an email address owner. The requirement to maintain accuracy must consider the interests of all individuals about whom information might be collected, including non-users.
The lack of email address verification creates unnecessary reputational risks in the lives of non-users.
At one point there was an argument that confirmation was an unfamiliar process and senders couldn’t trust the end users would confirm. That was true. It’s not longer true, though. While Facebook doesn’t publish their confirmation numbers, informal discussions tell me well over 90% of signups are confirmed. Confirmation is a standard process for users to go through these days.
One of the things some of us discussed, related to the Spamhaus issue, was that if enough government officials were hit then there might be legislation requiring some level of confirmation or protection. I don’t think it will happen any time soon. I don’t even think it’s likely. But there are the possibly apocryphal story of congress passing the TCPA because their fax machines were inundated with junk faxes. Could a similar attack on email addresses lead to legislation about open subscription forms?
A bit of older news, but worth a blog post. Early in August, Gmail announced changes to the inbox on both the web interface and the android client. They will be pushing authentication results into the interface, so end users can see which emails are authenticated.
These are not deliverability changes, the presence or absence of authentication will not affect inbox delivery. And the gmail Gmail support pages clarify that lack of authentication is not a sign that mail is spam.
This isn’t a huge change for most ESPs and most senders. In fact, Gmail has reported more than 95% of their mail is authenticated with either SPF or DKIM. Now, Gmail does a “best guess” SPF – if it looks like an IP should be authorized to send mail for a domain (like the sending IP is the same as the MX) then it’s considered authenticated.
It’s good to see authentication information being passed to the end user.
Brian Krebs posted a couple days ago about his experience with the subscription bomb over the weekend. He talks about just how bad it was over the weekend.
At approximately 9:00 a.m. ET on Saturday, KrebsOnSecurity’s inbox began filling up with new newsletter subscriptions. The emails came in at a rate of about one new message every 2-3 seconds. By the time I’d finished deleting and unsubscribing from the first page of requests, there would be another page or two of new newsletter-related emails. For most of the weekend until I got things under semi-control, my Gmail account was basically useless.
He also mentions this is something he’s been targeted with in the past.
This is malicious behavior on the part of the folks who are subscribing people. It is harassment.
I’m pleased at the number of ESPs and brands that are taking this seriously. We had a M3AAWG call this morning and much of the discussion was about how people are dealing with the issue. Some data is being shared here on the blog (signup IPs and stuff) and it’s very helpful.
If you are an ESP and you have data you want to share but don’t want to share it publicly contact me directly. The contact address works, I’m also on LinkedIn.
If you’re a recipient and you want some help cleaning up, feel free to contact me as well. I have some ideas of how we can help you and how you can help mitigate this for other people.
This isn’t a problem that’s going to just go away. We, as senders, cannot ignore the abuse. Now that this is out there we need to address it head on and protect both our brands, our network space and those unwilling recipients from being harassed through our services.
That does mean changes in behavior for all of us. Let’s not have the email space fall down on handling abuse like some of the social networking sites have.
Steve Linford, CEO of Spamhaus commented on my blog post about the current listings. I’m promoting it here as there is valuable information in it.
Excellent well summarized article Laura 🙂
No we’ve not changed SBL policy to require COI. It’s something we very strongly advise but we can not make a requirement. We’ll have to consider it if list-bombing of this magnitude can not be kept in check by list managers.
This incident involved a large number of government addresses belonging to various countries being subscribed to very large numbers of lists in a very short space of time by scripts run by the attacker(s). Most of the lists hit by the attack used COI and therefore only sent confirmation requests and did not subscribe any addresses. The attack undoubtably also hit lists which used Captcha in addition to COI and thus did not even proceed to COI (those list admins deserve some sort of community ‘hi 5’ award, since one can imagine how hard it is to convince one’s management to implement COI let alone put Captcha in front of it).
The issue is the badly-run ‘open’ lists which happily subscribed every address without any consent verification and which now continue as participants in the list-bombing of government addresses. These we are trying to address with SBL listings to prompt resolution by the Senders. As you noticed, most of these particular incident listings are for IPs ending “.0/32” which does not cause any mail issue to the Sender and is deliberately used where we have a good relationship with the Sender and know they will act quickly on the alert.
The Spamhaus Project
Efforts are ongoing to help ESPs clean up. Multiple commenters have been sharing data in the comments. If you have data you’d like to share with others, but don’t want to share it publicly please contact me directly.
Today Litmus announced they had partnered with Microsoft to fix many of the rendering issues with Outlook. Congrats, Litmus! This is awesome. I know a lot of folks have tried to get MS to the table to fix some of the problems with Outlook. Take a bow for getting this off the ground.
According to Litmus, the partnership has two parts.
- A rendering bugs feedback loop
- New Microsoft email clients available for testing in Litmus
Rendering bugs will be reported through Litmus to the MS development team. The new clients available for testing gives Litmus users the opportunity to code for existing and new email clients.
Great job, Litmus. This is amazing and awesome and I look forward to hearing about all the great things this partnership create.
A number of ESPs woke up to a more-than-usually-bad Monday morning. Last night Spamhaus listed 10s of networks, including ESPs, on the SBL. The listings all contained the following note:
The newsletter service () is using the referenced IP address to send bulk email. Unfortunately, the said newsletter service is not verifying the email address of new subscribers. Due to this, the service can be easily be abused to “listbomb” internet users.
To have this listing removed, the newsletter service needs to clean up their email address list and ensure that bulk emails are only being sent to recipients who have previously subscribed to their bulk email service.
In addition, the newsletter service needs to take the appropriate actions to prevent further abuse of their service:
a) Implementing CAPTCHA to prevent automated subscriptions
b) Implementing Confirmed Opt In (COI) to prevent that abusers can add random email addresses to the newsletter service that are not owned by the subscriber
c) Read the documentation below
Further information can be found on the referenced links below.
Mailing Lists -vs- Spam Lists:
Confirmed Opt In – A Rose by Any Name:
Spamhaus Marketing FAQ:
The first thing most folks did, when confronted with the listings, was reach out to other delivery folks. Is this something widespread or was just my ESP listed? The answer is many ESPs were involved.
Mail has been shooting back and forth all day between a number of players. Many folks reached out to contribute what they know. I think it’s a credit to the ESP and delivery community how free different folks have been with information.
(Note: the rest of this post is my synthesis of what I’ve been told from various sources, including Spamhaus. There are a lot of rumors here, but in the interest of getting things out quickly and calming some concerns I’m going to put this out now.)
That seems like a Spamhaus policy change.
I’ve not heard anything definitively one way or another about a policy change at Spamhaus. I think they’re all a bit busy. What I do know is that the listings are based on active abuse happening now. Over 100 addresses were added to mailing lists, many from IPs outside the US. These addresses are being mailed from the networks listed on the SBL and led directly to the listings.
It can’t be bad enough for a SBL listing.
Yeah, it can. I’ve had small subscription bombs in the past and they’re pretty damn annoying even when it’s only a couple dozen emails. The volumes I’m hearing here are significantly high that people cannot use their mailboxes. One sender identified fewer than 10 addresses each signed up to almost 10000 of their customer lists during a 2 week period. Most of those lists were actually COI, but even if they were all COI it still means tens of thousands of emails sent by one ESP to those email addresses. Expand that out to 10 ESPs and you have hundreds of thousands of emails sent to those email addresses.
Other senders have identified addresses that look to be part of the harassment campaign and are working to block mail to those addresses and get them off their lists.
So is this a policy change at Spamhaus?
Maybe, maybe not. It isn’t a policy change in that there is active email abuse coming from the listed networks. Spamhaus has long had the policy to list active systems actively involved in email abuse. It’s important to note that many (most?) of these listings are dot-zero listings and aren’t actually blocking mail. The goal is to get ESPs to clean up customers and stop the abuse.
What does Spamhaus expect us to do?
Speaking for myself, and without attempting to put any words in Spamhaus’ mouth, I think they expect you to stop the abuse currently coming from the listed networks. Right now, ESPs are being used as a conduit for abuse and people’s mailboxes are being rendered unusable by unsolicited mail from those networks. This is beyond the permission discussion, this is outright harassment and must be addressed.
How do we do that?
A number of ESPs have been searching through their client lists and identified addresses that have all been added to hundreds or thousands of lists. These are unlikely to be actual subscriptions and should be removed from lists. If the client insists on not removing these addresses, then I strongly suggest requiring they be confirmed with a positive confirmation (click here to continue receiving mail). These aren’t real subscriptions, though, I promise you. And, even if they are, even if that person was a great customer of yours and purchased from every mail they received, they will not be purchasing anything until the volume of their mail gets to something manageable.
The recipients should just unsubscribe.
That’s not really possible given the volume of mail. I’ve heard reports of some victims receiving over 100 emails per minute. More than 1 email per second. I don’t know about you, but I can’t unsubscribe in one second. This a form of harassment and will render a mailbox totally unusable. Subscription bombs like this are distributed denial of service attacks on individuals. They get so much mail from different places they are unable to use their mailbox for real mail. The hostile traffic can’t be blocked because the mail is coming from so many different sources.
What should we look for?
- Addresses that have signed up on many of your lists in August.
- The IP addresses used to sign up those addresses.
- Any other addresses signed up from those IPs.
This will give you a start at looking for the addresses that may be forged into forms. I’m seeing reports that some subscriptions started back on the 2nd and 3rd of August, so going back to Aug 1 makes a nice cutoff point.
OK, we’ve found them, now what?
Block them. Don’t allow your customers to mail them. You, and your customers, are being used as a vehicle to harass people. Then think about things you can do to identify this before it gets to the extreme of a SBL listing. This is the first public incident, I do not believe it will be the last.
Will Spamhaus be addressing this publicly?
I have been told that they should come out with a blog post over the next few days explaining some of the issue. They also know I’m writing about this issue, although they don’t know what I’m writing.
What do you think about this?
I think a number of things.
- I have hand waved over the risk of subscription bombs for years now. I really thought the era of widespread harassment using signups was over. I was wrong. This is an issue and it’s something the ESPs, and senders, are going to have to address.
- I’ve heard some talk over the last 16 – 22 months that indicated there was some low-level signup forgery going on. There was some discussion about whether or not this was bot activity and how this activity could be discovered and blocked. It never really went anywhere because we didn’t have good examples to investigate. We do now.
- I don’t believe this is a drastic shift in Spamhaus policy. They’ve always been about stopping mail to recipients who didn’t ask for it. This is a clear example of abuse and those companies listed are sending large amounts of unsolicited email, if only to a few people. Most of the listings aren’t blocking mail and from what I hear Spamhaus is working closely with the ESPs involved.
- I do believe this incident demonstrates why you need to pay attention to your subscription process and numbers. While in this case neither COI or a welcome series would minimize the effect of the subscription bomb, in less drastic cases you can avoid being a conduit for harassment by limiting the number of emails you send to someone who never, ever responds.
- Internet harassment seems to be a bigger and bigger issue. I don’t know if it’s because people are being more open about harassment or if it’s actually more common. In either case, it is the responsibility of networks to minimize the harassment. If your network is a conduit for harassment, you need to do something to stop it. I’m working on a couple pieces related to the responsibility of networks to prevent harassment through their services because it is becoming such a major issue.
Overall, I think this should be a major wakeup call for ESPs and senders. You’re being used as a conduit for harassment and you have a responsibility to the overall ecosystem and your customers to stop it.
We are trying to evaluate the success of our email programs, and I don’t have a good sense of what metrics we should be monitoring. We have a lot of data, but I don’t have a good sense of what matters and what doesn’t. Can you advise us what we should look at and why?
Metrics Are Hard
You’re not going to like this answer, but here goes.
If you’re sending newsletters and general brand mail, you’ll want to track clicks and opens to look at how engaged your recipients are with your content. This will help you evaluate the success of individual messages and campaigns, as well as your larger program efforts. You can also use this information to further segment and market to your most engaged (or least engaged) recipients.
If you’re sending marketing mail, you need to look at revenue as well. You need to understand how email engagement translates to purchases, both by campaign and over the customer lifetime.
And for any kind of mail you send, you need to keep an eye on bounces, complaints and unsubscribes. These can be valuable early indicators of both technical issues and marketing success.
The biggest question is: what data do you have access to? When we talk to clients, we often find that they have SO MUCH DATA, but they have no idea how to analyze it and make sense of what they’re seeing. As you point out, there are a lot of numbers to look at. Whether you’re sending mail directly or working with an email service provider, you likely have more dashboards and reports than you know what to do with. You need to figure out what you have and what matters most to you.
On the deliverability front, you can look at your logs to see if there are any ISPs temp failing mail. This will tell you if there’s some reputation issues. Y! and AOL both have specific codes for “come back later” and they’re helpful to ID if there’s something problematic with your reputation.
“Unknown users” is also a valuable metric. If you’re using a data hygiene service, you’ll want to monitor how many addresses they’re removing. If it’s more than 1 – 5%, then you need to look at your address collection process.
Opens and clicks are reasonable metrics to measure. Marketers also look at click-to-open-rate (CTOR), but that’s not something I use for deliverability — It’s more about how many people are interacting with your mail.
Mailbox monitoring tools are less useful than they were, but can still provide interesting information.
Another useful thing to consider is to identify what filters your mail goes through. We do this by taking the MX for every domain on a mailing list, and then identifying the number of email addresses behind each MX.
Overall, you want to make sure you’re looking at the same metrics over time so you can be aware of significant changes in delivery and marketing effectiveness. Depending on your mail types and volumes, there are numbers you’ll want to look at daily, others weekly or monthly, and still others only as needed. Ultimately, there’s no one-size-fits-all answer to what metrics matter to businesses — it’s up to you to determine what matters most to you.
Confused about delivery in general? Trying to keep up on changing policies and terminology? Need some Email 101 basics? This is the place to ask. We can’t answer specific questions about your server configuration or look at your message structure for the column (please get in touch if you’d like our help with more technical or forensic investigations!), but we’d love to answer your questions about how email works, trends in the industry, or the joys and challenges of cohabiting with felines.
One of the themes in some of my recent talks has been how some marketers teach their customers to become victims of phishing. Typically I’m talking about how companies register domains “just for email” and then use those for bulk messages. If customers get used to mail from company.ESP.com and companyemail.com they’re going to believe that company-email.com is also you.
There are other ways to train your customers to be phishing victims, too. Zeltzer security walks us through a couple emails that look so much like phishing that it fooled company representatives. Go take a read, they give a number of examples of both good and bad emails.
I was a little frustrated that the examples don’t include headers so we could look at the authentication. But the reality is only a teeny, tiny fraction of folks even know how to check headers. They’re not very useful for the average user.
Security is something we should never forget. As more and more online accounts are tied to our email addresses those of us who market to email addresses need to think about what we’re teaching our recipients about our company. DMARC and other authentication technologies can help secure email, but marketers also need to pay attention to how they are communicating with recipients.
I am honored to be included in the Learn to Fish document built by Adobe.