BLOG
The coming changes
Yesterday I talked about how I’m hearing warnings of a coming paradigm shift in the email industry. While these changes will affect all sender, ESPs in particular are going to need to change how they interact with both ISPs and their customers.
Currently, ESPs are able to act as “routine conveyers.” The traffic going across their network is generated by their customers and the ESP only handles technical issues. Responsible ESPs do enforce standards on their customers and expect mailings to meet certain targets. They monitor complaints and unknown users, they monitor blocks and reputation. If customers get out of line, then the ESP steps in and forces their customer to improve their practices. If the customer refuses, then the ESP disconnects them.
Currently standards for email are mostly dictated by the ISPs. Many ESPs take the stance that if any mail that is not blocked by the ISPs then it is acceptable. But just because a certain customer isn’t blocked doesn’t mean they’re sending mail that is wanted by the recipients.
It seems this reactive approach to customer policing may no longer be enough. In fact, one of the large spam filter providers has recently offered their customers the ability to block mail from all ESPs with a single click. This may become a more common response if the ESPs don’t start proactively policing their networks.
Why is this happening? ISPs and filtering companies are seeing increasing percentages of spam coming out of ESP netspace. Current processes for policing customers are extremely reactive and there are many ESPs that are allowing their customers to send measurable percentages of spam. This situation is untenable for the filtering companies or the ISPs and they’re sending out warnings that the ESPs need to stop letting so much spam leave their networks.
Unsurprisingly, there are many members of the ESP community that don’t like this and think the ISPs are overreacting and being overly mean. They do not think the ISPs or filtering companies should be blocking all an ESPs customers just because some of the customers are sending unwanted mail. Paraphrased, some of the things I’ve heard include:
- But we segregate out customers onto separate IPs, why can’t they just block the spammers?
- But we’re doing everything we can to police our customers, why can’t they just understand that?
- What more do they expect us to do?
- How can we stop our customers from spamming? We don’t send the mail.
The ISPs don’t really care about any of that. They’re seeing spam coming from an ESP and they expect the ESP to make it stop. This is it, ESPs, you’ve now been accepted as full members of the email ecosystem and are now expected to police the traffic coming off your IP space. It is no longer sufficient to segregate customers onto their own IPs and let the ISPs block unwanted mail. ESPs are now expected to do their own policing and their own monitoring.
This isn’t anything new. The ISPs went through this with regards to the email their customers were sending 8 years ago or so. There were ISPs that didn’t effectively police their user base. Infections, bots, spammers signing up… some ISPs would take spammer money and expect other ISPs to sort out wanted from unwanted (spam from non-spam) traffic. Finally, the non-spammer supporting ISPs got tired of it and started blocking the spammer supporting ISPs.
The widespread blocking caused a large shift in the industry. There was also a lot of Sturm und Drang about how wrong it all was and how legitimate customers were collateral damage. Despite this the message to ISPs was clear: police your networks. Policing networks proactively cost a lot of companies a lot of money as they work out how to identify bad traffic before it left their networks. They had to develop or purchase software to identify the traffic and block or mitigate it.
Now, it’s the ESPs turn. Much like happened to the ISPs years ago, the ESPs aren’t sure how to react or what they can monitor. Many ESPs do have proactive monitoring in place, but these strategies are failing. Spam is coming off some networks, and the whole network is at risk for blocking, not just the bad customers.
The truth is, though, that ESPs have as much control over their own IP space as ISPs do – and those ISPs are expected to control the amount of spam leaking out their systems. ISPs are starting to expect ESPs, who are now participants in MAAWG and IETF and such, to step up and control the amount of spam leaking out of their systems, too. This is why we’re starting to see wider blocking by ISPs and spam filters of ESPs and their customers.
There is a clear opportunity here for smart ESPs to stand out from their peers and competitors. ESPs are being told that things are changing, and how those things are going to change. How is your business going to adapt? What are you going to do to stop your customers from sending spam?
While I’m with you in spirit, I’m not optimistic that “blocking ESPs” will have much of a lasting impact. It’s the same old arms race. If a large ISP blocks an entire ESP, the ESP will respond by obtaining new IP space in such a way that it can’t be identified as a group. Cloud computing is only going to make this easier. If they block based on something in the headers or body, the ESPs will remove those those distinguishing marks.
A good example of this was the Precedence: Bulk header. At one time that was useful… now it is mostly regulated to mailing lists. Most email marketing software stopped using it because so many people would block based on it.
Please don’t get me wrong. I’m not trying to defend any ESPs or say that the ISP (or more importantly the user) doesn’t have the right to block them. I’m just skeptical that it will be effective on a large scale.
It won’t work for everyone, but that’s why I use disposable email addresses for everything I sign up for. That way I know I am in control and I get to decide when to stop receiving email from someone. Also, I know if they share my address or it gets stolen.
For the masses who aren’t going to take the time to use disposable addresses, I’m a big fan of the “bacn” direction that companies like OtherInbox (my company) and Mozilla Raindrop are taking, which to be honest, is even a bit more aggressive. Rather than targeting ESPs (which could be hard if they try to hide) we target all commercial email (which is actually pretty easy). An ESP can change their IPs or headers, but Amazon.com has to send their email from Amazon.com or people won’t recognize it and won’t open it. We don’t block it all, but we segregate it from the Inbox and give the user tools to better manage that type of bulk email. At the Defrag Conference last month a Microsoft researcher showed a similar type of solution that they were prototyping.
I’m also really hopefully that domain based reputation will continue to gain momentum and dominate over IP based reputation. I don’t think that solves all the problems, but I do think it will help.
Clearly, as you say, something has gotta give, and change is the winds. Spammers aren’t going to be able to get away with the same old tactics they have been using.
[…] Holding ESPs to the same “good neighbour” standards as ISPs Posted in Uncategorized by Roland Turner on December 10, 2009 An interesting post by Laura Atkins on a trend towards raising the bar for bulk senders. […]
Great article! Hopefully this will push ESPs and their clients into thinking more about their reputation and working with ISPs instead of seeing them as a hindrance. At the moment, I think that many ESPs are too lax in enforcing their own policies in regards to SPAM, especially the ones that charge on volume.
Josh says “If a large ISP blocks an entire ESP, the ESP will respond by obtaining new IP space in such a way that it can’t be identified as a group. Cloud computing is only going to make this easier. If they block based on something in the headers or body, the ESPs will remove those those distinguishing marks.” Well, we have “ESPs” who do that now. They’re called “spammers”. They do everything they can to obfuscate who is sending the message, in hopes that a few more will make it into the inbox. What they end up doing is proving the illegitimacy of their messages, and their disdain for the wishes of both end users and network owners. Properly permissioned mail, sent to engaged customers who have asked for, and want, the messages, is the way to ensure that your mail won’t be blocked. Engaging in questionable acts, gaming the system(s), and pushing the boundaries of what you want to call ‘permission’ (e-pending, for example) are the sorts of things upon which receiving ISPs are going to continue to frown.
David, it’s not that simple. The definition of obsfucation is that you can’t tell those messages from legitimate ones. Your message implies that ESPs who do this will be blocked. But the whole reason they are doing it is to make it so that they can’t be blocked. The more pressure is put on ESPs, the more that even legitimate ones with good permission practices will be forced to obsfucate their messages.
I think Constant Contact is a whitehat ESP with pretty good practices. They focus on customers that send good email and they have a very responsive abuse desk that takes immediate action against spammers. But they are also a public company. I guarantee you that if the big ISPs started blocking all email from ESPs that CC would respond by making their email look less like an ESP. From their perspective, they are being unfairly filtered and they are not trying to sneak in spam, they are trying to deliver opt-in emails that were requested.
That’s why I think its more effective to focus on the true sender rather than the ESP. Domain-based reputation is a big step in that direction. The true sender can’t obscure who they are.
The more pressure is put on ESPs, the more that even legitimate ones with good permission practices will be forced to obsfucate their messages.
This is where you and I disagree.
ISPs have to respond to the wants and needs of their userbase. For years, their customers have been complaining about the masses of what, for the sake of brevity, I’ll call the “illegitimate” spam – the fake Viagra and the cheap designer watches and the myriad other blatantly bad things. The ISPs have been working hard to stop those things from going into their customers’ inboxes and, for the most part, they’ve succeeded[1]. This means that end users’ inboxes are much cleaner than they’ve been historically, and it means that those end users now have a better chance of seeing and reading the mail that’s in them. Those end users are starting to see, and complain vociferously about, the “legitimate” spam they’re getting – the marketing material they didn’t actually ask for, but which is being sent to e-pended addresses, or the newsletter that they used to want but have tried three times to unsubscribe from, or the mounds of third-party offers that they’re receiving because they gave a website their email address.
Senders want to send mail to as many people as possible. ESPs want their IP addresses (and, indeed, their business names) to maintain good reputations. These wants are, in a way, mutually exclusive. It’s imperative that ESPs start taking seriously their responsibility for the survival of the email ecosystem. ESPs who do that – those who ensure that the customers they’re bringing on board have properly permissioned mailing lists, and who act quickly to terminate customers who don’t, and who teach senders what being a good net-neighbor is about – are not going to see delivery problems. ESPs who allow their customers to engage in practices that are less clearly legitimate, like e-pending or sending to suppression lists or failing to honor unsubscribe requests, are the ones who are going to have the Big Red Button of No Mail pressed.
Just as we expect ISPs to shut down botted customers to prevent them from flooding other networks, so too should we expect ESPs to terminate abusive customers. “But they pay us a lot of money” should never be an excuse for allowing abusive practices to continues.
[1] No, the ‘spam problem’ isn’t solved. Botnet, snowshoe, and other illicit delivery methods still account for the great majority of email traveling the network, and ISPs still have to work hard to stop it. But it’s not making it into the inbox and, from the point of view of the userbase, that’s what’s important.
Josh, the people that obtain new space under different names aren’t actually called “ESPs,” they’re called “potential future defendants.”
They are not ESPs, nor do ESPs do what they do. If you think ESPs do this, or would do this, your definition of ESP differs greatly from mine. I do NOT mean “leaders in the co-reg and lead gen space who already hide who they are, straddling 12 different ESPs and various European netspace, scooped up by the pound from RIPE.”
Also, the methodology falls down, as I know people are already experiencing. This anger from ISPs stems from observations of mail streams. Move the mail stream to another source, and it’s still observable. The ISP is going to pick up on that new location, and block in there too. Of course, sure, some people will try to dance around it. But again, those are people that are ALREADY dancing, and I am reliably informed that it gets harder and harder for them to reach the inbox successfully.
[…] has an outstanding post about coming changes to reputation systems up at her blog. If you haven’t had a look at it, you […]
[…] has an outstanding post about coming changes to reputation systems up at her blog. If you haven’t had a look at it, you […]
[…] has an outstanding post about coming changes to reputation systems up at her blog. If you haven’t had a look at it, you […]
[…] has an outstanding post about coming changes to reputation systems up at her blog. If you haven’t had a look at it, you […]
[…] has an outstanding post about coming changes to reputation systems up at her blog. If you haven’t had a look at it, you […]
[…] has an outstanding post about coming changes to reputation systems up at her blog. If you haven’t had a look at it, you […]
[…] has an outstanding post about coming changes to reputation systems up at her blog. If you haven’t had a look at it, you […]
Wow, Josh, maybe you deserve your reputation after all.
IF an ESP is whitehat, and they claim to have all good customers with Opt-in lists and they get blocked by a major ISP it is most likely not “unfair” treatment but instead indicative of a problem at that ESP that the ESP was not aware of. That ESP would then work towards cleaning up their act if they could figure out what they did wrong. Jumping IP space is the furthest thing from my mind as an ESP.
“There is a clear opportunity here for smart ESPs to stand out from their peers and competitors.”
Yes and no. If, as was indicated, some ISPs or filtering companies start blocking all ESPs indiscriminately then it actually isn’t. If, however, ISPs simply start using collateral damage within an ESPs range to force policy changes then it clearly can be a competitive differentiator for ESPs.
BTW, I don’t think we’ll see change from ESPs without this kind of pressure. Legitimate ESPs rationalize inappropriate client behavior all the time and continue to permit eappend, list purchase etc. Only actual business impact from ISPs is going to change them IMHO.
Yes and no. If, as was indicated, some ISPs or filtering companies start blocking all ESPs indiscriminately then it actually isn’t.
I actually have been talking with the filtering company offering it, and will be posting about that, soon. In the case where it’s happening it’s not the ISPs, it’s primarily targeted at corporate customers and the filtering is all opt-in.
“The avalanche has already started…” Expect other filter vendors to implement similar features (whether happy about it or not), simply because the current nature of filter competition requires that feature checkboxes used in comparative reviews be filled.
From where I sit, there already are a tiny number of ESPs whose exemplary efforts set them apart from the competition. In my opinion this is a very good thing, and should encourage filter vendors to include more than just one checkbox – enabling some end user control over specific ESP blocking. In my opinion it’s high time good sender behavior engenders a tangible competitive advantage.
Josh Baer wrote: If a large ISP blocks an entire ESP, the ESP will respond by obtaining new IP space in such a way that it can’t be identified as a group.
The difficulty that ISPs currently have blocking ESPs is precisely BECAUSE they’re ESPs — it’s hard to block someone who has occasionally spammy content, but who also has online stock trade confirmations, airline reservations, bank statements, and so on. The reason ESPs have been getting somewhat of a free pass (…until now, apparently) is because they really are different.
…however, as soon as they start acting more like spammers — …well, the ISPs are already pretty good at dealing with people who look like spammers. So, if you’re honestly suggesting this as a process model going forward? Good luck. Let me know how that works out for you.
[…] to the Wise blog: e.g. The coming changes and ISPs are speaking, is anyone […]
[…] The coming changes ISPs and filtering companies are seeing increasing percentages of spam coming out of ESP netspace. Current processes for policing customers are extremely reactive and there are many ESPs that are allowing their customers to send measurable percentages of spam. This situation is untenable for the filtering companies or the ISPs and they’re sending out warnings that the ESPs need to stop letting so much spam leave their networks. (tags: deliverability industry spam) […]
Huey – I’m not an ESP anymore… I’m a receiver protecting email inboxes at OtherInbox. So this isn’t about anything I want to do. I’m not trying to defend ESPs or derail Laura’s efforts – I support them! I am hoping to contribute positively to the conversation, express some concerns I have and suggest other constructive solutions to help address the issue.
My comments were not about meant to be about ISPs blocking an individual ESP, they were about the suggestion that major ISPs were going to provide an option to “block ALL ESPs” which implied that it would include the whitehats as well. In that case, I still do believe that even the best ESPs would be forced to obsfucate their messages.
I should have been more careful in the wording of my first comment – I really was focused on the idea of an ISP blocking ALL ESPs, which was the focus of Laura’s original post. This isn’t about one ESP being blocked or singled out – this is about ALL ESPs being blocked categorically.
Barry – take a look at what OtherInbox does. I’m 100% focused on putting the consumer in control of the email they receive and helping them to protect their inbox from spammers. I’m not trying to defend ESPs. I’m not suggesting that this is behavior that I’m going to partake in. If it came across otherwise then I apologize for not being clear.
Tara – my understanding of Laura’s original posting above is that it was not about ONE ESP getting blocked (nothing new there) but about ALL ESPs being categorically blocked by a single checkbox. This would clearly not be an ESP being singled out for bad behavior as you suggest. If this was just about one ESP, then I agree with your comments. But I don’t think that’s what we were talking about here.
To Al Iverson’s comment – many top tier ESPs provide customers with an option for dedicated IP space in the name of the customer sending the email. There is nothing illegal about this and there are often many legitimate reasons to do it. Then there are hybrid solutions like StrongMail that act mostly like an ESP but drop the mail server on the customer’s network so that none of the email has their name on it.
Again, I’m not defending ESPs. I’m not trying to derail the efforts to reduce spam, which I agree is out of control and just getting worse. I’m actively working on solutions to help consumers reduce spam and protect their inboxes, with much success. I’m not advocating that ESPs SHOULD obsfucate their email, I’m just concerned that if a “block all ESPs” approach is taken that many of them WILL do that. I think that there may be better ways to affect sender behavior.
Thank you to Laura for starting this conversation and allowing me to participate and comment.
I really was focused on the idea of an ISP blocking ALL ESPs, which was the focus of Laura’s original post. This isn’t about one ESP being blocked or singled out – this is about ALL ESPs being blocked categorically.
No, Josh, that wasn’t the focus of my original post. The focus of my original post is that ESPs are now expected to do more than the bare minimum. They are expected to actually police their users and stop spam from coming across their networks. It is a warning that there are changes in the air, and ESPs who fail to stop their customers from spamming may face larger blocking problems in the future than they do now.
[…] Iverson has responded to one of the comments on my “Changes” post. Tweet This!Share this on FacebookDigg this!Share this on del.icio.usEmail this to a […]
Thanks for the clarification Laura. I certainly agree.
When ISPs pressure each other to clean up their outbound traffic, they usually start with quiet, one-on-one conversations — but not always. Take a look at the archives for the NANOG or inet-access mailing lists, and you’ll find many examples of what is (in effect) public shaming of unresponsive providers. That was true eight years ago, and fifteen years ago, and it’s true today.
ESPs certainly haven’t been shy in complaining about what they consider to be unfair practices by ISPs. ISP and anti-spam vendor staff tend to be classier than that, but maybe that silence has been working against them?
ISP and anti-spam vendor staff tend to be classier than that, but maybe that silence has been working against them?
I don’t think the ISPs and anti-spam vendors have necessarily been silent. They are talking to the providers, individually pointing out that there are customer problems and that the providers need to stop their customers from continuing to spam. It’s not public shaming, yet. I also don’t see it becoming public, I think blocks will just start growing.
[…] been quite a bit of discussion on my post about upcoming changes that ESPs will be facing in the future. One thing some people read into the post is the idea that […]
Josh wrote: My comments were not about meant to be about ISPs blocking an individual ESP, they were about the suggestion that major ISPs were going to provide an option to “block ALL ESPs” which implied that it would include the whitehats as well. In that case, I still do believe that even the best ESPs would be forced to obsfucate their messages.
To steal from Gen Norman Schwarzkopf: I would characterize that suggestion as ‘bovine scatology’. The best ESPs are aware, perhaps more than anyone else, that when someone indicates they don’t want your mail, the first and most important thing you need to do is to stop sending it to them. And this principle remains sound whether that indication is made at an end-user level or at a domain-wide level. Why would people who are already trying to do all the right things spontaneously decide to start acting like spammers? How would that help?
[…] The coming changes from: Word to the Wise The ISP […] are seeing spam coming from an ESP and they expect the ESP to make it stop. This is it, ESPs, you’ve now been accepted as full members of the email ecosystem and are now expected to police the traffic coming off your IP space. It is no longer sufficient to segregate customers onto their own IPs and let the ISPs block unwanted mail. ESPs are now expected to do their own policing and their own monitoring. […]
[…] pourront s’appuyer sur une liste de confiance (sur le modèle de la certification Microsoft). E-mailing : les changements à venir […]
[…] my recent series of posts has indicated, I am seeing a lot of future changes in the email […]
[…] is something I talked about a few weeks ago, in a series of blog posts looking at the changes in 2010. The signs are out there, and companies with marginal […]