Email filtering: not going away.

VirusBlockI don’t do a whole lot of filtering of comments here. There are a couple people who are moderated, but generally if the comments contribute to a discussion they get to be posted. I do get the occasional angry or incoherent comment. And sometimes I get a comment that is triggers me to write an entire blog post pointing out the problems with the comment.
Today a comment from Joe King showed up for The Myth of the Low Complaint Rate.

laura, imagine situation where a postman is standing in front of your mailbox and is throwing your mail into the trash instead of your mailbox because he _thinks_ that this piece of mail is a junk.
thank God we not there, living in your bubble.
sooner or later congress will work on fixing internet mail and just like for fedex it is illegal to throw away your mail, it will be criminal offence for microsoft or yahoo to throw away your email.
and customers notice the flaw too. hotmail is a joke nobody uses it and yahoo mail numbers are on decline for years.

Let’s go with the easy bit first. Postal mail is not email. The sender of a letter pays the post office or the package delivery service to deliver that mail to recipients. In fact, the entire postal system is paid for by senders. This isn’t the case with email (or, really, any internet traffic, thus the battle over net neutrality). Sure, senders pay for their connectivity. But receivers pay for their connectivity as well. Some pay directly for their email address as part of their broadband or dialup accounts. Businesses pay for business emails in hardware or hosted solutions. Even free mail users (including Yahoo/Rocketmail users like Joe up there) “pay” for email by viewing advertising.
The postal world isn’t some sort of automatic delivery system either. There are rules, things that cannot be sent through the mail and filters to limit the amount of illegal postal mail. In fact, there are much stronger consequences for breaking postal regulations than simply having mail thrown away. The USPS has an entire division dedicated to protecting recipients from harmful mail and from scams carried out through the USPS. Certain types of mail are prohibited by the postal service and they screen and stop mail prohibited by law.
In the email world, we’re not looking at things like bombs or dangerous biologics, but there is and will always be a place for filtering. There’s too much bad mail out there. In fact, email has been a common vector in many of the large scale hacks at companies. Phishing was a primary vector for the Target hack. The RSA hack relied on phishing to get in the door and compromise their root keys. Other notable attacks using spear phishing include ICANN and Home Depot.
Sure, these are big corporations, but the same sorts of viruses and phishes are aimed at individual users on free mail systems every day. They might not have access to the data that large companies do, but their systems are still worthy of protection. Right now a lot of users are protected from most of the viruses and phishes and malware and other harmful mail due to filtering. They don’t even know they’re protected from it, because it never comes through them. But there is a LOT of bad mail out there and it should not be delivered.
What’s more, recipients want their email filtered. They want ISPs to filter out junk mail. They make aggressive use of the “this is spam” button. “Too much spam” is one of the reasons people change email addresses.
Finally, ISPs are major corporations with shareholders, bottom lines and lobbyists. Spam costs these ISPs money, real money, to handle. The ISPs have always been on the side of being able to block bad mail. They’re not going to let their Congress pass a law that requires them to deliver every incoming email message.
I don’t believe that there will be a law requiring ISPs to open the floodgates and deliver every piece of email. Even if there were, an unfiltered mailbox is a difficult thing to manage. I know, we run limited filters and I end up in my junk folder at least weekly because I got a little ^J happy and junked a message I shouldn’t have.

Related Posts

Abuse it and lose it

Last week I blogged about the changes at ISPs that make “ISP Relations” harder for many senders. But it’s not just ISPs that are making it a little more difficult to get answers to questions, some spam filtering companies are pulling back on offering support to senders.
For instance, Cloudmark sent out an email to some ESPs late last week informing them that Cloudmark was changing their sender support policies. It’s not that they’re overwhelmed with delisting requests, but rather that many ESPs are asking for specific data about why the mail was blocked. In December, Spamcop informed some ESPs that they would stop providing data to those ESPs about specific blocks and spam trap hits.
These decisions make it harder for ESPs to identify specific customers and lists causing them to get blocked. But I understand why the filtering companies have had to take such a radical step.
Support for senders by filtering companies is a side issue. Their customers are the users of the filtering service and support teams are there to help paying customers. Many of the folks at the filtering companies are good people, though, and they’re willing to help blocked senders and ESPs to figure out the problem.
For them, providing information that helps a company clean up is a win. If an ESP has a spamming customer and the information from the filtering company is helping the ESP force the customer to stop spamming that’s a win and that’s why the filtering companies started providing that data to ESPs.
Unfortunately, there are people who take advantage of the filtering companies. I have dozens of stories about how people are taking advantage of the filtering companies. I won’t share specifics, but the summary is that some people and ESPs ask for the same data over and over and over again. The filtering company rep, in an effort to be helpful and improve the overall email ecosystem, answers their questions and sends the data. In some cases, the ESP acts on the data, the mail stream improves and everyone is happy (except maybe the spammer). In other cases, though, the filtering company sees no change in the mail stream. All the filtering company person gets is yet another request for the same data they sent yesterday.
Repetition is tedious. Repetition is frustrating. Repetition is disheartening. Repetition is annoying.
What we’re seeing from both Spamcop and Cloudmark is the logical result from their reps being tired of dealing with ESPs that aren’t visibly fixing their customer spam problems. Both companies are sending some ESPs to the back of the line when it comes to handling information requests, whether or not those ESPs have actually been part of the problem previously.
The Cloudmark letter makes it clear what they’re frustrated about.

Read More

Engagement, it's not what you might think

Most delivery experts will tell you that ISPs measure recipient engagement as a part of their delivery. That’s absolutely true, but I think there’s a language difference that makes it hard for senders to understand what we mean by engagement.
ISPs, and other filtering companies, profile their user base. They know, for instance, who logs in and checks mail every day. They know who checks mail every 20 seconds. They know who gets a lot of spam. They know who hasn’t logged in for months. They know who accurately marks mail as spam and who is sloppy with the this-is-spam button. They know if certain recipients get the same mail, it’s likely to be spam.
Engagement at the ISPs is more about the recipient engaging with their email address and the mail in their mailbox then it is about the recipient engaging with specific emails.
 

Read More

ISP Relationships

Delivra has a new whitepaper written by Ken Magill talking about the value (or lack thereof) of relationships with ISPs. In Ken’s understated way, he calls baloney on ESPs that claim they have great delivery because they have good relationships with ISPs.
He’s right.
I get a lot of calls from potential clients and some calls from current clients asking me if I can contact an ISP on their behalf and “tell the ISP we’re really not a spammer”. My normal answer is that I can, but that there isn’t a place in the spam filtering process for “sender has hired Laura and she says they’re not a spammer.” I mean, it would be totally awesome if that was the case. But it’s not. It’s even the case where I’m close friends with folks inside the ISPs.
I’m pretty sure I’ve told the story before about being at a party with one of the Hotmail ISP folks. There was a sender that had hired me to deal with some Hotmail issues and I’d been working with Barry H. (name changed, and he’s not at Hotmail any more) to resolve it. During the course of the party, we started talking shop. Barry told me that he was sure that my client was sending opt-in mail, but that his users were not reacting well for it. He also told me there was no way he could override the filters because there wasn’t really a place for him to interfere in the filtering.
Even when folks inside the ISPs were willing and able to help me, they usually wouldn’t do so just because I asked. They might look at a sender on my request, but they wouldn’t adjust filters unless the sender met their standards.
These days? ISPs are cutting their non-income producing departments to the bone, and “sender services” is high up the list of departments to cut. Most of the folks I know have moved on from the ISP to the ESP side. Ken mentions one ISP rep that is now working for a sender. I actually know of 3, and those are just employees from the top few ISPs who are now at fairly major ESPs. I’m sure there are a lot more than that.
The reality is, you can have the best relationships in the world with ISPs, but that won’t get bad mail into the inbox. Filters don’t work that way anymore. That doesn’t mean relationships are useless, though. Having relationships at ISPs can get information that can shorten the process of fixing the issue. If an ISP says “you are blocked because you’re hitting spam traps” then we do data hygiene. If the ISP says “you’re sending mail linking to a blocked website” then we stop linking to that website.
I have a very minor quibble with one thing Ken said, though. He says “no one has a relationship with Spamhaus volunteer, they’re all anonymous.” This isn’t exactly true. Spamhaus volunteers do reveal themselves. Some of them go around openly at MAAWG with nametags and affiliations. A couple of them are colleagues from my early MAPS days. Other do keep their identities secret, but will reveal them to people they trust to keep those identities secret. Or who they think have already figured it out. There was one drunken evening at MAAWG where the nice gentleman I was joking with leaned over and says “You know I am elided from Spamhaus, right?” Uh. No? I didn’t. I do now!
But even though I have the semi-mythical personal relationship with folks from Spamhaus, it doesn’t mean my clients get preferential treatment. My clients get good advice, because I know what Spamhaus is looking for and can translate their requirements into solid action steps for the client to perform. But I can think of half a dozen ESP delivery folks that have the same sorts of relationships with Spamhaus volunteers.
Overall, relationships are valuable, but they are not sufficient to fix inbox delivery problems.

Read More