JD posed a question in my post about Postini and trying to sort out a customer getting marked as spam by their filtering mechanism and I think it bears more discussion than can be done in comments.
And sure, it’s a best practice for filtering companies to respond politely to requests from filterees. But is it a requirement? Do senders have a right to demand explanations?
There is not really an easy answer for that. My first response is “of course not!” but then I think about some of my clients who really have been trying to do the right thing and how we work through issue after issue and finally fix everything I can think of, but they still have delivery problems. These are not spammers, they are sending mail to people who have asked for it and by all measures do actually want it, but some mail is being blocked for reasons neither my client or I can figure out. In those cases it would be really nice if someone from the group doing the blocking would take 10 minutes to point me in the right direction and show me what I missed.
I have been doing this long enough to know that spamfilters are not 100% accurate. I know there are times when a specific block is outside the scope of what email the filter designer, or user, expected to block. Look at what happened when Yahoo started using the PBL a few months back. There was a bug in the implementation that neither Yahoo nor Spamhaus expected and that caused mail from IPs not listed on the PBL to be blocked because of the PBL. With a valid report of the problem, I could contact both Spamhaus volunteers and someone at Yahoo to point out there was a problem with the implementation. Yahoo and Spamhaus figured out the issue and fixed the problem and Yahoo is no longer blocking IPs not on the PBL for being on the PBL.
I do believe that there are times when feedback from senders and blockees is beneficial and can help improve the overall filters. I have clear evidence this is the case.
On the flip side, I also have been in the email business long enough to know that more than 99% of senders just want their mail delivered and do not care about anything other than getting into the inbox. They believe every block is a mistake and the ISP / spamfilter is wrong or broken. They are not interested in actually making sure the implementation of the filter meets the design goals, usually they do not care what the goals of that filter are. They are just interested in delivery of their mail. This creates a signal / noise ratio into the filters or ISPs that is so weighted to the noise side, there is almost no value to the filter or ISP in even having a channel for the small amount of signal.
The reality is that most senders do not spend a lot of time looking into a block before contacting the ISP. They use the ISP points of contact as a way to avoid doing hard work internally. This transfers lot more work onto the ISPs and makes them less conducive to working with any senders at all.
I also think there are slightly different obligations on commercial spamfiltering companies and ISPs in regards to listening to senders. Commercial spamfiltering companies are further removed from the end user than the ISPs are. In many cases the end user has no idea that the spamfiltering at their ISP has been outsourced to a commercial company and they have no internal resolution path. They can contact their ISP, but that is only useful if the ISP has an escalation path back to the filtering company. I think that this distance, and the fact that the spamfiltering companies are profiting directly from blocking mail, means that spamfiltering companies have more of a responsibility to be accessible to the people they are blocking. The irony is that the spamfiltering companies are generally less accessible to senders than ISPs are.
Overall I do not think that good spamfiltering happens in a vacuum, and that reliable reports from senders about inaccurate filtering help improve blocking schemes. Senders are not in a position to be making any demands of ISPs and filtering companies, however, I do believe that the end user experience would be better if there were more communication between senders and recipients. The problem is that the history of communication between the two groups has been contentious at best and there are only so many times the receivers are going to spend time listening to the senders, again.
I guess it boils down to no, senders do not have a right to demand explanations, but things might be better if more ISPs and spamfiltering companies engaged with non-spamming but blocked senders more often. Sorting out those non-spamming but blocked senders from legitimately blocked senders is the real trick and I expect if receivers could do that reliably, there would be no false positives.
RSS - Posts
No, spam-filterers don’t owe explanations of their methods to senders. However, they do have a duty to their own customers not to wrongly filter email that their customers do want to get.
The problem is that each recipient of a bulk email has little incentive to try to get a filtering decision reversed, but the sender of the bulk email has great incentive to do so. However, senders have that incentive regardless of whether the filtering decision was valid or not.
So, the problem for Filterers is that most of the people who try to get them to reverse their decisions will do so regardless of the validity of the decisions. So, they need a way to evaluate these requests so as to filter out the invalid ones, and then they need a way to act upon the valid ones.
As for whether such a remediation process is a “must,” I think the fact that many large ISPs (e.g., AOL, Hotmail) and other spam-filtering systems (e.g., Brightmail) have such processes, and have had them for years, provides sufficient evidence to establish that proposition’s truth.
Exactly, Tim.
Presumably, if a company is contracting with Postini to act as their agent in handling their e-mail delivery, they have the expectation that Postini will take reasonable care in delivering (or scoring) that e-mail.
If Postini willfully and systematically ignores information that they are failing to do so… I think there’s an argument that it’s negligence.
I’m explicitly *not* making the 1990s spammer argument that “Recipient ISPs have no right to filter”. My argument, then and now, was that recipient ISPs have the right to do whatever their customers want.
Rather, I’m stating that I don’t think Postini’s customers want Postini to do what Postini is actually doing. Who here has asked their ISP to knowingly drop legitimate, non-bulk e-mail on the floor and ignore complaints? Anyone?
Add to that:
* Postini is a division of Google
* Google is a publicly traded company
* Publicly traded companies must behave in compliance with many, many laws and regulations
* Postini explicitly markets themselves to law firms as a comprehensive e-mail solution
* Lawyers don’t like it when they can’t get e-mail from their clients
* Also, they sue a lot
* Postini’s behavior is arguably in conflict with Google’s published Code of Conduct
* Shareholders have a right to expect the company to enforce its Code
* Google’s Investor Relations department has failed to answer questions from at least one shareholder (Hi!) on this very matter
I think this could a make a lawyer rich someday. How many people are in the class of “everyone who has sent an e-mail to a Postini client”?
How do I permanently block all email from any and all postini addresses ?