Sender complaints about spamfiltering

JD posed a question in my post about Postini and trying to sort out a customer getting marked as spam by their filtering mechanism and I think it bears more discussion than can be done in comments.

And sure, it’s a best practice for filtering companies to respond politely to requests from filterees. But is it a requirement? Do senders have a right to demand explanations?

There is not really an easy answer for that. My first response is “of course not!” but then I think about some of my clients who really have been trying to do the right thing and how we work through issue after issue and finally fix everything I can think of, but they still have delivery problems. These are not spammers, they are sending mail to people who have asked for it and by all measures do actually want it, but some mail is being blocked for reasons neither my client or I can figure out. In those cases it would be really nice if someone from the group doing the blocking would take 10 minutes to point me in the right direction and show me what I missed.
I have been doing this long enough to know that spamfilters are not 100% accurate. I know there are times when a specific block is outside the scope of what email the filter designer, or user, expected to block. Look at what happened when Yahoo started using the PBL a few months back. There was a bug in the implementation that neither Yahoo nor Spamhaus expected and that caused mail from IPs not listed on the PBL to be blocked because of the PBL. With a valid report of the problem, I could contact both Spamhaus volunteers and someone at Yahoo to point out there was a problem with the implementation. Yahoo and Spamhaus figured out the issue and fixed the problem and Yahoo is no longer blocking IPs not on the PBL for being on the PBL.
I do believe that there are times when feedback from senders and blockees is beneficial and can help improve the overall filters. I have clear evidence this is the case.
On the flip side, I also have been in the email business long enough to know that more than 99% of senders just want their mail delivered and do not care about anything other than getting into the inbox. They believe every block is a mistake and the ISP / spamfilter is wrong or broken. They are not interested in actually making sure the implementation of the filter meets the design goals, usually they do not care what the goals of that filter are. They are just interested in delivery of their mail. This creates a signal / noise ratio into the filters or ISPs that is so weighted to the noise side, there is almost no value to the filter or ISP in even having a channel for the small amount of signal.
The reality is that most senders do not spend a lot of time looking into a block before contacting the ISP. They use the ISP points of contact as a way to avoid doing hard work internally. This transfers lot more work onto the ISPs and makes them less conducive to working with any senders at all.
I also think there are slightly different obligations on commercial spamfiltering companies and ISPs in regards to listening to senders. Commercial spamfiltering companies are further removed from the end user than the ISPs are. In many cases the end user has no idea that the spamfiltering at their ISP has been outsourced to a commercial company and they have no internal resolution path. They can contact their ISP, but that is only useful if the ISP has an escalation path back to the filtering company. I think that this distance, and the fact that the spamfiltering companies are profiting directly from blocking mail, means that spamfiltering companies have more of a responsibility to be accessible to the people they are blocking. The irony is that the spamfiltering companies are generally less accessible to senders than ISPs are.
Overall I do not think that good spamfiltering happens in a vacuum, and that reliable reports from senders about inaccurate filtering help improve blocking schemes. Senders are not in a position to be making any demands of ISPs and filtering companies, however, I do believe that the end user experience would be better if there were more communication between senders and recipients. The problem is that the history of communication between the two groups has been contentious at best and there are only so many times the receivers are going to spend time listening to the senders, again.
I guess it boils down to no, senders do not have a right to demand explanations, but things might be better if more ISPs and spamfiltering companies engaged with non-spamming but blocked senders more often. Sorting out those non-spamming but blocked senders from legitimately blocked senders is the real trick and I expect if receivers could do that reliably, there would be no false positives.

Related Posts

Articles I read today

It has been a rather busy day today, I do not have a full blog post. I did see a couple posts come across my RSS feeds. Both of them have content I want to talk about and discuss in a little more detail, as I think they touched on some very interesting issues.
Network World has an article interviewing Mark Risher from Yahoo. The article discusses Yahoo’s use of DomainKeys as part of their inbound mail filtering.
Mickey has an article about how to deal with ISPs when attempting to troubleshoot a blocking issue.
More details and commentary on both articles later this week.

Read More

SenderScore update

Matt has posted a bit more about the SenderScore Blacklist, following up on my post about the changes at Comcast. George Bilbrey, VP and General Manager, for Return Path followed up with him to explain a bit more about the blacklist. George says:

Read More

Yahoo and Spamhaus

Yahoo has updated and modified their postmaster pages. They have also put a lot of work into clarifying their response codes. The changes should help senders identify and troubleshoot problems without relying on individual help from Yahoo.
There is one major change that deserves its own discussion. Yahoo is now using the SBL, XBL and PBL to block connections from listed IP addresses. These are public blocklists run by Spamhaus. Each of them targets a different type of spam source.
The SBL is the blocklist that addresses fixed spam sources. To get listed on the SBL, a sender is sending email to people who have never requested it. Typically, this involves email sent to an address that has not opted in to the email. These addresses, known as spamtraps, are used as sentinel addresses. Any mail sent to them is, by definition, not opt-in. These addresses are never signed up to any email address lists by the person who owns the email address. Spamtraps can get onto a mailing list in a number of different ways, but none of them involve the owner of the address giving the sender permission to email them.
Additionally, the SBL will list spam gangs and spam supporters. Spam supporters include networks that provide services to spammers and do not take prompt action to remove the spammers from their services.
The XBL is a list of IP addresses which appear to be infected with trojans or spamware or can be used by hackers to send spam (open proxies or open relays). This list includes both the CBL and the NJABL open proxy list. The CBL list machines which appear to be infected with spamware or trojans. The CBL works passively, looking only at those machines which actively make connections to CBL detectors. NJABL lists machines that are open proxies and open relays.
The Policy Block List (PBL) is Spamhaus’ newest list. Spamhaus describes this list as

Read More