When spam filters fail

Spam filters aren’t perfect. They sometimes catch mail they shouldn’t, although it happens less than some people think. They sometimes fail to catch mail they should.
One of the reason filters fail to catch mail they should is because some spammers invest a lot of time and energy in figuring out how to get past the filters. This is nothing new, 8 or 9 years ago I was in negotiations with a potential client. They told me they had people who started working at 5pm eastern. Their entire job was to craft mail that would get through Hotmail’s filters that day. As soon as they found a particular message that made it to the inbox, they’d blast to their list until the filters caught up. When the filters caught up, they’d start testing again. This went on all night or until the full list was sent.
Since then I’ve heard of a lot of other filter bypass techniques. Some spammers set up thousands of probe accounts at ISPs and would go through and “not spam” their mail to fool the filters (ISPs adapted). Some spammers set up thousands of IPs and rotate through them (ISPs adapted). Some spammers register new domains for every send (ISPs adapted). Some spammers used botnets (ISPs adapted)
I’m sure, even now, there are spammers who are creating new techniques to get through filters. And the ISPs will adapt.

Related Posts

ISP filters are good for marketers

A throwback post from 2010 Attention is a limited resource.
Marketing is all about grabbing attention. You can’t run a successful marketing program without first grabbing attention. But attention is a limited resource. There are only so many things a person can remember, focus on or interact with at any one time.
In many marketing channels there is an outside limit on the amount of attention a marketer can grab. There are only so many minutes available for marketing in a TV or radio hour and they cost real dollars. There’s only so much page space available for press. Billboards cost real money and you can’t just put a billboard up anywhere. With email marketing, there are no such costs and thus a recipient can be trivially and easily overwhelmed by marketers trying to grab their attention.
Whether its unsolicited email or just sending overly frequent solicited email, an overly full mailbox overwhelms the recipient. When this happens, they’ll start blocking mail, or hitting “this is spam” or just abandoning that email address. Faced with an overflowing inbox recipients may take drastic action in order to focus on the stuff that is really important to them.
This is a reality that many marketers don’t get. They think that they can assume that if a person purchases from their company that person wants communication from that company.

Read More

Delivering to Gmail

Gmail is a challenge for even the best senders these days.
With the recent Gmail changes there isn’t any clear fix to getting open rates or inbox delivery back up. Some of it depends on what is causing Gmail to filter the mail. Changing subject lines, from name, from address may get mail back to the inbox in the short term, but it only works until the filters catch up.
What I am seeing, across a number of clients, is that Gmail is doing a lot of content reputation and that content reputation gets spread across senders of that content.  That means you want to look at who is sending any mail on your behalf (mentioning your domain or pointing at your website) and their practices. If they have poor practices, then it can reflect badly on you and result in filtering.
From what I’ve seen, these are very deliberate filtering decisions by Google. And it’s making mail a lot harder for many, many senders. But I think it is, unfortunately, the new reality.

Read More

Email filters

What makes the best email filter? There isn’t really a single answer to that question. Different people and different organizations have different tolerances for how false positives versus false negatives. For instance, we’re quite sensitive to false positives here, so we run extremely conservative filtering and don’t block very much at the MTA level. Other people I know are very sensitive to false negatives and run more aggressive filtering and block quite a bit of mail at the MTA level.
For the major ISPs, the people who plan, approve, design and monitor the filters usually want to maximize customer happiness. They want to deliver as much real mail as possible while blocking as much bad mail. Blocking real mail and letting through bad mail both result in unhappy customers and increase the ISP’s costs, either through customer churn or through support calls. And this is a process, filters are not static. ISPs roll out new filters all the time, sometimes they are an improvement and sometimes they’re not. When they’re not, they’re pulled out of production. This works both for positive filters like Return Path and negative filters like blocklists.
Then there is mail filtering that doesn’t have to do with spam. Business filters, for instance, often block non-business mail. Permission of the recipient often isn’t even a factor. Companies don’t often go out of their way to block personal mail, but if personal mail gets blocked (say the vacation plane ticket or the amazon receipt) they don’t often unblock it. But when you think about why a business provides email, it makes perfect sense. The business provides email to further its own business goals. Some personal usage is usually OK, but if someone notices and blocks personal email then it’s unlikely the business will unblock it, even if the employee opted in.
In the case of email filters, the free market does work. Different ISPs filter mail differently. Some people love Gmail’s filters. Other people think Hotmail has the best filtering. There are different standards for filtering, and that makes email stronger and more robust. Consumers have choices in their mail provider and spamfiltering.

Read More