Did the algorithm change?

When faced with unexplained deliverability changes one of the first questions many folks ask is “Did the algorithm change.” In many ways this is an meaningless question. Why? Because there are two obvious answers to the question.
A1: Of course it didn’t.
A2: Of course it did.
Both answers are correct, but they’re answering different underlying questions. When we understand how two diametrically opposed answers are both correct, we understand much more about filtering.

What is an algorithm?

When we’re talking about spam filtering the algorithm is the process or rules to follow.

Basically, an algorithm is a computer program that is set up to filter spam to the bulk folder and filter wanted mail into the inbox.  This algorithm doesn’t change. It can’t.
In the case of many (most?) spam filters, the filters incorporate features of machine learning. 
This means the algorithm is constantly changing, learning more and more about what is spam and what is wanted mail based on user interactions.

Algorithms don’t change

Overall, the algorithms don’t change that frequently. They are fed data (lots of data) on a continual basis. They take feedback from recipients (spam / not spam buttons) and developers (new data sets of known bad and known good mail) to learn what good mail looks like and what bad mail looks like. But the underlying code doesn’t change very frequently.

Results change

Machine learning algorithms are only as good as the data they’re fed. In the case of spam filters, the input data is constantly changing. So the output results change. Sometimes an email that was not-spam one day is spam the next because the algorithm caught up with a new threat or new behavior. 

Delivery is still in your control

This machine learning and reliance on end users to help tune filters may make it seem like spam filtering is completely out of the senders’ control. That there is nothing a sender can do to get into or out of the bulk folder. The good news is, the underlying algorithms are pretty simple: wanted mail goes to the inbox, unwanted mail goes to the bulk folder. As with everything, details matter. Senders who are focused on recipients usually don’t have a difficult time reaching the inbox. Companies that focus on themselves and try gimmicks find it much harder to consistently reach the inbox.
 

More on AOL transition to Oath Infrastructure
Following CAN SPAM isn't enough to reach the inbox

Related Posts

Doing email right

Over on the MarketingLand website, Len Shneyder talks about 3 companies (Uber, REI and eBay) that do email right. In there he shows how the companies use email to further their business goals while understanding and meeting the needs of their customers.
Meeting the needs of recipients is the way to get your mail to the inbox. Send email that your users want, and they will tell the ISPs when they don’t get your mail. It’s sometimes hard to convince senders of this. Instead they want to tweak URLs or authentication or IPs or domains. But none of those things are what deliverability is all about. Deliverability is about the recipient. Deliverability is about the relationship between the sender and recipient.
Send to the right people – and the right people are those who have asked for and want your mail – and deliverability problems don’t materialize. Sure, every once in a while something might happen that throws mail into the bulk folder for one reason or another. But fighting to get to the inbox isn’t an every day thing. Instead, senders can focus on knowing their users and sending mail that makes them happy when it shows up in the inbox.
 

Read More

It's not fair

In the delivery space, stuff comes in cycles. We’re currently in a cycle where people are unhappy with spam filters. There are two reasons they’re unhappy: false positives and false negatives.
False positives are emails that the user doesn’t think is spam but goes into the bulk folder anyway.
Fales negatives are emails that the user does thing is spam but is delivered to the inbox.
I’ve sat on multiple calls over the course of my career, with clients and potential clients, where the question I cannot answer comes up. “Why do I still get spam?”
I have a lot of thoughts about this question and what it means for a discussion, how it should be answered and what the next steps are. But it’s important to understand that I, and most of my deliverability colleagues, hate this question. Yet we get it all the time. ISPs get it, too.
A big part of the answer is because spammers spend inordinate amounts of time and money trying to figure out how to break filters. In fact, back in 2006 the FTC fined a company almost a million dollars for using deceptive techniques to try and get into filters. One of the things this company did would be to have folks manually create emails to test filters. Once they found a piece of text that would get into the inbox, they’d spam until the filters caught up. Then, they’d start testing content again to see what would get past the filters. Repeat.
This wasn’t some fly by night company. They had beautiful offices in San Francisco with conference rooms overlooking Treasure Island. They were profitable. They were spammers. Of course, not long after the FTC fined them, they filed bankruptcy and disappeared.
Other spammers create and cultivate vast networks of IP addresses and domains to be used in snowshoeing operations. Still other spammers create criminal acts to hijack reputation of legitimate senders to make it to the inbox.
Why do you still get spam? That’s a bit like asking why people speed or run red lights. You still get spam because spammers invest a lot of money and time into sending you spam. They’re OK with only a small percentage of emails getting through filters, they’ll just make it up in volume.
Spam still exists because spammers still exist.
 

Read More

Data Cleansing part 2

In an effort to get a blog post out yesterday before yet another doctor’s appointment I did not do nearly enough research on the company I mentioned selling list cleansing data. As Al correctly pointed out in the comments they are currently listed on the SBL. And when I actually did the research I should have done it was clear this company has a long term history of sending unsolicited email.
Poor research and a quickly written blog post led to me endorsing a company that I absolutely shouldn’t have. And I do apologize for that.
With all that being said, Justin had a great question in the comments of yesterday’s post about data cleansing.

Read More