Engagement filters for B2B mail

While I was doing some research for a client today I rediscovered Terry Zink’s blog. Terry is one of the MS email folks and he regularly blogs about the things MS is doing with Outlook.com and Office 365.
The post that caught my eye was discussing the Microsoft Spam Fighter program. The short version is that in order to train their spam filters, Microsoft asks a random cross-section of their users if the filters made the right decision about email. This data is fed back into the Microsoft machine learning engine.
As Terry explains it:

These votes from all the users across the entire Spam Fighters program are combined, and the messages combined to create a corpus, and then Smartscreen learns across numerous features within a message – sending IP, sending domains, authentication status, headers, body of message, attachments, encodings, and so forth. This feeds into our IP reputation, and into the Smartscreen spam filtering algorithm. This algorithm is what does the filtering for spam, malware, and phishing as well as legitimate email. It’s updated multiple times per day.

The SmartScreen filter is a source of pain for many senders. But, Microsoft checks it’s accuracy on an ongoing basis. When Microsoft says that SmartScreen data tells them this mail is unwanted, they are getting that information directly from the subscribers of the email. If the subscribers don’t want mail, it’s nearly impossible to get ISPs to deliver that mail.
Engagement based filtering is standard in the consumer space. The primary mailbox providers focus on providing their users with the mail that they want, while protecting them from malicious mail. Things are different in the business space as most business filters don’t care if the user engages with the mail or not. For businesses email is a tool and sometimes we don’t like our tools.
However, as Microsoft merges the backend for Hotmail/Outlook and Office365 engagement filtering may become more relevant at those domains hosted on Office 365. I don’t expect those filters to be identical – again these are different user bases with different priorities. But Smart Screen filters may start acting on business email in the future.

Related Posts

IPv6 and authentication

I just saw a post over on the mailop mailing list where someone had been bitten by some of the IPv6 email issues I discussed a couple of months ago.
They have dual-stack smarthosts – meaning that their smarthosts have both IPv4 and IPv6 addresses, and will choose one or the other to send mail over. Some domains they send to use Office 365 and opted-in to receiving mail over IPv6, so their smarthosts decided to send that mail preferentially over IPv6.
The mail wasn’t authenticated, so it started bouncing. This is probably going to happen more and more over the next year or so as domain owners increasingly accept mail over IPv6.
If your smarthosts are dual stack, make sure that your workflow authenticates all the mail you send to avoid this sort of delivery issue.
One mistake I’ve seen several companies make is to have solid SPF authentication for all the domains they send – but not for their IPv6 address space. Check that all your SPF records include your IPv6 ranges. While you’re doing that keep in mind that having too many DNS records for SPF can cause problems, and try not too bloat the SPF records you have your customers include.

Read More

Another way Gmail is different

I was answering a question on Mailop earlier today and had one of those moments of clarity. I finally managed to articulate one of the things I’ve known about Gmail, but never been able to explain. See, Gmail has never really put a lot of their filtering on the SMTP transaction and IP reputation. Other ISPs do a lot of the heavy lifting with IP filters. But not Gmail.
While I was writing the answer I realized something. Gmail was a late entrant into the email space. AOL, Hotmail, Yahoo, even the cable companies, were providing email services in the 90s. When spam started to be a problem, they started with IP based blocking. As technology got better and content filtering became viable, improvements were layered on top of IP based blocking.

Gmail didn’t enter the mailbox market until the 2000’s. When they did, they had money, lots of hardware, and internal expertise to do content filtering. They didn’t start with IP based filtering, so their base is actually content filtering. Sure, there were some times when they’d push some mail away from the MTAs, but most of their filtering was done after the SMTP transaction. The short version of this is I never really pay any attention to IP reputation when dealing with Gmail. It’s just another factor. Unless you’re blocked and if you get blocked by Gmail, wow, you really screwed up.
Gmail does, of course, do some IP based blocking. But in my experience IP filters are really only turned against really egregious spam, phishing and malicious mail. Most email marketers reading my blog won’t ever see IP filters at Gmail because their mail is not that bad.
Other companies aren’t going to throw away filters that are working, so the base of their filters are IPs. But Google never had that base to work from. Their base is content filters, with some IP rep layered on top of that.
That’s a big reason Gmail filters are different from other filters.

Read More

Catching up from MAAWG SF

Had a great time a M3AAWG last week. So many familiar faces and a lot of new ones, too. I’ve got a lot of interesting stuff that I can share with readers over the next few days.
One of the things I have received permission to share is the new Office 365 IP delisting link. I botched the first time I posted it, so I’m going to try again. Office 365 IP Delisting Page. Many thanks to the Microsoft guys for getting this together for people.
While I’m talking about Microsoft, there is a bit of a problem with folks signing up with their FBL. Some people are finding that the process gets stuck and FBLs aren’t enabled. MS is aware of the issue and they are working on fixing it. As I know more I’ll share.
Unsurprisingly, authentication was a big topic of conversation, both in the hallways and in the sessions. There were some strong opinions stated. I think, though, that we’re pretty clear that we’re going to get to a more authenticated world. But we have some different opinions on how and how fast that’s going to happen.

Read More