Gmail filtering in a nutshell

Gmail’s approach to filtering; as described by one of the old timers. This person was dealing with network abuse back when I was still slinging DNA around as my job and just reading headers as a hobby.

Gmail uses a 10+ year old neural network that analyzes thousands of factors, related to email, IP, and web, integrated with all Google products, and with 99.9%+ accuracy for identifying certain types of messages, combined with an email-specific domain-based reputation system that combines IP reputation, content, read rates, reputation of other senders with similar content.

This excerpt was shared with a bunch of delivery experts and every one of them agreed. The Gmail filters are incredibly complex and they measure thousands of different things about email. Yes, sometimes you can remove a link or a URL and get mail to the inbox for a while. That doesn’t mean the block was against the URL, simply that changing the URL changed the score enough for the mail to go to the inbox.
This is part of what makes Gmail delivery issues so difficult to troubleshoot. There isn’t one thing, it’s all the things that contribute to where an email ends up. We, as senders and deliverability experts, don’t have access to the Gmail data. The poster goes on to say:

Trying to fix this using only inaccurate proxy data where there is no mediation pathway in a matter of weeks is complex.  We consume data from a multitude of sources, compile and analyze the data, determine which of the hundreds of factors we can influence should be adjusted, come up with the easiest plan to address the most influential factors, and explain that to the customer using the clearest language possible to individuals who are not educated on the definition of a complaint.

We do our best, with limited data and try and tell you how to fix things.
One of the biggest challenges with Gmail delivery is I am convinced they look at your profile of recipients. They can map someone who is collecting addresses through third parties, or buying lists based on the specific Gmail accounts targeted by a mailing. Gmail has publicly stated and has on their website that they don’t think co-reg or purchased lists are opt-in. They have the technology and ability to track that. I think it’s one reason senders trying to use email for acquisition have such a challenge getting into the inbox and Gmail. I think it’s a feature, not a bug for them.
 

Related Posts

Inbox challenges and dull email in the tabbed inbox

Getting to the inbox is becoming a greater and greater challenge for many marketers. According to Return Path, 22% of opt in mail doesn’t make it to the inbox.
The challenge to marketers is that a lot of opt in mail isn’t important to the recipient. Sure, they’re happy enough to get it if they notice it, but if it’s not there then they don’t care. They’ll buy from an email ad, but it might not be something they’ll seek out. Recipient behaviour tells the ISPs that the mail isn’t all that important, and a lot of it is just background noise so the ISP not delivering it to the inbox doesn’t matter.
Email marketing is like the Girl Scout of the Internet. If the Girl Scout shows up at your doorstep, you’re probably going to buy those 3 boxes of thin mints. But if she doesn’t, that’s OK. If you really want the cookies, you’ll find the co-worker who is taking orders for his daughter. Or you’ll find the table outside the local coffee shop. The Girl Scout showing up on your doorstep makes it more convenient, but she’s not critical to get your fix. Of course, the bonus of the Girl Scout on the doorstep is that a lot of people who won’t go find the cookies will buy when she’s on the doorstep.
A lot of email marketing triggers purchases that recipients would make anyway. They think they might want a particular product, and when they get that coupon or discount or even just a reminder they make the purchase. The email triggers the purchase of a product the buyer intends to purchase anyway. Some email marketing trigger purchases of things the recipient didn’t know existed, but is so enticing after one email they can’t live without. Some email marketing triggers an impulse purchase. In most of these categories, if mail doesn’t show up in the inbox, the recipient really doesn’t miss it.
Many marketers, despite loud protests that all their mail is important and wanted, know this. That’s why so many marketers are having conniptions about the new Gmail tabbed inbox. They’re losing access to the impulse.
From the data I’ve seen, tabs are effecting email marketing programs. Some programs are seeing more revenue, some are seeing less. I think it really remains to be seen what the long term effects are. For many recipients the new tabbed inbox is a new way to interact with their email. Change is hard, and there is a period of adaptation whenever an interface changes. We really don’t know what the long term effect of tabs on sales will be. Sales may go back to previous levels, sales may increase over previous levels, sales may decrease from current levels or sales may stay at their current levels. The full effect isn’t going to be obvious for a while.
It does mean, though, that email marketers need to step up their game. Email marketing in the age of a tabbed inbox might be less about the impulse purchase and more about cultivation and long term branding.
 
 
 

Read More

IP reputation and email delivery

IP reputation is a measure of how much wanted mail a particular IP address sends.  This wanted mail is measured as a portion of the total email sent from that IP. Initially IP reputation was really the be all and end all of reputation, there was no real good way to authenticate a domain or a from address. Many ISPs built complex IP reputation models to evaluate mail based on the IP that sent the mail.
These IP reputation models were the best we had, but there were a lot of ways for spammers to game the system. Some spammers would create lots of accounts at ISPs and use them to open and interact with mail. Other spammers would trickle their mail out over hundreds or thousands of IPs in the hopes of diluting the badness enough to get to the inbox. Through it all they kept trying to get mail out through reputable ESPs, either by posing as legitimate customers or compromising servers.
These things worked for a while, but the ISPs started looking harder at the recipient pool in order to figure out if the interactions were real or not. They started looking at the total amount of identical mail coming from multiple IP addresses. The ISPs couldn’t rely on IP reputation so they started to dig down and get into content based filtering.
As the ISPs got better at identifying content and filtering on factors other than source IP, the importance of the IP address on inbox delivery changed. No longer was it good enough to have a high reputation IP sending mail.
These days your IP reputation dictates how fast you can send mail to a particular ISP. But a high reputation IP isn’t sufficient to get all the mail in the inbox. It’s really content that drives the inbox / bulk folder decisions these days.
 
Generally IPs that the ISP has not seen email traffic from before start out with a slight negative reputation. This is because most new IPs are actually infected machines. The negative reputation translates to rate limiting. The rate limiting minimizes people getting spam while the ISP works out if this is a real sender or a spammer.
Some ISPs put mail in the inbox and bulk foldering during the whitelisting process. In this case what they’re doing is seeing if your recipients care enough about your mail to look for it in the bulk folder. If they do, and they mark the mail as “not spam” then this feeds back to the sender reputation and the IP reputation.
If you’re seeing a lot of bulk foldering of mail, it’s unlikely there’s anything IP reputation based to do. Instead of worrying about IP reputation, focus instead on the content of the mail and see what you may need to do to improve the reputation of the domains and URLs (or landing pages) in the emails.

Read More

Email marketing OF THE FUTURE!

ISPs are continually developing tools for their users. Some of the newer tools are automatic filters that help users organize the volumes of mail they’re getting. Gmail released Priority Inbox over a year ago. Hotmail announced new filters as part of Wave 5 back in October.
All of these announcements cause much consternation in the email marketing industry. Just today there was a long discussion on the Only Influencers list about the new Hotmail filtering. There was even some discussion about why the ISPs were doing this.
I think it’s pretty simple why they’re creating new tools: users are asking for them. The core of these new filters is ISPs reacting to consumer demand. They wouldn’t put the energy into development if their users didn’t want it. And many users do and will use priority inbox or the new Hotmail filtering.
Some people are concerned that marketing email will be less effective if mail is not in the inbox.

Read More