Gmail

Google, Alignment and DMARC

Google has been making a number of changes to their systems over the last few weeks. Folks are seeing a lot of changes in Google postmaster tools and they’re seeing changes in how Google is displaying headers in the “show original” tab.

Don’t trust Gmail’s Show Original

It’s not always easy to know what the actual headers and body of an email as sent look like. For a long time accepted wisdom was that you could send a copy to your gmail account, and use the Show Original menu option to, well, see the original message as raw text.

New Requirements for Bulk Senders

UPDATE: You need to authenticate with both DKIM and SPF.

Is email dead?

These last few years have been something, huh? Something had to give and, in my case, that something was blogging. There were a number of reasons I stopped writing here, many of them personal, some of them more global. I will admit, I was (and still am a little) burned out as it seemed I was saying and writing the same things I’d been saying and writing for more than a decade. Taking time off has helped a little bit, as much to focus on what I really want to talk about.

Step by Step guide to fixing Gmail delivery

I regularly see folks asking how to fix their Gmail delivery. This is a perennial question (see my 2019 post and the discussions from various industry experts in the comments). Since that discussion I haven’t seen as much complaining about problems.

Back at it

Back at the office after traveling to visit a bunch of our US friends recently. A lot of news, both in and out of the email space, happened while we were gone. The biggest stories are outside the email space and I will admit to following the coronavirus news probably closer than I should. (My graduate work was done across the hall from one of the major avian epidemic monitoring labs. This is the kind of thing we discussed at lunch and over beers.)

When opens hurt reputation

Podia has scraped the Word to the Wise blog and I’m currently receiving an ongoing drip campaign from them absolutely begging me to mention them in my blog post on cold emails.

Same MX, different filters

One of the things I do for clients is look at who is really handling mail for their subscribers. Steve’s written a nifty tool that does a MX lookup for a list of domains. Then I have a SQL script that takes the raw MX lookup and categorizes not by the domain or even the MX, but by the underlying mail filter.

Google IP reputation bad

This morning hundreds of delivery folks logged into their Google Postmaster Tools account to see their IP reputation was bad.

Tulsi Gabbard Sues Google

Today Tulsi Gabbard’s campaign sued Google for $50 million. Why? Because during the night of the first debate Google disabled her “advertising account” (I’m assuming she means adwords) preventing her from being able to purchase ads to direct searchers to her website. There’s also a paragraph in there that they’re “disproportionally putting her email into the spam folder.”

Google Postmaster is Back

Late last night folks started mentioning they were seeing data trickled into Google Postmaster tools. This morning, some of the domains for some of my clients are showing data.

More Google issues

Not necessarily more but more information about the current Google Postmaster Tools (GPT) outage. I’ve been reliably informed by folks inside Google that they’re aware of the outage and are working on it.

Google problems

It’s been a bit of a problematic week for Google. In the last few days they’ve had a number of outages or problems across different services. There was a major outage of Google Calendar. All email, including some spam, was delivering to the primary tab instead of the correct tab. Additionally, Google postmaster tools hasn’t been updated in over a week.

Google Suspicious Link Warnings

A number of folks in the sender space are reporting intermittent “This link may be suspicious” warnings on their emails. I first heard about it a few weeks ago from some clients. One wasn’t sure what was going on, the other found a bunch of malware uploaded into their customer accounts.

What’s up with gmail?

Increasingly over the last few months I’ve been seeing questions from folks struggling with reputation at Gmail and inbox delivery. It seems like everything exploded in the beginning for 2019 and everything changed. I’ve been avoiding blaming it all on TensorFlow, but maybe the addition of the new ML engine really did fundamentally change how things were working at gmail.

Gmail suddenly puts mail in the bulk folder

One of the delivery challenges that regularly comes up in various delivery discussion spaces is the “Gmail suddenly put my mail in spam.” From my perspective, there is rarely a “suddenly” about Gmail’s decision making process.

TLS and Gmail delivery

I’m seeing some questions about TLS and Gmail. Folks are seeing a correlation between sending without TLS and the mail going to bulk.

My domain reputation is bad, should I get a new domain?

Many companies have the occasional “oops” where they send email they probably shouldn’t have. This can often cause a decrease in reputation and subsequent delivery problems. Some companies rush to fix things by changing domains.

Marking mail as spam says what?

I wear a number of hats and have a lot of different email addresses. I like to keep the different email addresses separate from each other, “don’t cross the streams” as it were.

Fun with spam filters

I recently had a challenging travel experience in the Netherlands, trying to get from Schipol airport to a conference I was speaking at. As part of my attempt to get out of the airport, I installed UBER on my phone. There were some challenges with getting UBER to authorise my phone number, so I tried linking it to my Gmail account.

Can I get access to Google Postmaster tools if I’m using an ESP?

The answer is almost certainly yes, but there are definitely cases where it the answer is no.

Dodgy PDF handling at Gmail

We sent out some W-9s this week. For non-Americans and those lucky enough not to have to deal with IRS paperwork those are tax forms.
They’re simple single page forms with the company name, address and tax ID numbers on them. Because this is the 21st Century we don’t fill them in with typewriters and snail mail them out, we fill in a form online at the IRS website which gives us PDFs to download that we then send out via email.

We started to get replies from people we’d sent them to that we hadn’t included the tax ID number. Which was odd, because it was definitely there in the PDFs we’d sent.
The reports of missing numbers came from Google Apps users, so we sent a copy to one of our Gmail addresses to see. Sure enough, when you click on the attachment it’s mostly there, but some of the digits of the tax ID number are missing.

And all the spaces have been stripped from our address.

The rest of the form looked fine, but the information we’d entered was scrambled. Downloading the PDF from Gmail and displaying it – everything is there, and in the right place.
Weird. After a brief “Are gmail hiding things that look like social security numbers?” detour I realized that the IRS website was probably generating the customized forms using PDF annotations.
PDF is a very powerful, but very complex, file format. It’s not just an image, it’s a combination of different elements – images, lines, vector artwork, text, interactive forms, all sorts of things – bundled together into a single file. And you can add elements to an existing PDF file to, for example, overlay text on to it. These “annotations” are a common way to fill in a PDF form, by adding text in the right place over the top of an existing template PDF.
I cracked the PDF open with some forensics tools and sure enough, the IRS had generated the PDF form using annotations.

Affiliate marketing overview

Most retailers have realized that sending unsolicited email is bad for their overall deliverability. Still, the idea they can send mail to people who never heard of them is seductive.
Enter affiliate email. That magical place where companies hire an agency, or a contractor, or some other third party to send email advertising their new product. Their mail and company reputation is protected because they aren’t sending the messages. Even better, affiliates assure their customers that the mail is opt-in. I’m sure some of them even believe it.
The reality is a little different from what affiliates and their customers want to believe.

SNDS issues and new Gmail

A bunch of folks reported problems with Microsoft’s SNDS page earlier today. This afternoon, our friendly Microsoft rep told the mailop mailing list that it should be fixed. If you see problems again, you can report it to mailop or your ESP and the message will get shared to the folks who can fix it.
The other big thing that happened today was Gmail rolled out their new inbox layout.
It’s… nice. I’ll be honest, I am not a big gmail user and have never been a huge fan. I got my first account way-back-during-the-beta. I used it to handle some of my mailing list mail. I could never work out how to get it to stop breaking threads by deciding to put some mail into the junk folder. I just gave up and went back to my shell with procmail (now sieve) scripts. I still have a couple lists routed to my gmail account, and the filtering is much improved – I can at least tell it to never bulk folder certain email.
The feature I’m really interested in is the confidential, expiring email. I’m interested in how that’s going to work with non-Gmail accounts. Within Gmail makes perfect sense, but I don’t think Gmail can control mail once it’s off their system.

My best guess is that Gmail will end up sending some type of secure link to recipients using non-Gmail mail servers. The message itself will stay inside Google and recipients will only be able to view mail through the web. That’s how the vast majority of secure mail systems work.
If anyone has the secure message already, feel free to send me a secure message. I’ll report back as to how it works.

What kind of mail do filters target?

All to often we think of filters as a linear scale. There’s blocking on one end, and there’s an inbox on the other. Every email falls somewhere on that line.
Makes sense, right? Bad mail is blocked, good mail goes to the inbox. The bulk folder exists for mail that’s not bad enough to block, but isn’t good enough to go to the inbox.
Once we get to that model, we can think of filters as just different tolerances for what is bad and good. Using the same model, we can see aggressive filters block more mail and send more mail to bulk, while letting less into the inbox. There are also permissive filters that block very little mail and send most mail to the inbox.
That’s a somewhat useful model, but it doesn’t really capture the full complexity of filters. There isn’t just good mail and bad mail. Mail isn’t simply solicited or unsolicited. Filters take into account any number of factors before deciding what to do with mail.

Change is coming…

A lot of email providers are rolling out changes to their systems. Some of these changes are so they will comply with GDPR. But, in other cases, the changes appear coincidental with GDPR coming into effect.
It seems, finally, some attention is being paid to the mail client. Over the last few years the webmail providers have tried to upgrade their interface. Many of the upgrades are about managing high volumes of email in a more efficient manner. Google uses tabs while Microsoft has sweep and focused inbox.
It’s about time the mail client got an overhaul. My Apple mail client doesn’t look all that different from the desktop client I was using back in the late 90s on OS/2 Warp back in the late 90s. In some ways the OS/2 client was actually more functional. And, well, I do miss a lot of the flexibility of mutt in the shell.
Today, Google announced to Google Suite administrators that they would be rolling out a major client overhaul. G Suite admins who want to can join the early adopter program in the coming week. Techcrunch has a sketch of what the new mailbox layout looks like, done by someone who says they saw a Google engineer working on a train.
What’s interesting about the sketch is it seems tabs are going away. Given how many senders hate tabs I’m sure this is a welcome relief. We’ll see, though, if there’s not more inbox management built into the new client or not. The nifty new features are “snooze” – hide this email for some period of time and bring it back at some point in the future. The other big thing is calendar access right from the mail client.
I expect, too, that as OATH: brings the Yahoo and AOL mailboxes under one banner, there will also be some changes there. All of this amounts to more uncertainty in the email delivery space. But we’ll get through, we always do.

How long does it take to change reputation at Gmail?

Today I was chatting with a potential client who is in the middle of a frustrating warmup at Gmail. They’re doing absolutely the right things, it’s just taking longer than anyone wants. That’s kinda how it is with Gmail, while their algorithm can adapt quickly to changes. Sometimes, like when you’re warming up or trying to change a bad reputation, it can take 3 – 4 weeks to see any direct progress.This is a screenshot of IP reputation on Google Postmaster Tools. The sender made some significant changes in mail sending on some of their IP addresses starting in mid to late December. You can see, that the tools noticed and the reputation of those IPs bad to good fairly rapidly. It took a few more weeks of consistent sending for those two IPs to switch to yellow. And it took around another month for the reputation to flip to high.
Because this company is doing all the right things, and they’re seeing (as they describe it) some small amounts of improvement, I told them to give it another couple weeks. If they weren’t happy with their progress I could help them. But, frankly, until we can tell if this is something other than a normal warmup there isn’t much else to do.
When I got off the phone I felt very much like a doctor telling a patient to take two aspirin and call me in the morning. But, honestly, sometimes that is the right answer. Give it time.

Google makes connections

One of the client projects I’m working on includes doing a lot of research on MXs, including some classification work. Part of the work involves identifying the company running the MX. Many of the times this is obvious; mail.protection.outlook.com is office365, for instance.

There are other cases where the connection between the MX and the host company is not as obvious. That’s where google comes into play. Take the domain canit.ca, it’s a MX for quite a few domains in this data set. Step one is to visit the website, but there’s no website there. Step 2 is drop the domain into google, who tells me it’s Roaring Penguin software.
In some cases, though, the domain wasn’t as obvious as the Roaring Penguin link. In those cases, Google would present me with seemingly irrelevant hosting pages. It didn’t make sense until I started digging through hosting documentation. Inevitably, whenever Google gave me results that didn’t make sense, they were right. The links were often buried in knowledge base pages telling users how to configure their setup and mentioning the domain I was searching for.
The interesting piece was that often it was the top level domain, not the support pages, that Google presented to me. I had to go find the actual pages. Based on that bit of research, it appears that Google has a comprehensive map of what domains are related to each other.
This is something we see in their handling of email as well. Gmail regularly makes connections between domains that senders don’t expect. I’ve been speaking for a while about how Gmail does this, based on observation of filtering behavior. Working through multiple searches looking at domain names was the first time I saw evidence of the connections I suspected. Gmail is able to connect seemingly disparate hostnames and relate them to one another.
For senders, it means that using different domains in an attempt to isolate different mainstreams doesn’t work. Gmail understands that domainA in acquisition mail is also the same as domainB in opt-in mail is the same as domainC in transactional mail. Companies can develop a reputation at Google which affects all email, not just a particular mail stream. This makes it harder for senders to compartmentalize their sends and requires compliance throughout the organization.
Acquisition programs do hurt all mail programs, at least at Gmail.

Gmail survey rough analysis

I closed the Google Postmaster Tools (GPT) survey earlier today. I received 160 responses, mostly from the link published here on the blog and in the M³AAWG Senders group.
I’ll be putting a full analysis together over the next couple weeks, but thought I’d give everyone a quick preview / data dump based on the analysis and graphs SurveyMonkey makes available in their analysis.
Of 160 respondents, 154 are currently using GPT. Some of the folks who said they didn’t have a GPT account also said they logged into it at least once a day, so clearly I have some data cleanup to do.
57% of respondents monitored customer domains. 79% monitored their own domains.
45% of respondents logged in at least once a day to check. Around 40% of respondents check IP and/or domain reputation daily. Around 25% of respondents use the authentication, encryption and delivery errors pages for troubleshooting.
10% said the pages were very easy to understand. 46% said they’re “somewhat easy” to understand.
The improvements suggestions are text based, but SurveyMonkey helpfully puts them together into a word cloud. It’s about what I expected. But I’ll dig into that data.
10% of respondents said they had built tools to scrape the page. 50% said they hadn’t but would like to.
In terms of the problems they have with the 82% of people said they want to be able to create alerts, 60% said they want to add the data to dashboards or reporting tools.

97% of respondents who currently have a Google Postmater Tools account said they are interested in an API for the data. I’m sure the 4 who aren’t interested won’t care if there is one.
47% of respondents said if there was an API they’d have tools using it by the end of 2017. 73% said they’d have tools built by end of Q1 2018.
33% of respondents send more than 10 million emails per day.
75% of respondents work for private companies.
70% of respondents work for ESPs. 10% work for retailers or brands sending through their own infrastructure.
That’s my initial pass through the data. I’ll put together something a bit more coherent and some more useful analysis in the coming week and publish it. I am already seeing some interesting correlations I can do to get useful info out.
Thank you to everyone who participated! This is interesting data that I will be passing along to Google. Rough mental calculation indicates that respondents are responsible for multiple billions of emails a day.
Thanks!

Google Postmaster Tools: Last Chance!

I’ll be closing down the Google Postmaster Tools survey Oct 31. If you’ve not had a chance to answer the questions yet, you have through tomorrow.
This data will be shared here. The ulterior motive is to convince Google to make an API available soon due to popular demand.

Tell us about how you use Gmail Postmaster Tools

One of the things I hear frequently is that folks really want access to Google Postmaster Tools through an API. I’ve also heard some suggestions that we should start a petition. I thought a better idea was to put together a survey showing how people are using GPT and how high the demand is for an API.
They’re a data company, let’s give them data.

I’ve put together a survey looking at how people are using GPT. It’s 4 pages and average time to take the survey is around 7 minutes. Please give us your feedback on GPT usage.
I’m planning on leaving the survey open through the first week in November. Then I’ll pull data together and share here and with Google.

Warmup advice for Gmail

Getting to the Gmail inbox in concept is simple: send mail people want to receive. For a well established mail program with warm IPs and domains, getting to the inbox in practice is simple. Gmail uses recipient interaction with email to determine if an email is wanted or not. These interactions are easy when mail is delivered to the inbox, even if the user has tabs enabled.
When mail is in the bulk folder, even if it’s wanted, users are less likely to interact with the mail. Senders trying to change their reputation to get back to the inbox face an uphill battle. This doesn’t mean it’s impossible to get out of the bulk folder at Gmail, it’s absolutely possible. I have many clients who followed my advice and did it. Some of these clients were simply warming up new IPs and domains and needed to establish a reputation. Others were trying to repair a reputation. In both cases, the fixes are similar.

When I asked colleagues how they handled warmup at Gmail their answers were surprisingly similar to one another. They’re also very consistent with what I’ve seen work for clients.

August 2017: The month in email

Hello! Hope all are keeping safe through Harvey, Irma, Katia and the aftermath. I know many people that have been affected and are currently out of their homes. I am proud to see so many of my fellow deliverability folks are helping our displaced colleagues with resources, places to stay and money to replace damaged property.
Here’s a mid-month late wrapup of our August blog posts. Our favorite part of August? The total eclipse, which was absolutely amazing. Let me show you some pictures.

Ok, back to email.
We’re proud of the enormous milestone we marked this month: ten years of near-daily posts to our Word to the Wise blog. Thanks for all of your attention and feedback over the past decade!
In other industry news, I pointed to some interesting findings from the Litmus report on the State of Email Deliverability, which is always a terrific resource.
I also wrote about the evolution of filters at web-based email providers, and noted that Gmail’s different approach may well be because it entered the market later than other providers.
In spam, spoofing, and other abuse-related news, I posted about how easy it is for someone to spoof a sender’s identity, even without any technical hacks. This recent incident with several members of the US presidential administration should remind us all to be more careful with making sure we pay attention to where messages come from. How else can you tell that someone might not be wholly legitimate and above-board? I talked about some of what I look at when I get a call from a prospective customer as well as some of the delightful conversations I’ve had with spammers over the years.
In the security arena, Steve noted the ongoing shift to TLS and Google’s announcement that they will label text and email form fields on pages without TLS as “NOT SECURE”. What is TLS, you ask? Steve answers all your questions in a comprehensive post about Transport Layer Security and Certificate Authority Authorization records.
Also worth reading, and not just for the picture of Paddington Bear: Steve’s extremely detailed post about local-part semantics, the chunk of information before the at sign in an email address. How do you choose your email addresses (assuming they are not assigned to you at work or school…)? An email address is an identity, both culturally and for security purposes.
In subscription best practices — or the lack thereof — Steve talked about what happens when someone doesn’t quite complete a user registration. Should you send them a reminder to finish their registration? Of course! Should you keep sending those reminders for 16 months after they’ve stopped engaging with you? THE SURPRISING ANSWER! (Ok, you know us. It wasn’t that surprising.)

Google Postmaster bad IP reputation

There are widespread reports this morning (9/11/17) that Google postmaster tools is showing bad IP reputation for IPs starting on 9/9. This issue is affecting just about everyone. Looking through my client’s postmaster pages, I’m seeing red for IP reputation on every client. Even my clients with generally good reputation are seeing bad reputation since 9/9.

This looks like a reporting or a display error on the part of Google. Many people who are reporting the bad IP reputation are not seeing any significant change in Gmail deliverability.
Looking through client data it appears that domain reputation reporting stopped on 9/8. I am seeing FBL reports for 9/9 and 9/10, for some but not all clients.
My current read on the situation is that something broke internally with the Gmail postmaster reporting. This does not currently appear to be affecting delivery of mail. (If anyone sees differently, drop me an email or tweet me @wise_laura).
I know folks are making sure Google knows. I know that some Gmail folks were directly notified and another Google person is active on Mailop. And we have confirmation that they are aware and are working on fixing it. I will let you know if I hear of a fix timeline.
EDIT: It’s been fixed. Google even fixed the older data. Same client, screenshot from this morning.

Improving Gmail Delivery

Lately I’m hearing a lot of people talk about delivery problems at Gmail. I’ve written quite a bit about Gmail (Another way Gmail is different, Gmail filtering in a nutshell, Poor delivery at Gmail but no where else, Insight into Gmail filtering) over the last year and a half or so. But those articles all focus on different parts of Gmail delivery and it’s probably time for a summary type post.

July 2017: The month in email

August is here, and as usual, we’re discussing spam, permissions, bots, filters, delivery challenges, and best practices.

One of the things we see over and over again, both with marketers and with companies that send us email, is that permission is rarely binary — companies want a fair amount of wiggle room, or “implied permission” to send. There are plenty of examples of how companies try to dance around clear permissions, such as this opt form from a company we used to do business with. But there are lots of questions here: can you legitimately mail to addresses you haven’t interacted with in 5 years? 10 years? What’s the best way to re-engage, if at all?
We frequently get questions about how to address deliverability challenges, and I wrote up a post about some of the steps we take as we help our clients with this. These are short-term fixes; for long-term success, the most effective strategy is sending email that people want and expect. Engagement is always at the core of a sustainable email program.
We’ve also discussed the rise of B2B spam, and the ways in which marketing technologies contribute to the problem. B2B marketers struggle to use social and email channels appropriately to reach customers and prospects, but still need to be thoughtful about how they do it. I also wrote about some of the ways that marketing automation plugins facilitate spam and how companies should step up to address the problem. Here’s an example of what happens when the automation plugins go awry.
I wrote a few posts about domain management and the implications for security and fraud. The first was about how cousin domain names can set users up for phishing and fraud, and the second was a useful checklist for looking at your company’s domain management. We also looked at abuse across online communities, which is an increasing problem and one we’re very committed to fighting.
I also highlighted a few best practices this month: guidelines for choosing a new ESP and active buttons in the subject line for Gmail.
And finally, we celebrated the 80th birthday of the original SPAM. If you’re a regular reader of this blog, you probably already know why unwanted email is called SPAM, but just in case, here’s a refresher….

Another way Gmail is different

I was answering a question on Mailop earlier today and had one of those moments of clarity. I finally managed to articulate one of the things I’ve known about Gmail, but never been able to explain. See, Gmail has never really put a lot of their filtering on the SMTP transaction and IP reputation. Other ISPs do a lot of the heavy lifting with IP filters. But not Gmail.
While I was writing the answer I realized something. Gmail was a late entrant into the email space. AOL, Hotmail, Yahoo, even the cable companies, were providing email services in the 90s. When spam started to be a problem, they started with IP based blocking. As technology got better and content filtering became viable, improvements were layered on top of IP based blocking.

Gmail didn’t enter the mailbox market until the 2000’s. When they did, they had money, lots of hardware, and internal expertise to do content filtering. They didn’t start with IP based filtering, so their base is actually content filtering. Sure, there were some times when they’d push some mail away from the MTAs, but most of their filtering was done after the SMTP transaction. The short version of this is I never really pay any attention to IP reputation when dealing with Gmail. It’s just another factor. Unless you’re blocked and if you get blocked by Gmail, wow, you really screwed up.
Gmail does, of course, do some IP based blocking. But in my experience IP filters are really only turned against really egregious spam, phishing and malicious mail. Most email marketers reading my blog won’t ever see IP filters at Gmail because their mail is not that bad.
Other companies aren’t going to throw away filters that are working, so the base of their filters are IPs. But Google never had that base to work from. Their base is content filters, with some IP rep layered on top of that.
That’s a big reason Gmail filters are different from other filters.

Active buttons in the subject line

This morning I waded into a twitter discussion with a bunch of folks about some issues they were having with delivery to gmail. The discussion started with a blog post at detailed.com describing how some senders are seeing significant drops in open rates. I thought I’d take a look and see if I can help, because, hey, this is an interesting problem.
I signed up for a bunch of the mail that was seeing gmail problems and discovered that one of them had the confirmation link in the subject line. How cool is that?

I’ve known about the Gmail subscription line functionality for a while, but this is the first time I’ve seen it in the wild.
The action is in a <div> tag at the bottom of the email. Gmail has been allowing actions in subject lines for a while, this is just the first time I’ve seen it used for subscriptions. It’s so cool.
Want to add one to your post? Instructions are available from Google on their Email Markup pages.

Filtering by gestalt

One of those $5.00 words I learned in the lab was gestalt. We were studying fetal alcohol syndrome (FAS) and, at the time, there were no consistent measurements or numbers that would drive a diagnosis of FAS. Diagnosis was by gestalt – that is by the patient looking like someone who had FAS.
It’s a funny word to say, it’s a funny word to hear. But it’s a useful term to describe the future of spam filtering. And I think we need to get used to thinking about filtering acting on more than just the individual parts of an email.

Filtering is not just IP reputation or domain reputation. It’s about the whole message. It’s mail from this IP with this authentication containing these URLs. Earlier this year, I wrote an article about Gmail filtering. The quote demonstrates the sum of the parts, but I didn’t really call it out at the time.

The cycle goes on

Monday I published a blog post about the ongoing B2B spam and how annoying it is. I get so many of these they’re becoming an actual problem. 3, 4, 5 a day. And then there’s the ongoing “drip” messages at 4, 6, 8, 12 days. It is getting out of control. It’s spam. It’s annoying. And most of it’s breaking the law.
But, I can also use it as blog (and twitter!) fodder.

From the archives: Taking Permission

From February 2010, Taking Permission.

Permission is always a hot topic in email marketing. Permission is key! the experts tell us. Get permission to send email! the ISPs tell us.
Marketers have responded by setting up processes to “get” permission from recipients before adding them to mailing lists. They point to their privacy polices and signup forms and say “Look! the recipient gave us permission.”
In many cases, though, the permission isn’t given to the sender, permission is taken from the recipient.
Yes, permission is being TAKEN by the sender. At the point of address collection many senders set the default to be the recipient gets mail. These processes take any notion of giving permission out of the equation. The recipient doesn’t have to give permission, permission is assumed.
This isn’t real permission. No process that requires the user to take action to stop themselves from being opted in is real permission. A default state of yes takes the actual opt-in step away from the recipient.
Permission just isn’t about saying “well, we told the user if they gave us an email address we’d send them mail and they gave us an email address anyway.” Permission is about giving the recipients a choice in what they want to receive. All too often senders take permission from recipients instead of asking for permission to be given.
Since that post was originally written, some things have changed.
CASL has come into effect. CASL prevents marketers from taking permission as egregiously as what prompted this post. Under CASL, pre-checked opt-in boxes do not count as explicit permission. The law does have a category of implicit permission, which consists of an active consumer / vendor relationship. This implicit permission is limited in scope and senders have to stop mailing 2 years after the last activity.
The other change is in Gmail filters. Whatever they’re doing these days seems to really pick out mail that doesn’t have great permission. Business models that would work a few years ago are now struggling to get to the inbox at Gmail. Many of these are non-relationship emails – one off confirmations, tickets, receipts. There isn’t much of a relationship between the sender and the recipient, so the filters are biased against the mail.
Permission is still key, but these days I’m not sure even informed permission is enough.

Gmail filtering in a nutshell

Gmail’s approach to filtering; as described by one of the old timers. This person was dealing with network abuse back when I was still slinging DNA around as my job and just reading headers as a hobby.

Sharing access to Google Postmaster Tools

As a delivery consultant, I always ask clients to share their Google postmaster reports with me. As Gmail is one of the bigger delivery challenges for a lot of senders, having access to the postmaster tools helps tease out issues. I had some issues earlier this week getting access to tools and so brought up a conversation on one of the delivery lists. The nice folks there helped me get it solved.
A few hours later someone asked me how do I get access and I thought that was a brilliant idea for a blog post today.

December 2016: The Month in Email

Happy New Year! We’re looking forward to some interesting new projects this year, both for our clients and for Word to the Wise. Stay tuned!
December was a slow month for blogging, with everything going on. But we’re back on the horse now and ready to blog for 2017.

List and subscription management continue to be hot topics, especially in the wake of the listbombing attacks earlier this year. Earlier this month, I presented a webinar on listbombing for the EEC and DMA to review the attacks and discuss best practices for companies to manage subscriptions. For Ask Laura, I wrote about the unsubscribe process and how senders can best manage those requests to keep their lists current and compliant.
With all the holiday mail flying around, Steve wrote up a good post about the challenges of DNS hosting and issues customers may have reaching your site. He also wrote about canonicalization, a process for comparing things to see if they are the same, which is useful for understanding how messages change during the delivery process. It’s important to understand how this works with DKIM, as that process specifically looks at changes to messages in delivery to validate them.
I wrote a post about how delivery at Gmail is a bit different from other mail providers, which can lead to intermittent delivery problems, and got some useful information in the comments about some upcoming process changes. And as always, unwanted email is SPAM. It doesn’t matter if you call it outreach or prospecting, or “here’s something you might find interesting!” Still SPAM.

Poor delivery at Gmail but no where else

I’ve mentioned before that I can often tell what ISP is making filter changes by what my calls are about. The last few weeks it’s been Gmail where folks are struggling to get to the inbox. One of the things most clients and potential clients have mentioned is that they’re not having any problems at the other major ISPs.

Parasites hurt email marketing

As a small business owner I am a ripe target for many companies. They buy my address from some lead generation firm, or they scrape it off LinkedIn, and they send me a message that pretends to be personalized but isn’t really.
“I looked at your website… we have a list of email addresses to sell you.”
“We offer cold calling services… can I set up a call with you?”
“I have scheduled a meeting tomorrow so I can tell you about our product that will solve all your technical issues and is also a floor wax.”
None of these emails are anything more than spam. They’re fake personalized. There’s no permission. On a good day they’ll have an opt out link. On a normal day they might include an actual name.
These are messages coming to an email address I’ve spent years trying to protect from getting onto mailing lists. I don’t do fishbowls, I’m careful about who I give my card to, I never use it to sign up for anything. And, still, that has all been for naught.
I don’t really blame the senders, I mean I do, they’re the ones that bought my address and then invested in business automation software that sends me regular emails trying to get me to give them a phone number. Or a contact for “the right person at your business to talk to about this great offer that will change your business.”
The real blame lies with the people who pretend that B2B spam is somehow not spam. Who have pivoted their businesses from selling consumer lists to business lists because permission doesn’t matter when it comes to businesses. The real blame lies with companies who sell “marketing automation software” that plugs into their Google Apps account and hijacks their reputation to get to the inbox. The real blame lies with list cleansing companies who sell list buyers a cleansing service that only hides the evidence of spamming.
There are so many parasites in the email space. They take time, energy and resources from large and small businesses, offering them services that seem good, but really are worthless.
The biologically interesting thing about parasites, though, is that they do better if they don’t overwhelm the host system. They have to stay small. They have to stay hidden. They have to not cause too much harm, otherwise the host system will fight back.
Email fights back too. Parasites will find it harder and harder to get mail delivered in any volume as the host system adapts to them. Already if I look in my junk folder, my filters are correctly flagging these messages as spam. And my filters see a very small portion of mail. Filtering companies and the business email hosting systems have a much broader view and much better defenses.
These emails annoy me, but I know that they are a short term problem. As more and more businesses move to hosted services, like Google Apps and Office365 the permission rules are going to apply to business addresses as well as consumer addresses. The parasites selling products and services to small business owners can’t overwhelm email. The defenses will step in first.

iOS List Unsubscribe Functionality

Al did a great post over on Spamresource about the how the new list unsubscribe function in the default mail client from iOS10. What’s been interesting to me is how much I’m hearing from ESP folks about how their customers want it gone.
If you don’t know what we’re talking about, in the default mail client on iOS10, Apple is now offering a way to unsubscribe from list mail by placing an unsubscribe link at the top of the message.

As you can see, this isn’t just for commercial mail, it’s in place for every mailing list that has a List-Unsubscribe header. (This is a screenshot from something I posted to OI this morning). For me, it’s somewhat intrusive. I’m on a lot of discussion lists – technical, marketing, business and even a couple social ones. Reading them on my phone has become a challenge, as every email in a thread contains the “unsubscribe” button now.
Luckily, you can dismiss the message for all posts to that mailing list by hitting the ⮾⮾⮾⮾x. Interestingly, once you’ve turned it off there seems to be no way to turn it back on for that list.
Senders have different complaints, however, they do not have to do with intrusiveness or usability issues.
I’ve heard complaints about placement and about how easy it makes it to unsubscribe. One person even stated that everyone knows the place for an unsubscribe is at the bottom of a message and it should never be at the top of a message. I find these arguments unpersuasive. Unsubscribing should be easy. Unsubscribing should be trivial. People should be able to stop getting mail on a whim. Particularly here in the US, where unsolicited mail is legal, being able to quickly opt-out is the only thing keeping some of our mailboxes useful.
I’ve also heard some concerns that are a little more understandable. One company was concerned that unsubscribes go directly to their ESP rather than directly to them. This is a somewhat more understandable concern. Good senders use unsubscribes as part of their KPIs and as part of their campaign metrics. They know how much an unsubscribe costs them and will use that as part of their metrics for defining a successful campaign. Still, though, it’s not that big a concern. ESPs are already handling these kinds of unsubscribes from providers like gmail and hotmail.
Almost 7 years ago I blogged about a sender who wanted an unsubscribe link in the email client. It was a bit of snark on my part. The interesting part, though, is that some senders want unsubscribe mediated in the client and others things it’s horrible. I think this tells me that there’s no universal right answer. It Depends might be the most hated statement in deliverability, but it is the absolutely the reality of the situation.

Mail Client Improvements

There’s been extensive and ongoing development of email through the years, but much of it has been behind the scenes. We were focused on the technology and safety and robustness of the channel. We’re not done yet, but things are much better than they were.
The good part of that is there is some space to make improvements to the inbox as well. Over the last few months there have been a number of announcements from different mail client providers about how they’re updating their mail client.

Responsive design just got easier at Gmail

Today Gmail announced they are supporting media queries in Gmail and Google Inbox. This should simplify the creation of emails for multiple platforms. The full list of supported rules can be found on the Google Developer Site.

Gmail showing authentication results to endusers

A bit of older news, but worth a blog post. Early in August, Gmail announced changes to the inbox on both the web interface and the android client. They will be pushing authentication results into the interface, so end users can see which emails are authenticated.

These are not deliverability changes, the presence or absence of authentication will not affect inbox delivery. And the gmail Gmail support pages clarify that lack of authentication is not a sign that mail is spam.
This isn’t a huge change for most ESPs and most senders. In fact, Gmail has reported more than 95% of their mail is authenticated with either SPF or DKIM. Now, Gmail does a “best guess” SPF – if it looks like an IP should be authorized to send mail for a domain (like the sending IP is the same as the MX) then it’s considered authenticated.
It’s good to see authentication information being passed to the end user.

Gmail / Apps authentication issues

I’ve seen several reports of unexpected rejections for unauthenticated email to Google over IPv6 today. Unauthenticated mail over IPv6 is a bad idea, but Google usually spam folders it rather than rejecting it.
The Gmail status dashboard is reporting an issue “Some messages sent to consumer Gmail accounts are being rejected due to authentication enforcement” so something isn’t working as intended.

Insight into Gmail filtering

Last week I posted a link to an article discussing how Gmail builds defenses to protect their users from malicious mail. One of the things I found very interesting in that article was the discussion about how Gmail deploys many changes at once, to prevent people from figuring out what the change was.
Let’s take a look at what Gmail said.

Don't mess with my email

One thing we tell clients is that people consider their mailbox a very personal space. They’re offended when people invade that personal space without permission, sometimes to an extent that doesn’t seem proportional to the scale of the offense. And we advise senders who have been invited into the inbox to treat it with respect.
Google don’t seem to realize that.
Today, they replaced one of the two “Send Mail” buttons (and the associated key sequence that people have in their finger memory) with one that silently attached a Minions mic-drop gif to the mail, and then hid any future replies to that mail thread. Quite apart from the fact that people use their gmail accounts for professional communications, this is also sabotaging what many people consider their most personal online space. (And, to make it worse, they had a bug such that sometimes the gif would also be added to mail using the other “Send Mail” button).
There’s No Way This Could Go Horribly Wrong.
drop
People were very, very unamused. Google had already pulled the feature by the time I heard about it this morning.
Never take peoples’ mailboxes for granted. Never.

Gmail showing authentication info

Yesterday Gmail announced on their blog they would be pushing out some new UI to users to show the authentication and encryption status of email. They are trying to make email safer.
There are a number of blog posts on WttW for background and more information.

Things you need to read: 2/5/16

Ask the Expert: How Can Email Marketers Stay Out of Gmail Jail and in the Inbox? The expert in question is an old friend of mine, Andrew Barrett. I met Andrew online in the late 90s, and we worked together (briefly) at MAPS. He was out of email for a while, but I’m pleased he came back to share his talents with us. The information in the article is valuable for anyone who struggles with getting to the Gmail inbox.
Unclutter Your Inbox, Archive & Keep Your Messages. Shiv Shankar talks about some new features at Yahoo Mail. With a simple click, you can archive email so it’s available to search, but not cluttering up your inbox. One of the things that jumped out at me from that article is that Yahoo is providing 1 TB of storage. That’s more than Google!
The EEC is doing a survey on the impact of CASL and want to hear from marketers. Go check out their blog post and take their survey.
Sparkpost has a guest blog from Alex Garcia-Tobar, co-founder of Valimail about common DKIM failures. I’ve met Alex a few times and I’ve always found him a pleasure to talk to. Alex is somewhat new in the email space, but he really gets some of the challenges in the authentication space. A lot of the issues he mentions in that blog post like lack of key rotation and shared keys are some of the technical debt I was talking about in my predictions for 2016 post.
What links have you read this week that are worth sharing?

What do you think about these hot button issues?

It’s been one of those weeks where blogging is a challenge. Not because I don’t have much to say, but because I don’t have much constructive to say. Rants can be entertaining, even to write. But they’re not very helpful in terms of what do we need to change and how do we move forward.
A few different things I read or saw brought out the rants this week. Some of these are issues I don’t have answers to, and some of them are issues where I just disagree with folks, but have nothing more useful to say than, “You’re wrong.” I don’t even always have an answer to why they’re wrong, they’re just wrong.
I thought today I’d bring up the issues that made me so ranty and list the two different points of views about them and see what readers think about them. (Those of you who follow me on Facebook probably know which ones my positions are, but I’m going to try and be neutral about my specific positions.)

Filter complexity

During the Q&A last week, I mentioned an example of a type of filter trying to demonstrate how complex the filters are. There was some confusion about what I was saying, so I thought I’d write a blog post explaining this.

Thanks for the great session

I had a great time answering questions at the 2015 All About eMail Virtual Conference & Expo today. Thanks so much to everyone who participated and asked questions. They were great and I’m sorry we didn’t have more time.
I did get some questions on twitter (@wise_laura) afterwards. One was about an example I gave to explain how filters are complex. There have been rumors going around recently that Gmail is filtering mail with more than 3 URLs in it. Let me just say right now THIS IS NOT TRUE emails with more than 3 URLs in them are being delivered just fine to Gmail.
There is a situation involving the number (and type) of URLs that I think are a useful example of the filter complexity happening at some places, like Gmail. I started working on it, but don’t quite have time to finish it today, but will keep working on and it should go up in the next day or so.
Thanks again to everyone who joined the session. You asked some great questions and I had fun answering them.

SPF debugging

Someone mentioned on a mailing list that mail “from” intuit.com was being filed in the gmail spam folder, with the warning “Our systems couldn’t verify that this message was really sent by intuit.com“. That warning means that Gmail thinks it may be phishing mail. Given they’re a well-known financial services organization, I’m sure there is a lot of phishing mail claiming to be from them.
But I’d expect that a company the size of Intuit would be authenticating their mail, and that Gmail should be able to use that authentication to know that the mail wasn’t a phish.
Clearly something is broken somewhere. Lets take a look.
Looking at the headers, the mail was being sent from Salesforce, and (despite Salesforce offering DKIM) it wasn’t DKIM signed by anyone. So … look at SPF.
SPF passes:

July 2015: The Month in Email

Once again, we reviewed some of the ways brands are trying (or might try) to improve engagement with customers. LinkedIn, who frequently top lists of unwanted-but-legitimate email, announced that they’ll be sending less mail. Josh wrote about giving subscribers options for both the type and frequency of messages, and about setting expectations for new subscribers. In each case, it’s about respecting that customers really want to engage with brands in the email channel, but don’t want the permission they’ve granted to be abused. I also wrote a brief post following up on our June discussion on purchased lists, and as you’d predict, I continue to discourage companies from mailing to these recipients.

Google Postmaster Tools

Earlier this month Google announced a new set of tools for senders at their Postmaster Tools site. To get into the site you need to login to Google, but they also have a handy support page that doesn’t require a login for folks who want to see what the page is about.
We did register, but don’t send enough mail to get any data back from Google. However, the nice folks at SendGrid were kind enough to share their experiences with me and show me what the site looked like with real data, when I spoke at their recent customer meeting.
Who can register?
Anyone can register for Google Postmaster tools. All you need is the domain authenticated by DKIM (the d= value) or by SPF (the Return Path value).
Who can see data?
Google is only sharing data with trusted domains and only if a minimum volume is sent from those domains. They don’t describe what a trusted domain is, but I expect the criteria include a domain with some history (no brand new domains) and a reasonable track record (some or all of the mail is good).
For ESPs who want to monitor all the mail they send, every mail needs to be signed with a common d= domain. Individual customers that want their own d= can do so. These customers can register for their own access to just their mail.
ESPs that want to do this need to sign with the common key first, and then with the customer’s more selective key.
How does it work?
Google collects data from DKIM and/or SPF authenticated mail, aggregates it and presents it to a Google user that has authenticated the domain.
How do I authenticate?

Gmail having issues

As of 7/22/15, 1:17 PM, Google reports the issue is resolved.

Over on the mailop list multiple people are reporting delivery problems to Gmail.
The Google status page confirms this:

Gmail Postmaster Tools for Senders

logo_2x Google announced new postmaster tools for senders sending to Gmail. The Gmail Postmaster Tools are to help “qualified high-volume senders analyze their email, including data on delivery errors, spam reports, and reputation.” The updated postmaster pages also include Gmail’s best practices for bulk senders.
Postmaster Tools by Gmail http://gmail.com/postmaster
Update: ReturnPath has a blog post that includes data and definitions for each of the data points.

Thoughts on Gmail filtering

Gmail has some extremely complex filters. They’re machine learning based and measure hundreds of things about incoming mail. The filters are continually adjusting to changes and updating how they treat specific mail.
One consequence of continually adjusting machine learning filters is that filtering is not static. What passes to the inbox now, may not pass in a couple hours.
One of the other challenges with Gmail filters is that they look at all the mail mentioning a particular domain and so affiliate mail and 3rd party mail can affect delivery of corporate mail.
The good news is that continually adjusting filters adapt to positive changes as well as negative ones. In fact, I recently made a segmentation suggestion to a client and they saw a significant increase in inbox delivery at Gmail the next day.
Gmail can be a challenge for delivery, but send mail users want and mail does go to the inbox.

Mythbusting deliverability and engagement

Yesterday I published an article talking about an engagement webinar hosted by the EEC and DMA. I made a couple predictions about what would be said.

Email predictions for 2015

Welcome to a whole new year. It seems the changing of the year brings out people predicting what they think will happen in the coming year. It’s something I’ve indulged in a couple times over my years of blogging, but email is a generally stable technology and it’s kind of boring to predict a new interface or a minor tweak to filters. Of course, many bloggers will go way out on a limb and predict the death of email, but I think that’s been way over done.

Even major technical advancements, like authentication protocols and the rise of IPv6, are not usually sudden. They’re discussed and refined through the IETF process. While some of these changes may seem “all of a sudden” to some end users, they’re usually the result of years of work from dedicated volunteers. The internet really doesn’t do flag days.
One major change in 2014, that had significant implications for email as a whole, was a free mail provider abruptly publishing a DMARC p=reject policy. This caused a lot of issues for some small business senders and for many individual users. Mailing list maintainers are still dealing with some of the fallout, and there are ongoing discussions about how best to mitigate the problems DMARC causes non-commercial email.
Still, DMARC as a protocol has been in development for a few years. A number of large brands and commercial organizations were publishing p=reject policies. The big mail providers were implementing DMARC checking, and rejection, on their inbound mail. In fact, this rollout is one of the reasons that the publishing of p=reject was a problem. With the flip of a switch, mail that was once deliverable became undeliverable.
Looking back through any of the 2014 predictions, I don’t think anyone predicted that two major mailbox providers would implement p=reject policies, causing widespread delivery failures across the Internet. I certainly wouldn’t have predicted it, all of my discussions with people about DMARC centered around business using DMARC to protect their brand. No one mentioned ISPs using it to force their customers away from 3rd party services and discussion lists.
I think the only constant in the world of email is change, and most of the time that change isn’t that massive or sudden, 2014 and the DMARC upheaval notwithstanding.
But, still, I have some thoughts on what might happen in the coming year. Mostly more of the same as we’ve seen over the last few years. But there are a couple areas I think we’ll see some progress made.

Google's Inbox Team answers questions on Reddit

The team behind Google’s new Inbox app did an “Ask us Anything” Q&A with reddit on December 3rd. The team consisted of a Product Manager, Designer, and Software Engineer and for two hours the team answered all sorts of questions.
Most of the questions were about new features or supporting additional email providers and it showed just how new this app is, it’s not quite ready to be your primary email client as Inbox only supports personal Gmail accounts. The Inbox team mentions they are working on supporting additional mail providers but does not give a timeline of when that would be available.
For email marketers, Google Inbox shares the same HTML sanitizer and media queries that Gmail does and when asked about email filtering it was mentioned that the direct marketing community would benefit by having a place for their emails within the Promos tab. They describe the Promo tab as

Gmail announces new "Inbox" product

Gmail announced today on their blog a new product “Inbox” to help make the inbox more useful and more of a center of activity.
“We get more email now than ever, important information is buried inside messages, and our most important tasks can slip through the cracks—especially when we’re working on our phones. For many of us, dealing with email has become a daily chore that distracts from what we really need to do—rather than helping us get those things done.”
Inbox lets people organize their emails to help them get things done. Creating tasks, organizing threads and discussions, all of that can now be done in this new application.

August 2014: The Month in Email

Isn’t August the month where things are supposed to slow down? We’re still waiting for that to happen around here… it’s been great to be busy, but we’re hoping to continue to carve out more time for blogging as we move into the fall.
August
As usual, we reported on a mix of industry trends and news, the persistence of spam, and did a deep dive into an interesting technical topic. Let’s start there: Steve wrote a post explaining Asynchronous Bounces (yes, it’s a GNFAB), with some examples of how they’re used and how they can cause operational problems.
In industry news, we did a roundup post of some Gmail changes and a followup post on security issues with non-Latin characters in addresses. We also celebrated the long-awaited release of a wonderful resource from MAAWG that I am very proud to have helped author, the white paper Help! I’m on a Blocklist! (PDF link). We receive dozens of these calls every week, and though we are always happy to help people solve urgent delivery crises, we spend most of our consulting time and attention working with people to build sustainable email programs, so this document is a great “self-service” resource for people looking to troubleshoot blocklist issues on their own.
In other industry and MAAWG-related news, we noted that the nomination period for the J.D. Falk award has opened (you have just a few more days, procrastinators) and took a moment to reminisce about our friend J.D. and his incredible contributions to the field.
On the topic of creating, sending, and reading more attractive email, we posted some resources from Mailchimp and crowdsourcing templates from Send With Us. We also incorrectly reported on a not-actually-new interface from AOL, Alto. Interesting to note that there’s been so little followup from AOL (and almost no post-launch coverage) in the two years since launch.
We also touched on a few myths: email saves trees and low complaint volume is good.
And finally, in November of 2013, I unsubscribed from every possible email I received on a specific account. I followed up on that briefly in a Part 2 post, and this month went back and wrote a Part 3 followup. Spoiler alert: spam is still a problem. Of course, we got some comments that we were probably doing it wrong, so Unsubscribe Barbie showed up to add her thoughts. We try not to be snarky around here, but sometimes we just don’t try very hard.

Protecting users from look-alike accounts

Gmail recently started accepting mail (and calendar invitations) with non-Latin characters. A lot of fraudulent emails use non-Latin characters as a way to fool users. Google is on top of these security issues, however, and is now throwing away some mail with non-Latin characters.

June 2014: The month in email

Each month, we like to focus on a core email feature or function and present an overview for people looking to learn more. This month, we addressed authentication with SPF.
We also talked about feedback mechanisms, and the importance for senders to participate in FBL processes.
In our ongoing discussions about spam filters, we took a look at the state of our own inboxes and lamented the challenge spam we get from Spamarrest. We also pointed out a post from Cloudmark where they reiterate much of what we’ve been saying about filters: there’s no secret sauce, just a continuing series of efforts to make sure recipients get only the mail they want and expect to receive. We also looked at a grey area in the realm of wanted and expected mail: role accounts (such as “marketing@companyname.com”) and how ESPs handle them.
As always, getting into the Gmail inbox is a big priority for our clients and other senders. We talked a bit about this here, and a bit more about the ever-changing world of filters here.
On the subject of list management, we wrote about the state of affiliate mailers and the heightened delivery challenges they face getting in the inbox. We got our usual quota of spam, and a call from a marketer who had purchased our names on a list. You can imagine how effective that was for them.
And in a not-at-all-surprising development, spammers have started to employ DMARC workarounds. We highlighted some of the Yahoo-specific issues in a post that raises more questions.
We also saw some things we quite liked in June. In the Best Practices Hall of Fame, we gave props to this privacy policy change notification and to our bank’s ATM receipts.
We also reviewed some interesting new and updated technology in the commercial MTA space, and were happy to share those findings.

Delivering to Gmail

Gmail is a challenge for even the best senders these days.
With the recent Gmail changes there isn’t any clear fix to getting open rates or inbox delivery back up. Some of it depends on what is causing Gmail to filter the mail. Changing subject lines, from name, from address may get mail back to the inbox in the short term, but it only works until the filters catch up.
What I am seeing, across a number of clients, is that Gmail is doing a lot of content reputation and that content reputation gets spread across senders of that content. That means you want to look at who is sending any mail on your behalf (mentioning your domain or pointing at your website) and their practices. If they have poor practices, then it can reflect badly on you and result in filtering.
From what I’ve seen, these are very deliberate filtering decisions by Google. And it’s making mail a lot harder for many, many senders. But I think it is, unfortunately, the new reality.

What are affiliate mailers?

Affiliate mailers collect email addresses and then rent access to those addresses out to 3rd parties. There are a wide range of vendors that fall into the affiliate category. Some vendors compile lists through co-registration, others compile lists themselves through website opt-ins and some affiliate vendors fulfill mailing requests by hiring affiliates. There are, of course, some senders in the affiliate space that don’t even pretend to send opt-in mail, they just buy, compile or harvest addresses and blast mail to those addresses.

May 2014: The month in email

It’s been a busy and exciting month for us here.
Laura finished a multi-year project with M3AAWG, the Messaging, Malware and Mobile Anti-Abuse Working Group (look for the results to be published later this year) and continued working with clients on interesting delivery challenges and program opportunities. Steve focused on development on the next version release of Abacus, our flagship abuse desk tool, which will also be available later this year.
And as always, we had things to say about email.
The World of Spam and Email Best Practices
We started the month with a bit of a meta-discussion on senders’ fears of being labeled spammers, and reiterated what we always say: sending mail that some people don’t want doesn’t make you evil, but it is an opportunity to revisit your email programs and see if there are opportunities to better align your goals with the needs of people on your email lists. We outlined how we’ve seen people come around to this position after hitting spamtraps. That said, sometimes it is just evil. And it’s still much the same evil it’s been for over a decade.
We also wrote a post about reputation, which is something we get asked about quite frequently. We have more resources on the topic over at the WiseWords section of our site.
Gmail, Gmail, Gmail
Our friends over at Litmus estimate Gmail market share at 12%, which seems pretty consistent with the percentage of blog posts we devote to the topic, yes? We had a discussion of Campaign Monitor’s great Gmail interview, and offered some thoughts on why we continue to encourage clients to focus on engagement and relevance in developing their email programs. We also wrote a post about how Gmail uses filters, which is important for senders to understand as they create campaigns.
SMTP and TLS
Steve wrote extensively this month about the technical aspects of delivery and message security. This “cheat sheet” on SMTP rejections is extremely useful for troubleshooting – bookmark it for the next time you’re scratching your head trying to figure out what went wrong.
He also wrote a detailed explanation of how TLS encryption works with SMTP to protect email in transit, and followed that with additional information on message security throughout the life of the message. This is a great set of posts to explore if you’re thinking about security and want to understand potential vulnerabilities.
DKIM
Steve also wrote a series of posts about working with DKIM (DomainKeys Identified Mail), the specification for signing messages to identify and claim responsibility for messages. He started with a detailed explanation of DKIM Replay Attacks, which happens when valid email is forwarded or otherwise compromised by spammers, phishers or attackers. Though the DKIM signature persists (by design) through a forward, the DKIM specification restricts an attacker’s ability to modify the message itself. Steve’s post describes how senders can optimize their systems to further restrict these attacks. Another way that attackers attempt to get around DKIM restrictions is by injecting additional headers into the message, which can hijack a legitimately signed message. If you’re concerned about these sort of attacks (and we believe you should be), it’s worth learning more about DKIM Key Rotation to help manage this. (Also of note: we have some free DKIM management tools available in the WiseTools section of our site.)
As always, we’re eager to hear from you if there are topics you’d like us to cover in June.

Using Google to taunt coworkers

Happy Friday, all. This has been a rough week for so many people, I thought we needed a little humor.
From Tim Norton (@norton_tim) on Twitter.

Thoughts on Gmail and the inbox

Over the last few months more and more marketers are finding their primary delivery challenge is the Gmail inbox. I’ve been thinking about why Gmail might be such a challenge for marketers. Certainly I have gotten a lot of calls from people struggling to figure out how to get into the Gmail inbox. I’ve also seen aggressive domain based filtering from Gmail, where any mention of a particular domain results in mail going to the bulk folder.
It’s one of those things that’s a challenge, because in most of these cases there isn’t one cause for bulk foldering. Instead there’s a whole host of things that are individually very small but taken together convince Gmail that the mail doesn’t need to be in the inbox.
A pattern that I’m starting to see is that Gmail is taking a more holistic look at all the mail from a sender. If the mail is connected to an organization, all that mail is measured as part of their delivery decision making. This is hurting some ESPs and bulk senders. I’ve had multiple ESPs contact me in the last 6 months looking for help because all their customer emails are going to bulk folder.
Gmail’s filtering is extremely aggressive. From my perspective it always has been. I did get an invite for a Gmail account way back in the day. I moved a couple mailing lists over to that account to test it with some volume and discussion lists. I gave up not long after because no matter what I did I couldn’t get gmail to put all the mail from that list into the tag I had set up for it. Inevitably some mail from some certain people would end up in my spam folder.
Gmail has gotten better, now they will let you override their filters but give you a big warning that the message would have been delivered to spam otherwise.

What are mailers to do? Right now I don’t have a good answer. Sending mail people want is still good advice for individual senders. But I am not sure what can be done about this ESP wide filtering that I’m starting to see. It’s possible Gmail is monitoring all the mail from a particular sender or ESP and applying a “source network” score. Networks letting customers send mail Gmail doesn’t like (such as affiliate mail or payday mail, things they mentioned specifically at M³AAWG) are having all their customers affected.
I suspect this means that ESPs seeing problems across their customer base are going to have to work harder to police their customers and remove problematic mail streams completely. Hopefully, ESPs that can get on the Gmail FBL can identify the problem customers faster before those customers tank mail for all their senders.

More from Gmail

Campaign Monitor has an interview with Gmail looking at how to get mail to the Gmail inbox. It’s a great article and I think everyone should go read it.
One of the most important things it talks about is how complex filters are.

Is gmail next?

I’m hearing hints that there are some malware or phishing links being sent out to gmail address books, “from” those gmail addresses. If that is what’s happening then it’s much the same thing as has been happening at Yahoo for a while, and AOL more recently, and that triggered their deployment of DMARC p=reject records.
It’s going to be interesting to see what happens over the next few days.
I’ve not seen any analysis of how the compromises happened at Yahoo and AOL – do they share a server-side (XSS?) security flaw, or is this a client-side compromise that affects many end users, and is just being targeted at freemail providers one at a time?
Does anyone have any technical details that go any deeper than #AOLHacked and #gmailhacked?

Sendgrid's open letter to Gmail

Paul Kincaid-Smith wrote an open letter to Gmail about their experiences with the Gmail FBL and how the data from Gmail helped Sendgrid find problem customers.
I know a lot of folks are frustrated with Gmail not returning more than statistics, but there is a place for this type of feedback within a comprehensive compliance desk.

Gmail promotions tab improves for marketers

The official Gmail blog announced today that they’re testing a new way of displaying emails in the Promotions tab. This display method will show users a featured image instead of the normal subject line.
Email marketers that want to take advantage of this should visit the Gmail developers pages for information on how to set a featured image for Gmail.
More innovation from Gmail in the mailbox. This one feels pretty consumer friendly, although I still have memories of XXX spam from years ago showing rather explicit images. Gmail must have a lot of confidence in their filtering to push image display to the inbox.

Gmail FBL update

Last week Gmail started contacting ESPs that signed up for their new FBL with more information on how to set up mailings to receive FBL emails.
One of the struggles some ESPs are having is the requirement for DKIM signing. Many of the bigger ESPs have clients that sign with their own domains. Gmail is telling these ESPs to insert a second DKIM signature to join the FBL.
There are a couple reasons this is not as simple or as doable as Gmail seems to think, and the challenges are technical as well as organizational.
The technical challenges are pretty simple. As of now, not all the bulk MTAs support multiple signatures. I’ve heard that multiple signatures are being tested by these MTA vendors, but they’re not in wide use. This makes it challenging for these ESPs to just turn on multiple signatures. For ESPs that are using open source software, there’s often a lot of customization in their signing infrastructure. Even if they have the capability to dual sign, if they’re not currently using that there is testing needed before turning it on.
None of the technical challenges are show stoppers, but they are certainly show delayers.
The organizational challenges are much more difficult to deal with. These are cases where the ESP customer doesn’t want the ESP to sign. The obvious situation is with large banks. They want everything in their infrastructure and headers pointing at the bank, not at their ESP. They don’t want to have that second signature in their email for multiple reasons. I can’t actually see an ESP effectively convincing the various stakeholders, including the marketing, security and legal staff, that allowing the ESP to inset a second signature is good practice. I’m not even sure it is good practice in those cases, except to get stats from Gmail.
Hopefully, Gmail will take feedback from the ESPs and change their FBL parameters to allow ESPs to get information about their customers who sign with their own domain.

Gmail image caching update

Late last year Gmail started caching images on their servers, breaking open tracking in some circumstances. This image caching was good for senders, in that images were back on by default. But it was also bad for senders because it broke dynamic content and didn’t allow for tracking of multiple opens by the same recipient.
According to a new blog post by Moveable Ink this issue has now been resolved and Google is respecting cache headers so senders who are using dynamic content or want to track multiple opens can do so.

Best practices: A Gmail Perspective

At M³AAWG 30 in San Francisco, Gmail representatives presented a session about best practices and what they wanted to see from senders.
I came out of the session with a few takeaways.

ISPs speak at M3AAWG

Last week at M3AAWG representatives from AOL, Yahoo, Gmail and Outlook spoke about their anti-spam technologies and what the organizations were looking for in email.
This session was question and answers, with the moderator asking the majority of the questions. These answers are paraphrased from my notes or the MAAWG twitter stream from the session.
What are your biggest frustrations?
AOL: When senders complain they can’t get mail in and we go look at their stats and complaints are high. Users just don’t love that mail. If complaints are high look at what you may have done differently, content does have an effect on complaints.
Outlook: When we tightened down filters 8 years ago we had to do it. Half of the mail in our users inbox was spam and we were losing a steady number of customers. The filter changes disrupted a lot of senders and caused a lot of pain. But these days only 0.5% of mail in the inbox is spam. Things happen so fast, though, that the stress can frustrate the team.
Gmail: Good senders do email badly sometimes and their mail gets bulked. Senders have to get the basic email hygiene practices right. Love your users and they’ll love you back.
What’s your philosophy and approach towards mail?
AOL: There is a balance that needs to be struck between good and bad mail. The postmaster team reminds the blocking team that not all mail is bad or malicious. They are the sender advocates inside AOL. But the blocking team deals with so much bad mail, they sometimes forget that some mail is good.
Yahoo: User experience. The user always comes first. We strive to protect them from malicious mail and provide them with the emails they want to see. Everything else is secondary.
Gmail: The faster we stop spam the less spam that gets sent overall. We have highly adaptive filters that can react extremely quickly to spam. This frustrates the spammers and they will give up.
Outlook: The core customer is the mailbox user and they are a priority. We think we have most of the hardcore spam under control, and now we’re focused on personalizing the inbox for each user. Everyone online should hold partners accountable and they should expect to be held accountable in turn. This isn’t just a sender / ESP thing, ISPs block each other if there are spam problems.
What are some of your most outrageous requests?
We’ve been threatened with lawsuits because senders just don’t want to do the work to fix things. Some senders try to extort us. Other senders go to the advertising execs and get the execs to yell at the filtering team.
Coming to MAAWG and getting cornered to talk about a particular sender problem. Some senders have even offered money just to get mail to the spam folder.
Senders who escalate through the wrong channels. We spent all this money and time creating channels where you can contact us, and then senders don’t use them.
Confusing business interests with product interests. These are separate things and we can’t change the product to match your business interest.
What are your recommendations for changing behaviors?
Outlook: We provide lots of tools to let you see what your recipients are doing. USE THE TOOLS. Pay attention to your recipient interaction with mail. Re-opt-in recipients periodically. Think about that mail that is never opened. Monitor how people interact with your mail. When you have a problem, use our webpages and our forms. Standard delivery problems have a play book. We’re going to follow that playbook and if you try to get personal attention it’s going to slow things down. If there’s a process problem, we are reachable and can handle them personally. But use the postmaster page for most things.
Gmail: Get your hygiene right. If you get your hygiene right, deliverability just works. If you’re seeing blocking, that’s because users are marking your mail as spam. Pay attention to what the major receivers publish on their postmaster pages. Don’t just follow the letter of the law, follow the spirit as well. Our responsibility, as an ISP, is to detect spam and not spam. Good mailers make that harder on us because they do thinks that look like spammers. This doesn’t get spammer mail in more, it gets legitimate mail in less. Use a real opt-in system, don’t just rely on an implied opt-in because someone made a purchase or something.
Yahoo: ESPs are pretty good about screening their customers, so pay attention to what your ESPs are saying. Send mail people want. Verify that the email addresses given to you actually belong to people who want your mail. Have better sender practices.
What do you think about seed accounts?
The panel wasn’t very happy about the use of seed accounts. Seeds are not that useful any longer, as the ISPs move to more and more personalized delivery. Too much time and too many cycles are used debugging seed accounts. The dynamic delivery works all ways.
When things go wrong what should we do?
AOL: Open a ticket. We know we’ve been lax recently, but have worked out of our backlog and are caught up to date. Using the ticketing system also justifies us getting more headcount and makes everyone’s experience better. Also, don’t continue what you’re doing. Pausing sending while you’re troubleshooting the issue. We won’t adjust a rep for you, but we may be able to help you.
Gmail: Do not jump the gun and open a ticket on the first mail to the spam folder. Our filters are so dynamic, they update every few minutes in some cases. Be sure there is a problem. If you are sure you’re following the spirit and letter of the sender guidelines you can submit a ticket. We don’t respond to tickets, but we work every single one. When you’re opening a ticket provide complete information and full headers, and use the headers from your own email address not headers from a seed account. Give us a clear and concise description of the problem. Also, use the gmail product forum, it is monitored by employees and it’s our preferred way of getting information to the anti-abuse team. Common issues lots of senders are having will get addressed faster.
Outlook: Dig in and do your own troubleshooting, don’t rely on us to tell you what to fix. The support teams don’t have a lot of resources so use our public information. If you make our job harder, then it takes longer to get things done. But tell us what changes you’ve made. If you’ve fixed something, and tell us, our process is different than if you’re just asking for a delisting or asking for information. When you’ve fixed things we will respond faster.
How fast should users expect filters to respond after making changes?
Filters update continually so they should start seeing delivery changes almost immediately. What we find is people tell us they’ve made changes, but they haven’t made enough or made the right ones. If the filters don’t update, then you’ve not fixed the problem.

Gmail pilots new FBL

Yes, it’s true. Gmail announced last Thursday at M3AAWG that they were piloting a new Feedback loop.
The Gmail FBL is currently for ESPs only. The announcement during MAAWG was that only MAAWG ESP members were eligible. They are requiring a DKIM signature for the FBL, but ESPs using individual customer d= values can get a FBL based on IPs. They are also not providing ANY information that reveals the complainer. Gmail’s intention is only to give ESPs feedback so that ESPs can prevent abuse. They are not giving feedback so complainers can be removed.
The email has a .csv attachment that has 3 columns: date, identifier and complaint rate.
The identifier is an ESP provided customer identifier. One of the ESPs I talked to said they were adding an X-header into their emails.
I’ve heard from beta testers that there is a minimum of 100 complaints before you’ll get any report.
Reports are sent daily if there is sufficient traffic to trigger them.
If you’re a MAAWG member, check the senders list for the signup URL.

Gmail opens… anyone seeing changes?

I’m wondering if people are seeing any changes in open rates now that gmail defaulted to on.
Anyone got any quick feedback?

Gmail speaks on image caching

Gmail released a blog post last week discussing their new image caching and why they implemented it. The short version is this is a way to improve the gmail user experience by screening images for malicious activity and serving the images faster from the Google caching machines.

FAQ about opens and Gmail caching

I had hoped to blog about something else today, but this still seems to be a big concern for a number of people. There are a lot of questions running around, some of which we don’t have answers to, others of which we have answers based on some evidence.
It’s important to remember that we’ve seen Gmail roll things out and then roll things back and do phased transitions during deployment. What various people are reporting about images and caching and headers are accurate at the time they are tested. But they may not be accurate tomorrow or in a week or in a month.
I’ve also discovered through this process that a lot of different providers use significantly different image tracking in order to record image loads. Some of these techniques seem to be more resistant to Google’s new image loading process than others.
Why is this all so important?
Image tracking has become a fundamental part of email marketing. It’s something that can be measured, and so a lot of marketers evaluate the effectiveness of an email send based partially on open rate.
How does open tracking work?
For open tracking, ESPs inject a uniquely tagged image into the email. When the recipient opens an email and has images on, the email client calls to the sender server and asks the sender server for all the images in the email. When the tagged image is returned to the recipient, the server records an “open.”
How does caching break open tracking?
Caching means that only the first load of an image is provided by the sender’s server. Subsequent loads of an image are served by the caching proxy. Caching proxies are nothing new; they just haven’t affected email enough in the past for us to have to talk about it.
Why are some people reporting zero problems?
The first load of a unique image always happens. Some folks don’t measure repeat opens, so they’re not even noticing any changes in their reporting thus are saying they’re seeing no problems.
What else is image tracking used for?
Image tracking can also be used for device detection by reading the “user-agent” string that each device returns. Gmail is currently rewriting the “user-agent” string thus breaking all device detection. The string is unique enough that it would be possible to tag those opens as “opened through gmail web interface.” Gmail may decide to pass through the user agent in the future, the HTTP standard does allow for that.
Image tracking can also be used for geolocation. Some senders use the location of an IP address to return images relevant to a user’s location. The accuracy of geolocation is totally dependent on the accuracy of the IP to location database used; it is a best guess of the user’s location. Gmail is currently not passing through the user’s IP address when requesting the original image. I don’t expect them to start, given they also don’t reveal user IPs when Gmail web users send mail. This falls in the same category of privacy protection.
Is there a workaround?
I have heard of a few people claiming they have a fix. The problem is all of the fixes I have seen involve doing things that violate the HTTP RFCs. For instance, the “fix” or “workaround” discussed at E-Mail Marketing Tipps is to not send back an image at all. This is working now to track repeat opens, but Gmail may adapt and block this as well. It’s also possible that Gmail may decide people trying to “work around” Gmail’s cache should be blocked outright for violating the HTTP spec.
Where can I find more information?
Other blog posts on the issue, including research on what people have seen.

More info about Gmail image caching

A lot of people are discussing the new Gmail image caching around the web.
This doesn’t yet appear to be rolled out across all of Google’s network, so some people in different parts of the world are reporting different behaviors. This is leading to a little bit of confusion, as folks are reporting things like seeing multiple opens for a single image. These reports are clearly accurate, but may only be an artifact of a slow rollout across the network.
There are a couple bullet points I think are important.

Gmail deploys image proxy servers

This afternoon Justin Foster of LiveClicker posted to the OnlyInfluencers list asking about Gmail rewriting links.

Do Gmail tabs hurt email marketing?

Earlier this year, Gmail rolled out a new way for users to organize their inbox: tabs. Tabs were an attempt by Gmail to help Gmail users organize their mail, particularly programmatically generated email like social media alerts and marketing mail. While many of us took a wait and see approach, a number of email marketers took this as one of the 7 signs of the apocalypse and the end of email marketing as we know it.
Dozens of marketers wrote article with such titles as “7 ways to survive Gmail tabs” and headlines that declared “Thanks to Gmail’s new tabs, promotional e-mails are now shunted off to a secondary inbox. If you rely on e-mail marketing, you should be worried.” Marketers large and small responded by sending emails to recipients begging them to move marketing mail out of the promotions tab and into the inbox.
A number of bloggers, reporters and marketers, myself included, tried to tame the panic. Not because we necessarily supported tabs, but because we really had no insight into how this would affect recipients interacting with email.
This week Return Path published a whitepaper on the effect of Gmail tabs on email marketing (.pdf link).
Not only did Return Path’s research show little negative effect of tabs, they actually saw some positive effects of tabs on how recipients interact with commercial email. Overall, the introduction of tabs in the gmail interface may be a improvement for email marketers.

This month in email: September 2013

Looking back through the month of September there were a couple things talked about on the blog.

Google wiretapping case, what the judge ruled

Yesterday I reported that the judge had ruled on Google’s motion to dismiss. Today I’ll take a little bit deeper look at the case and the interesting things that were in denial of the motion to dismiss.
Google is being sued for violations of federal wiretapping laws, the California invasion of privacy act (CIPA) and wiretapping laws in Florida, Pennsylvania and Maryland. This lawsuit is awaiting class certification for the following groups.

Judge sides with plaintiff, refuses to dismiss wiretapping suit against Google

Judge Koh published her ruling on Google’s motion to dismiss today.
It’s a 43 page ruling, which I’m still digesting. But the short answer is that Google’s motion was denied almost in total. Google’s motion was granted for two of the claims: that email is confidential as defined by the California Invasion of Privacy Act (CIPA, section 632) and dismissal of a claim under Pennsylvania law.

Gmail tabs … good for marketers?

It appears to be Google’s turn as the subject of most of my blog posts these days.
Consumerist had a post up today talking about the new Gmail tabs. Interestingly enough, they’re quoting an Ad Age article that says the new tabs are not hurting engagement.

Gmail promotions tab

Chad White explains why you shouldn’t ask recipients to move mail from the promotions tab to the inbox.

Gmail says no expectation of privacy, kinda.

Consumer Watch put out a press release yesterday about a court filing made by Gmail that says Gmail users have no expectation of privacy. I pulled a bunch of the docs yesterday, but have had no real time to read or digest them.
For recap users everything I pulled (and stuff other people have pulled) are available at Archive.org.
The initial complaint was filed under seal at the request of Google. The redacted complaint doesn’t tell us a lot, but it’s available for people to read if they’re interested.
The doc everyone is talking about is Google’s Motion to Dismiss. Everyone is up in arms about Google saying, in that filing, “a person has no legitimate expectation of privacy in information he voluntarily turns over to third parties.” (page 28, line 9). What no one seems to have mentioned is that this is actually a quote from a case that Google is referencing. The whole paragraph may lead one to a different conclusion.

Are the new Gmail ads email?

I’ve seen lots of opinions over the last few weeks about whether or not the new ads in the Gmail promotions tab are email or not.

Ads in the Gmail Tabbed Inbox

One of the features of the new Gmail tabbed inbox is email-like ads placed by Gmail.

Inbox challenges and dull email in the tabbed inbox

Getting to the inbox is becoming a greater and greater challenge for many marketers. According to Return Path, 22% of opt in mail doesn’t make it to the inbox.
The challenge to marketers is that a lot of opt in mail isn’t important to the recipient. Sure, they’re happy enough to get it if they notice it, but if it’s not there then they don’t care. They’ll buy from an email ad, but it might not be something they’ll seek out. Recipient behaviour tells the ISPs that the mail isn’t all that important, and a lot of it is just background noise so the ISP not delivering it to the inbox doesn’t matter.
Email marketing is like the Girl Scout of the Internet. If the Girl Scout shows up at your doorstep, you’re probably going to buy those 3 boxes of thin mints. But if she doesn’t, that’s OK. If you really want the cookies, you’ll find the co-worker who is taking orders for his daughter. Or you’ll find the table outside the local coffee shop. The Girl Scout showing up on your doorstep makes it more convenient, but she’s not critical to get your fix. Of course, the bonus of the Girl Scout on the doorstep is that a lot of people who won’t go find the cookies will buy when she’s on the doorstep.
A lot of email marketing triggers purchases that recipients would make anyway. They think they might want a particular product, and when they get that coupon or discount or even just a reminder they make the purchase. The email triggers the purchase of a product the buyer intends to purchase anyway. Some email marketing trigger purchases of things the recipient didn’t know existed, but is so enticing after one email they can’t live without. Some email marketing triggers an impulse purchase. In most of these categories, if mail doesn’t show up in the inbox, the recipient really doesn’t miss it.
Many marketers, despite loud protests that all their mail is important and wanted, know this. That’s why so many marketers are having conniptions about the new Gmail tabbed inbox. They’re losing access to the impulse.
From the data I’ve seen, tabs are effecting email marketing programs. Some programs are seeing more revenue, some are seeing less. I think it really remains to be seen what the long term effects are. For many recipients the new tabbed inbox is a new way to interact with their email. Change is hard, and there is a period of adaptation whenever an interface changes. We really don’t know what the long term effect of tabs on sales will be. Sales may go back to previous levels, sales may increase over previous levels, sales may decrease from current levels or sales may stay at their current levels. The full effect isn’t going to be obvious for a while.
It does mean, though, that email marketers need to step up their game. Email marketing in the age of a tabbed inbox might be less about the impulse purchase and more about cultivation and long term branding.

One letter off…

I’m working on a blog post about the new Gmail tabbed inbox and the messages Gmail is inserting into the promotions tab. The messages aren’t showing up on most of my accounts, so I logged into an infrequently used account of mine. Ads are there, I got my screenshots and some data about the behaviour of the messages. So far so good.
I also discovered that at least two other women are using my address. One of them apparently ordered a bunch of wedding stuff from David’s Bridal shop using my email address. I hope Kirstie got her special order in time.
The other case is more interesting. I found dozens of emails in my inbox from what appeared to be friends including me in their email forward chain.
The Comic Sans. The FW:FW:FW:FW:FW subject lines. The horribly drawn cartoons. The inspirational messages. The prayer requests. The invites to bridge night. The followup demands that I reply to their invites for bridge night. The sad emails that I didn’t go to bridge night. There were emails from grandchildren. Questions about where I’d been and if I moved. Prayer chains. The messages go on and on.
Looking back through my inbox, this has been going on since sometime late in 2012. (Told you this was an infrequently used account). I looked and looked and I think I figured out what happened. A woman named Helen appears to to have an email address one letter off from mine (string@ vs stringsstring@) and one of her church friends tried to reply to her and dropped the ‘s’ from the email address. Once she did that, everyone else just kept hitting “reply all” and are including me in their forward chain.
It’s not commercial, it’s not spam. It’s just a bunch of people mistyping an email address and sending mail to someone they don’t know. I’m kinda glad it was a bunch of church ladies rather than Carlos Danger sending … well… Carlos Danger type messages.
People get email addresses wrong sometimes. It happens (ask me about the time I almost got my mailserver blocked because I mistyped an address while sending mail to a blocklist maintainer and hit a trap address by mistake…). The problem is that it can overwhelm an uninvolved person’s mailbox, even when it’s not commercial. Sure, if I was logging in to this account more often I’d probably have shut it down, but if they were paying attention they would have realized Helen is never replying to anything they send.
I kinda feel the same about commercial mailers that send me mail over and over and over again. I never open it, I never reply to it, I never respond to it. I wonder if there is actually anyone actually sending the mail, or if there’s just a lonely mailserver bricked up in a wall somewhere continually sending out spam.
Don’t be the bricked up server in the wall. Pay attention to what your recipients are doing.

Gmail's new inbox tabs. News at 11.

Yesterday Gmail announced a change to their UI. This new UI lets users configure tabs in their inbox for different sorts of email. This change has greatly upset some marketers. Yesterday I heard it described as war on marketers, as a conspiracy to stop all email marketing and as a horrible injustice to legitimate marketers. I even saw a few people call for an organized boycott of Google AdWords.
While I do appreciate many of us don’t like change, I can’t quite jump on the histrionic bandwagon. This change isn’t Google declaring war on marketers. Google is, at the end of the day, a marketing company. They live and die by marketing dollars. And before you ask, I don’t really think email marketers can organize a boycott that actually has any real impact on Google’s bottom line and causes them to change their interface.
There are a lot of reasons I don’t think this is the actual end of the world and that marketers should just take a deep breath and chill.
The tabbed interface is really just Priority Inbox v. 2. Priority inbox was rolled out a few years ago and there was quite a bit of noise about how that was going to make email marketing more difficult. While getting email to the inbox at Gmail is a challenge for many marketers, I don’t think Priority Inbox is the underlying reason. I think Gmail has gotten a lot stricter on filters, particularly content filters thus making it harder for borderline mail to get to the inbox instead of the bulk folder.
The tabbed interface is just another way of organizing mail in the inbox. Mail is not moved to any different folders, it’s still in the inbox. Users can enable or disable the settings as they desire and all of the mail stays in their inbox.
The interface is not on by default. Users have to actually go in and turn on the setting. For users who don’t set up filters anyway, it’s unlikely they’re going to take advantage of the tabs. I did take a look at the configuration settings. Gmail tries to make it clear what kinds of mails will end up in what tabs by telling you what From: addresses currently in your inbox will end up in a tab if you enable it.
Overall, I don’t think this is really going to cause horrible repercussions to email marketers. In fact, this does seem to offer some benefit to email marketers that use consistent branding. According to Mickey Chandler at Exacttarget, the interface “not only display[s] the number of new emails in the tab, but [also displays the] names of the brands whose mails are in that tab.” This is a good thing for marketers, who now have the chance to get their name in the inbox interface.
One thing I did notice, too, was that when I enabled tabs, Gmail presented me with more advertising in the “promotions” tab and provided no advertising in any other tabs.

4 things the new outlook ads tell us about email

Microsoft has a new TV ad showing how trivial it is to remove unwanted email from the inbox. Various busy people use the “sweep” and “delete” functions to clean up mail. The commercial even have a segment counting up the hundreds of emails deleted.
This tells me a few things.

The challenge of Gmail

A lot of my sales inquiries recently are about getting good inbox delivery at Gmail. I’ve mentioned before, I can usually tell when an ISP changes things because they suddenly become the subject of a great many phone calls.
In this case, Gmail seems to have turned up their engagement filters and is sending a lot more mail to the bulk folder. I have also noticed other people are blogging about Gmail delivery problems. Al eventually determined that it was mailings sent from other IPs that were degrading the delivery of his customer’s emails.
Gmail, more than the other major ISPs, seems to not be weighting IP reputation very heavily these days. They’re looking at domain reputation and they’re using all mentions of a domain in that reputation. A lot of senders, some of them spammers, segregate their email streams (acquisition, marketing, transactional) across IP addresses in order to stop poorly performing mails from harming delivery of other emails they’re sending. But Gmail’s current filtering scheme seems designed to focus on domain reputation and minimize the impact of IP reputation.
This is making the Gmail inbox tough to reach for a lot of mailers these days. Even in cases where the mailer isn’t hiring affiliates or actively partitioning mail, if a domain is seen frequently in spam then delivery for that whole domain is hurting. Signing with DKIM and publishing a DMARC record may help. But the reality right now is that there doesn’t seem to be a silver bullet into the Gmail inbox.

DKIM and Gmail

After they were a a little embarrassed by their own DKIM keys being poorly managed a few months ago, Google seem to have been going through their inbound DKIM handling and tightening up on their validation so that badly signed mail that really shouldn’t be treated as DKIM signed, won’t be treated as signed by Gmail.
This is a good thing, especially as things like DMARC start to be layered on top of DKIM, but it does mean that you really need to check your signing configuration and make sure you’re not doing anything silly.

Gmail sending out warnings for 512 bit DKIM keys

As an update to yesterday’s post, Gmail is contacting postmasters at domains signing with 512 bit keys to warn them of the upcoming changes. This message also clarifies “DKIM keys failing.” Messages signed with 512 bit keys or less will be treated as unsigned by Gmail in the next week or so.

How long is your DKIM key?

While we were at M³AAWG, Wired published an article talking about how simple it was to crack DKIM keys. I didn’t post about it at the time because it didn’t really seem like news. DKIM keys smaller than 1024 are vulnerable and not secure and the DKIM spec does not recommend using keys smaller than 1024. When I asked the DKIM-people-who-would-know they did tell me that the news was that the keys had been cracked and used in the wild to spoof email.
Fair enough.
If you are signing with DKIM, use a key 1024 or longer. Anything shorter and your risk having the key cracked and your mail fraudulently signed.
This morning M³AAWG published recommendations on keeping DKIM keys secure.

Gmail filtering

Derek Harding has a pair of articles on ClickZ about Gmail giving their users information about why a particular email message was filtered.
What Gmail Teaches Us about Spam Filtering
Gmail Filtering: The Spam Disposition
Both articles are worth a read. They talk about what we know about Gmail and what we can infer from the data they provide to senders.

Getting rid of the via at Gmail

There was a question submitted today about the verification process at Gmail.

Gmail and the bulk folder

Earlier this week Gmail announced they were providing reasons for why they delivered a particular mail to the bulk folder. I’m sure a lot of senders are rejoicing over the clear feedback. After all this is exactly what they’ve been asking for “tell us why you’re filtering our mail and we’ll fix it.”
I am not sure, however, that this is going to help the majority of senders seeing mail going to the bulk folder. On the Gmail support pages, they list a number of the explanations they’re be providing.

Email marketing OF THE FUTURE!

ISPs are continually developing tools for their users. Some of the newer tools are automatic filters that help users organize the volumes of mail they’re getting. Gmail released Priority Inbox over a year ago. Hotmail announced new filters as part of Wave 5 back in October.
All of these announcements cause much consternation in the email marketing industry. Just today there was a long discussion on the Only Influencers list about the new Hotmail filtering. There was even some discussion about why the ISPs were doing this.
I think it’s pretty simple why they’re creating new tools: users are asking for them. The core of these new filters is ISPs reacting to consumer demand. They wouldn’t put the energy into development if their users didn’t want it. And many users do and will use priority inbox or the new Hotmail filtering.
Some people are concerned that marketing email will be less effective if mail is not in the inbox.

Spammers and Google+

I have a google+ account, but don’t check it very often. There seems to be a significant amount of noise on the feeds and trying to keep up with all the people who added me to circles was driving all the real mail out of my gmail inbox.
This morning I realized the noise just got louder. It seems spammers are buying very, very old lists scraped from usenet and inviting everyone on those lists to join them on Google+. Yup, an address of mine that has not been used in 7 or 8 years and is not very publicly associated with me got a Google+ invite from someone I’ve never heard of before.
I know there have been a lot of complaints about spammers abusing Google+. I thought it was possible, but I didn’t realize they were actually purchasing email lists to load into Google and spam people.

Gmail abuse and postmaster addresses

A long time ago, Steve wrote a post about setting up abuse and postmaster addresses for Google hosted domains. Google has gone through a couple iterations of the interface since then, as you can see by the comment stream.
I checked with some people who have Google hosted domains and they have confirmed that abuse@ and postmaster@ addresses can be set up by creating a group. When you create the group you can then add yourself to the group and get the mail that comes into abuse@ and postmaster@.

Return Path speaks about Gmail

Melinda Plemel has a post on the Return Path blog discussing delivery to Gmail.

Gmail and the via

I was hoping to have a detailed post up today about the conditions where gmail presents the user with a “via” but time seems to have gotten away from me. But I can give you the conclusions.

Gmail shows authentication data to the recipient

Yesterday Gmail rolled out some changes to their interface. One of the changes is that they are now showing end users authentication results in the user screen.
It’s really the next step in email authentication, showing the results to the end user.
So how does Google do this? Google is checking both SPF and DKIM. If mail is authenticated and the authentication matches the from address then they display the email as:

If we click on “details” for that message, we find more specific information.
In this case the mail went through our outgoing mailserver to gmail.
Mailed-by indicates that the message passed SPF and that the IP address is a valid source of mail from wordtothewise.com.
Signed-by shows the domain in the DKIM d=. In this case, we signed with the subdomain dt.wordtothewise.com. That’s what happens when you sign using the domain in the From address (or a subdomain of it).
For a lot of bulk senders, though, their mail is signed using their ESP’s domain instead. In that case Gmail shows who signed the mail as well as the from address.

And when we click on “details” for that message we see:
This is an email from a sender using Madmimi as an ESP. Madmimi is handling both the SPF authentication and the DKIM authentication.
As an aside, this particular sender has a high enough reputation that Gmail is offering me an unsubscribe option in their interface.
Gmail is distinguishing between first party and third party signatures in authentication. If the mail is authenticated, but the authentication appears to be handled by a separate entity, then Gmail is alerting recipients to that fact.
What does this mean for bulk senders?
For senders that are signing with a domain that matches their From: domain, there is no change. Recipients will not see any mention of your ESP in the headers.
However, if you are using an ESP that is signing your mail with a domain they own, then your recipients will see that information displayed in the email interface. If you don’t want this to be displayed by Gmail, then you will need to move to first party signing. Talk to your ESP about this. If they’re unsure of how to manage it, you can point them to DKIM Core for an Email Service Provider.
Gmail blogpost about the changes
Gmail help page about authentication results

Gmail reports spear phishing attack

No one, it seems, is immune from account compromise attempts. Today Google reported they had identified a systemic campaign to compromise Gmail accounts belonging to “senior U.S. government officials, Chinese political activists, officials in several Asian countries (predominantly South Korea), military personnel and journalists.”
Google offers a number of solutions for users, including the ability to add 2 factor authentication to your Gmail account. I strongly recommend anyone who uses Gmail to do this.
This isn’t a security blog, but email is one of the major vectors used to infect machines. We’ve seen numerous break ins targeting email senders and ESPs, resulting in customer and recipient data being stolen and then used for spam. Everyone who uses email needs to be aware of the risks and maintain their email account integrity. Be careful clicking links in emails. Be careful opening webpages. Keep your antivirus software up to date.
Everyone is a target.

Changes at Gmail

As I’ve said before, I can usually tell when some ISP changes their filtering algorithm because I start getting tons and tons of calls about delivery problems at that ISP. This past month it’s been Gmail.
There have been two symptoms I’ve been hearing about. One is an increase in bulk folder delivery for mail that previously was reliably hitting the inbox. The other is a bit more interesting. I’ve heard of 3 different mailers, with good reputations and very clean lists, that are seeing 4xx delays on some of their mail. The only consistency I, and my colleagues at some ESPs, have identified is that the mail is “bursty.”
The senders affected by this do send out mail daily, but the daily mail is primarily order confirmations or receipts or other transactional mails. They send bi-weekly newsletters, though, exploding their volume from a few tens of thousands up to hundreds of thousands. This seems to trigger Gmail to defer mail. It does get delivered eventually. It’s frustrating to try and deal with because neither side is really doing anything wrong, but good senders are seeing delivery delays.
For the bulk foldering, Bronto has a good blog post talking about the changes and offering some solid suggestions for how to deal with them. I’m also hearing from some folks who are reliable that Gmail may be rolling back some of the bulk foldering changes based on feedback from their users.
So if you’re seeing changes at Gmail, it’s not just you.

Gmail Evolution

All the cool kids are doing infographics, so here’s our take on the new Gmail Priority Inbox.

Gmail and SenderScore

Return Path discusses that a high (>80) SenderScore is correlated with inbox delivery at Gmail.

Limited email at gmail

Mike Monteiro has a screenshot of what happens when you actually fill up a gmail inbox.
How much of that mail is spam, I wonder?
HT: Laughing Squid

Gmail and the PBL

Yesterday I wrote about the underlying philosophy of spam filtering and how different places have different philosophies that drive their filtering decisions. That post was actually triggered by a blog post I read where the author was asking why Gmail was using the PBL but instead of rejecting mail from PBL listed hosts they instead accepted and bulkfoldered the mail.
The blog post ends with a question:

Why do ISPs do that?

One of the most common things I hear is “but why does the ISP do it that way?” The generic answer for that question is: because it works for them and meets their needs. Anyone designing a mail system has to implement some sort of spam filtering and will have to accept the potential for lost mail. Even the those recipients who runs no software filtering may lose mail. Their spamfilter is the delete key and sometimes they’ll delete a real mail.
Every mailserver admin, whether managing a MTA for a corporation, an ISP or themselves inevitably looks at the question of false positives and false negatives. Some are more sensitive to false negatives and would rather block real mail than have to wade through a mailbox full of spam. Others are more sensitive to false positives and would rather deal with unfiltered spam than risk losing mail.
At the ISPs, many of these decisions aren’t made by one person, but the decisions are driven by the business philosophy, requirements and technology. The different consumer ISPs have different philosophies and these show in their spamfiltering.
Gmail, for instance, has a lot of faith in their ability to sort, classify and rank text. This is, after all, what Google does. Therefore, they accept most of the email delivered to Gmail users and then sort after the fact. This fits their technology, their available resources and their business philosophy. They leave as much filtering at the enduser level as they can.
Yahoo, on the other hand, chooses to filter mail at the MTA. While their spamfoldering algorithms are good, they don’t want to waste CPU and filtering effort on mail that they think may be spam. So, they choose to block heavily at the edge, going so far as to rate limit senders that they don’t know about the mail. Endusers are protected from malicious mail and senders have the ability to retry mail until it is accepted.
The same types of entries could be written about Hotmail or AOL. They could even be written about the various spam filter vendors and blocklists. Every company has their own way of doing things and their way reflects their underlying business philosophy.