Email Address
August 2017: The month in email
Hello! Hope all are keeping safe through Harvey, Irma, Katia and the aftermath. I know many people that have been affected and are currently out of their homes. I am proud to see so many of my fellow deliverability folks are helping our displaced colleagues with resources, places to stay and money to replace damaged property.
Here’s a mid-month late wrapup of our August blog posts. Our favorite part of August? The total eclipse, which was absolutely amazing. Let me show you some pictures.
Ok, back to email.
We’re proud of the enormous milestone we marked this month: ten years of near-daily posts to our Word to the Wise blog. Thanks for all of your attention and feedback over the past decade!
In other industry news, I pointed to some interesting findings from the Litmus report on the State of Email Deliverability, which is always a terrific resource.
I also wrote about the evolution of filters at web-based email providers, and noted that Gmail’s different approach may well be because it entered the market later than other providers.
In spam, spoofing, and other abuse-related news, I posted about how easy it is for someone to spoof a sender’s identity, even without any technical hacks. This recent incident with several members of the US presidential administration should remind us all to be more careful with making sure we pay attention to where messages come from. How else can you tell that someone might not be wholly legitimate and above-board? I talked about some of what I look at when I get a call from a prospective customer as well as some of the delightful conversations I’ve had with spammers over the years.
In the security arena, Steve noted the ongoing shift to TLS and Google’s announcement that they will label text and email form fields on pages without TLS as “NOT SECURE”. What is TLS, you ask? Steve answers all your questions in a comprehensive post about Transport Layer Security and Certificate Authority Authorization records.
Also worth reading, and not just for the picture of Paddington Bear: Steve’s extremely detailed post about local-part semantics, the chunk of information before the at sign in an email address. How do you choose your email addresses (assuming they are not assigned to you at work or school…)? An email address is an identity, both culturally and for security purposes.
In subscription best practices — or the lack thereof — Steve talked about what happens when someone doesn’t quite complete a user registration. Should you send them a reminder to finish their registration? Of course! Should you keep sending those reminders for 16 months after they’ve stopped engaging with you? THE SURPRISING ANSWER! (Ok, you know us. It wasn’t that surprising.)
Open subscription forms going away?
A few weeks ago, I got a call from a potential client. He was all angry and yelling because his ESP had kicked him off for spamming. “Only one person complained!! Do you know him? His name is Name. And I have signup data for him! He opted in! How can they kick me off for one complaint where I have opt-in data? Now they’re talking Spamhaus listings, Spamhaus can’t list me! I have opt-in data and IP addresses and everything.”
We talked briefly but decided that my involvement in this was not beneficial to either party. Not only do I know the complainant personally, I’ve also consulted with the ESP in question specifically to help them sort out their Spamhaus listings. I also know that if you run an open subscription form you are at risk for being a conduit for abuse.
This abuse is generally low level. A person might sign up someone else’s address in an effort to harass them. This is a problem for the victim, but doesn’t often result in any consequences for the sender. Last week’s SBL listings were a response to subscription abuse happening on a large scale.
The source of deliverability problems
Most deliverability problems don’t start where many people think they do. So very often people call looking for deliverability help and tell me all about the things they’re doing to reach the inbox. They’ll tell me about content, they’ll tell me about bounces, they’ll talk about complaints, engagement, opens and clicks. Rarely will they bring up their list source without some prompting on my part.
The reality is, though, that list source is to root of deliverability success and deliverability problems. Where did those addresses come from and what do the people who gave them think you’re going to do with them?
Outsourcing collection to a third party can cause significant issues with delivery. Letting other people collect addresses on your behalf means you lack control over the process. And if you’re paying per address, then there monetary incentive for that company to pad the list with bogus addresses.
Sometimes there are even issues with having your own employees collect addresses from customers. For instance, a retailer requires sales associates collect a minimum percentage of addresses from customers. The company even ties the associates’ evaluations to that percentage. Associates have an incentive to submit addresses from other customers. Or a retailer will offer a discount for an address and customers want the discount but not the mail, so they give a fake address.
All of these things can affect deliverability.
Address collection is the key to delivery, but too many companies just don’t put enough attention to how they’re collecting addresses and entering into the relationship with subscribers. This is OK for a while, and delivery of small lists collected like this can be great. But as lists grow in size, they come under greater scrutiny at the ISPs and what used to work doesn’t anymore.
The first step to diagnosing any delivery problem is to look at the list. All of the things ISP use to measure reputation measure how well you’re collecting addresses. Changing IPs or domains or content doesn’t change the reason mail is being filtered. It just means the filters have to figure out something new to key on.
Want great deliverability? Start with how you’re collecting addresses.
Want to fix deliverability? Start with how you’ve collected addresses, how you’ve stored them and how you’ve maintained them.
Horses, not zebras
I was first introduced to the maxim “When you hear hoofbeats, think horses not zebras” when I worked in my first molecular biology lab 20-some-odd years ago. I’m no longer a gene jockey, but I still find myself applying this to troubleshooting delivery problems for clients.
It’s not that I think all delivery problems are caused by “horses”, or that “zebras” never cause problems for email delivery. It’s more that there are some very common causes of delivery problems and it’s a more effective use of time to address those common problems before getting into the less common cases.
This was actually something that one of the mailbox provider reps said at M3AAWG in SF last month. They have no problem with personal escalations when there’s something unusual going on. But, the majority of issues can be handled through the standard channels.
What are the horses I look for with delivery problems.
This month in email: February 2014
After a few months of hiatus, I’m resurrecting the this month in email feature. So what did we talk about in February?
Industry News
There was quite a bit of industry news. M3AAWG was in mid-February and there were actually a few sessions we were allowed to blog about. Gmail announced their new pilot FBL program. Ladar Levinson gave the keynote talking about the Lavabit shutdown and his new darkmail program. Brian Krebs won the Mary Litynski award for his work in investigating online security issues. The 4 major mailbox providers talked about their spam filters and spam filtering philosophy.
February was also the month where different companies evaluated their success or failure of products. LinkedIn announced the shutdown of their Intro product and Facebook announced the shutdown of their Facebook.com email service.
Security Issues
Cloudmark published their 2013 report on the Global Spam Threat and we discovered that the massive Target breach started through phishing. I also noticed a serious uptick in the amount of phishing mails in my own mailbox. There is new round of denial of service attacks using NTP amplification. We provided information on how to secure your NTP servers.
Address Collection
The Hip Hop group De La Soul released their entire catalog for free, online, using a confirmed opt-in email process. On the flip side, the M3AAWG hotel required anyone logging into the wifi network to give an email address and agree to receive marketing mail. We also discovered that some political mailing lists were being used in ways the politicians and recipients didn’t expect.
Email Practices
I talked about how to go about contacting an ISP that doesn’t have a postmaster page or a published method of contact. Much of that information is actually relevant for contacting ISPs that do have a contact method, too. Finally, I talked about how ISPs measure engagement and how that’s significantly different from how ESPs think it is.
Growing your list carefully
Karl Murray wrote a great set of recommendations for growing an email marketing list. I really can’t think of anything I would have said differently. Touching customers and getting contact information from them is great, but there are situations where this gets bad addresses. Too many bad addresses can impact delivery.
So how do you grow your list without falling into a delivery trap? The specific recommendations, as always, depend on your specific situation. But knowing how bad addresses get onto your list will allow you to implement mitigation strategies that actually work.
Yahoo retiring user IDs: why you shouldn't worry
A couple weeks ago, Yahoo announced that they were retiring abandoned user IDs. This has been causing quite a bit of concern among email marketers because they’re not sure how this is going to affect email delivery. This is a valid concern, but more recent information suggests that Yahoo! isn’t actually retiring abandoned email addresses.
You have to remember, there are Yahoo! userIDs that are unconnected to email addresses. People have been able to register all sorts of Yahoo! accounts without activating an associated email account: Flickr accounts, Yahoo groups accounts, Yahoo sports accounts, Yahoo news accounts, etc,. Last week, a Yahoo spokesperson told the press that only 7% of the inactive accounts had associated email addresses.
Turning that around, 93% of the accounts currently being deactivated and returned to the user pool have never accepted an email. Those addresses will have hard bounced every time a sender tried to send mail to that address.
What about the other 7%? The other 7% will have been inactive for at least a year. That’s a year’s worth of mail that had the opportunity to hard bounce with a 550 “user unknown.”
If you’re still concerned about recycled Yahoo userIDs then take action.
About the @ sign
The @ sign is ubiquitous online. We use it and we don’t think about it. But the history of the @ sign is more complicated than we realize.
Read MoreCan you verify email addresses in real time?
In a recent discussion about spamtraps and address lists and data collection a participant commented, “[E]very site should be utilizing a real-time email address hygiene and correction service on the front end.” He went on to explain that real time hygiene prevents undeliverable addresses and spamtraps and all sorts of list problems. I was skeptical to say the least.
Yes, there are APIs that can be queried at some of the larger ISPs to identify if an account name is taken, but this doesn’t mean that there is an associated email address. Yes, senders can do a real time SMTP transaction, but ISPs are quick to block SMTP transactions that quit before DATA.
I decided to check out one service to see how accurate it was. I’m somewhat lucky in that I created a username at Yahoo Groups over a dozen years ago but never activated the associated email address. This means that the account is shown as taken and no one else can register that address at Yahoo. But the address doesn’t accept any mail.
A brief guide to spamtraps
“I thought spamtraps were addresses harvested off webpages.”
“I thought spamtraps were addresses that were valid and now aren’t.”
Read MoreSetting expectations at the point of sale
In my consulting, I emphasize that senders must set recipient expectations correctly. Receiver sites spend a lot of time listening to their users and design filters to let wanted and expected mail through. Senders that treat recipients as partners in their success usually have much better email delivery than those senders that treat recipients as targets or marks.
Over the years I’ve heard just about every excuse as to why a particular client can’t set expectations well. One of the most common is that no one does it. My experience this weekend at a PetSmart indicates otherwise.
As I was checking out I showed my loyalty card to the cashier. He ran it through the machine and then started talking about the program.
Cashier: Did you give us your email address when you signed up for the program?
Me: I’m not sure, probably not. I get a lot of email already.
Cashier: Well, if you do give us an email address associated with the card every purchase will trigger coupons sent to your email address. These aren’t random, they’re based on your purchase. So if you purchase cat stuff we won’t send you coupons for horse supplies.
I have to admit, I was impressed. PetSmart has email address processes that I recommend to clients on a regular basis. No, they’re not a client so I can’t directly take credit. But whoever runs their email program knows recipients are an important part of email delivery. They’re investing time and training into making sure their floor staff communicate what the email address will be used for, what the emails will offer and how often they’ll arrive.
It’s certainly possible PetSmart has the occasional email delivery problem despite this, but I expect they’re as close to 100% inbox delivery as anyone else out there.
Signing up for lists
How many email marketers hand over email addresses whenever asked? Are those of us in the email field more or less likely than the average consumer to sign up for something?
I sign up for a lot of mail, but there are different categories of that mail.
Mail I actually want from a company. Usually these are local companies where I visit their brick and mortar or an online only company that I actively buy from. I read the emails for the content and because I’m interested in the company and their products. I occasionally will actually analyze their headers and think about their sending practices. Usually I’m just interested in the sale they’re offering or the information they’re sharing. These companies get a tagged email address that goes into my main mailbox.
Mail where I’m interested in how the company is using email. Generally these are big, national brands. Sometimes they’ll ask me for an address during an offline transaction, other times I’ll make a purchase from. I’m not really interested in what they’re offering, but it’s good to keep an eye on how email is being used by large companies with expensive ad agencies and marketing departments. I do look at the headers of the mail, check their authentication and look at the format of the emails. These companies also get tagged address that goes right to my main mailbox.
One thing I don’t do is automatically provide email addresses to companies. This annoys some to no end. “We don’t have an email address on file for you. Do you have an email address?” They never ask if I want to give them the address, they just ask if I have one. I expect a lot of people just say, “Yes, it’s laura@example.com” and don’t think for a second this means they are opting in to mail from that company. I also think that some companies train their phone and sales reps to ask this way in order to get email addresses from people without informed consent.
I also do a lot of signups to client lists. This is mail I want as without copies of the email I can’t do the audits they’ve contracted me to do. I have a set of addresses that go to a special account and are automatically tagged with client and signup information so I can sort and filter by client and website and all sorts of fancy things. I spend a lot of time looking at the structure of the email. I look at headers for compliance with standards and to confirm any authentication is set up correctly. I look at the body for similar reasons.
I also sign up for some mail that I don’t really want to receive. For these classes of mail I have disposable addresses. This can be investigating affiliates (or potential affiliates) for clients. This can be for an ESP client who wants one of their customers investigated. Sometimes I can’t believe a website is for real so I sign up just to see what their hook is.
Using different addresses and different filtering schemes helps me keep all these email uses separate and clear. I can tell what category a mail is in just by the address that it was sent to. I can also filter on “To” addresses, meaning that mail I’ve signed up for doesn’t get caught in my spam filters. Complex? Yes. But it keeps me up to date not only on offers from companies I purchase from, but also on what others are doing in the email marketing world.
Watch those role accounts
Ben at Mailchimp has a post up explaining what role accounts are and why mailing to them can be a problem.
Read MoreProtecting customer data
There have been a number of reports recently about customer lists leaking out through ESPs. In one case, the ESP attributed the leak to an outside hack. In other cases, the ESPs and companies involved have kept the information very quiet and not told anyone that data was leaked. People do notice, though, when they use single use addresses or tagged addresses and know to whom each address was submitted. Data security is not something that can be glossed over and ignored.
Most of the cases I am aware of have actually been inside jobs. Data has been stolen either by employees or by subcontractors that had access to it and then sold to spammers. There are steps that companies can take to prevent leaks and identify the source when or if they do happen.
Email address validation
One thing anyone collecting email addresses anywhere has to think about is address validation. How do you prevent users from typing bad addresses into your forms?
I ran into this yesterday attempting to take an online quiz. Before I was allowed to take the quiz, I had to provide my name, phone number and email address. Initially I attempted to use a tagged email address. This is one that delivers to my wordtothewise.com mailbox, but lets me identify who I initially gave the address to. The form wouldn’t let me give a tagged address “contains invalid characters.” Well, no, it doesn’t, but there are a lot of websites that think + and – and other characters are invalid.
So what did I do? I ended up using a yahoo address associated with my yahoo IM account. An account that may actually not be accepting mail any longer as I rarely log into it.
What did address validation get them in this case? Well, it got them an address I don’t read and may not even be active rather than the address I wanted to give them which would have delivered directly to my primary mail box. Somehow I don’t think this solution is really ideal for them. (It’s great for me, I’ll never know if they ever attempt to contact me.)
Coincidentally, UserGlue posted about email address validation and alternatives to “make them type their address twice.” (Do people do this? I typically cut and paste my address instead of retyping.)
How are other people validating email addresses?
Privacy policies in court
Venkat has an analysis of a case where an individual provided a unique address to a vendor and that vendor released the address in violation of the posted privacy policy. The federal court rejected the suit due to the failure of the plaintiff to provide evidence of harm.
I posted last week about privacy policies and how often they are intentionally or unintentionally violated and when email addresses leak. Courts have consistently ruled against plaintiffs. It seems that the courts believe merely revealing information, even in contradiction to a posted privacy policy, is not actionable by the plaintiff.
As a consumer, I really don’t like the ruling. If a company is going to post a privacy policy, then they should follow it and if they don’t, I should be able to hold them responsible for their lies. Back in the land of reality, I am not surprised at the rulings. Individuals have never owned their personal information, it is the property of the people who compile and sell data
It does mean, however, that privacy polices are not worth the paper they’re written on.
What is an email address? (part three)
As promised last week, here are some actual recommendations for handling email addresses.
First some things to check when capturing an email address from a user, or when importing a list. These will exclude some legitimate email addresses, but not any that anyone is likely to actually be using. And they’ll allow in some email addresses that are technically not legal, by erring on the side of simple checks. But they’re an awful lot better than many of the existing email address filters.
What is an email address? (part two)
Yesterday I talked about the technical definitions of an email address. Eventually on Monday I’m going to talk about some useful day-to-day rules about email address acquisition and analysis, but first I’m going to take a detour into tagging or mailboxing email addresses.
Tagging an email address is something the owner of an email address can do to make it easier to handle incoming email. It works by adding an extra word to the local part of the email address separated by a special character, such as “+”, “=” or “-“. So, if my email address is steve@example.com, and I’m signing up for the MAAWG mailing lists I can sign up with the email address steve+maawg@example.com. When mail is sent to steve+maawg@example.com it will be delivered to my steve@example.com mailbox, but I’ll know that it’s mail from MAAWG. I can use that tag to whitelist that mail, to filter it to it’s own mailbox and a bunch of other useful things.
In some ways this is similar to recent disposable email address services, but rather than being a third party service it’s something that’s been built in to many mailservers for well over a decade. It doesn’t require me to create each new address at a web page, instead I can make tags up on the fly. And it works at my regular mail domain.
If you’re an ESP it can be interesting to look for tagged addresses in uploaded lists. If it’s a list owned by Kraft and you see the email address steve+gevalia@example.com in the list, that’s a strong sign that that email address at least was really volunteered to the list owner. If you see the email address steve+microsoft@example.com then it’s a strong sign that it wasn’t, and you might want to look harder at where the list came from.
One reason that this is relevant to email address capture is that tagged addresses are something that you should expect people, especially more sophisticated users of email, to use to sign up to mailing lists and that they’re something you don’t want to discourage. Yet many web signup forms forbid entering email addresses with a “+” or, worse, have bugs in them that map a “+” sign in the email address to a space – leading to the signup failing at best, or the wrong email address being added to the list at worst. This really annoys people who use tagged addresses to help manage their email, and they’re often exactly the sort of tech-savvy people who make a lot of online purchases you want to have on your lists.
More on Monday.
What is an email address? (part one)
Given we deal with email addresses every day, dozens or thousands or millions of them, it seems a bit strange to ask what an email address is – but given some of the problems people have with the grubbier corners of address syntax it’s actually an interesting question.
There are two real standards that define what is a valid email address and what isn’t. The most complex is RFC 5322 – Internet Message Format, which describes all sorts of things about the structure of an email, including what’s valid to put in From: and To: headers. It’s really too liberal in what it allows an email address to look like to be terribly useful, but it does provide for one very commonly used feature – the friendly from where the name that’s displayed to the recipient is not just the email address.