Ironport have rolled out an update to their rule engine which has a bug causing mail problems. According to discussion on the mailop list, the new rule engine is folding the header with a line feed (LF) rather than a carriage return (CRLF). This is breaking things, including DKIM signatures. Ironport is aware of the issue. I expect an updated rollout shortly.
AOL migrated the last few users onto the Yahoo infrastructure today. It’s really gone. Also, while looking up some info related to this post, I discovered Verizon Media has a postmaster blog hosted on Tumblr.
I’ve seen a number of reports in different areas talking about an increase in Yahoo user unknowns. The same thing happened almost a year ago. This may be when Yahoo does clean up or it could be a coincidence. Folks at Verizon Media are looking into it
Be careful purchasing lists of DigiMarCon NY 2019 Attendees. There’s at least one list being sold out there that is just a scraped list of addresses. I’ve gotten emails asking for meetings and offering services because I am on the exhibitor list. I’m actually not on the exhibitor list, nor have I ever been to that conference in any capacity.
The ex-AOL and Yahoo folks have done a great job providing information for senders to help troubleshoot their mail. There is a direct link to receive sender support. As I’ve said before, you should always use the form when looking for sender support.
Thanks to all the VM folks who have worked so hard to get a new postmaster site up and running with useful information.
We knew this day would come, but somehow it doesn’t make it any easier.
The AOL postmaster site is gone.
Raise a toast tonight to all the people who created and maintained the site for the last decade and a half.
Raise a toast tonight to the email industry that was.
Thanks to all the AOL folks throughout the years that poured so much into making email better for their customers and the rest of us. Your contributions to the industry from the “this is spam” button to FBLs to ARF.
I’ve been waiting for this to happen. An email verification vendor has left their database of 800 million email addresses along with detailed individual data. unprotected on the internet. Bob Diachenko reported the discovery yesterday on his blog. Wired also ran an article (An Email Marketing Company Left 809 Million Records Exposed Online) based on his findings.
It’s not really a secret I don’t have much time for the vast majority of email verification companies and their business models. The first iteration was to hammer on SMTP servers without sending mail. This wasn’t horribly successful because the process looked like a dictionary attack. ISPs instituted protections against dictionary harvesting long before verification companies were a thing, so this was never terribly successful.
SMTP verification became even less useful when Yahoo, now Verizon Media, moved all delivery failures to the very end of the SMTP transaction. This requires verification companies send actual mail in order to determine if an email address is valid. The verification companies don’t want to do this so they can’t tell anything about @yahoo.com addresses.
What did the verification companies do? They pivoted to maintaining vast amounts of data about individual email addresses. In some cases, I believe they are even taking open and click data from their customers or other sources. Now, when you upload a list to verify they don’t test the address, they just compare it with their current database.
Clearly there are issues here. One is that 30% of email addresses go bad over a year. The verification companies have to be doing something to keep their databases current. I don’t know what that is, but taking delivery data from their customers is one way to do it.
The other issue is that there are data aggregators that collect personal data on us and our online activities and then sell that on. None of us have given permission for verification.io or any of their competitors to collect and store our data. Yet they not only do that, many of them also sell the data to any company who wants it.
This breach, of course, wasn’t based on someone cracking into the vendor’s system. The vendor just left their entire database publicly accessible. But now that it’s clear just how much data about us verification vendors have, it’s not out of the question they’ll be a target moving forward.
I have a lot of objection to email verification in general. They don’t actually verify permission or whether or not mail is wanted or even if the recipient gave the address to the vendor. When they use SMTP probing they are abusing resources belonging to third parties to support their business model. Now they’re aggregating and selling data on hundreds of millions of people.
There are a couple of companies in the space that are different. They’re not just “cleaning” data, but providing platforms so senders can actually collect true permission from their customers. And that’s the real crux of it. Bad data verification companies are all about helping senders get addresses that don’t look like spam by keeping bounces and spamtraps low. Good data verification companies are about helping senders curate lists of recipients who want their mail.
Data ownership and privacy is a big deal. Hundreds of billions of dollars have been made by companies collecting and selling PII. They know all sorts of things about consumers, but the consumers have no control over who has their data or what they do with it. Governments are trying to start regulating PII. GDPR was the first, but there are a lot of groups fighting any sort of privacy laws in the US. I think over the long term consumers are going to expect and require more transparency from data aggregators.
I don’t know where the verification industry is going. I do think it’s going to have to significantly change if it’s going to survive. There are significant filtering advantages to handling all rejections after data (like Verizon Media is doing). But there are a few companies in the space that are trying to change how the industry works and make it, overall, less abusive and more consumer friendly.
A few weeks ago we closed on our new house in Dublin. This weekend we’re going to one of those ‘home shows’ where people try and sell you all sorts of things for your home. We know there are some things we want to do with the house so we’re headed out to the convention centre this weekend. Tickets are “free” but they ask for contact information, including an email address.
Given who we are, this sparked a discussion about the email address we wanted to give them. Right now, we’re in a place where we actually want a lot of email about home stuff. We know we need attic insulation and a new heating system and furniture and so yeah, email from the show vendors is good right now. But we also know that this email address will be traded and sold for the next 20 years. We could set up a tagged address and just route it to /dev/null when we’re tired of mail. Instead, we decided to set up a whole new address to use for house things, one that we could set to bounce when we were bored of getting house related mail.
It will be interesting to see what kind of mail we get over time to this address. Are the marketers smart enough to change what they send based on how long they’ve had our address? Or will the mail change with the seasons? Both are legitimate marketing techniques. It will also be interesting to see how they handle this data in the context of GDPR.
In this case, we’re a clear target for this marketing and, in many ways, receptive to all the stuff they’re selling. We’re receptive, they’re going to send us email, it’s all good. We know what we’re getting into, they are getting good subscribers. Everybody is happy. We’ll continue to be happy with the mail until we’re moved in and feel like all the bits are finished and then we’ll either unsubscribe from everything or, more likely, just turn the address off.
Sometimes I don’t know how savvy marketers always are about their audience, though. Two recent examples come to mind.
A friend of mine got engaged last week. She’s looking at planning a wedding. This is another major opportunity for marketing to collect information and bridal shows are huge. Many brides of the digital generation know what they’re getting into when the give an email address to a bridal magazine or to a bridal show. I’ve seen some discussions that the right thing to do is open a gmail account just to handle wedding planning and subscriptions. But, like buying and furnishing a new house, a wedding is a limited amount of time. Anywhere from a few days to 2 or so years. From what I’ve heard, though, not all wedding vendors are that great about sunsetting addresses.
In another case, I was talking with a startup. They’re a fairly new news / political insight organization that was working with some marketing experts to grow their lists. They decided to use co-reg and it was successful increasing their list size by an order of magnitude. It also tanked their delivery. Part of my end of the conversation was about how to fix their delivery. But that was only part of the conversation. A much bigger piece of the conversation was walking them through some discussion of what audience they were looking for and whether or not co-reg was a good way to find that audience.
Fundamentally, though, “people we can get to give us an email address” do not always equate to “people who want our mail.” Recently one of my ESP clients was dealing with a customer who had a lot of delivery challenges and we eventually worked out the problem set of addresses was from wifi logins. Yes, lots of places expect an email address for a wifi login. Lots of users don’t want any mail based on that login. If those addresses become too big a portion of the mailing list, then it can tank delivery for all subscribers.
Part of the challenge of running a successful email marketing program is understanding your subscribers and your collection processes. Email is an amazing communication channel that is constantly evolving. The audience is evolving in what they want and what their needs are. Technology is evolving. Filters are evolving to handle the morphing threats. What worked yesterday might work today but not tomorrow. Marketers have to evolve, too, or risk not reaching the inbox or their audience.
SenderID was basically SPF version 2. It tried to use the same mechanism as SPF to authenticate the visible from address. In some ways it was a predecessor to DMARC. It was an authentication method championed by Microsoft.
The really important thing to remember is that Hotmail was the only domain that really used SenderID. It was a check built into some versions of Exchange servers, too. But it was never really used outside of Microsoft.
In 2012, the IETF published an informational RFC that looked at deployment of SPF and Sender ID. The author looked at a number of different things and concluded there wasn’t much use of Sender ID.
The absence of significant adoption of the [SUBMITTER] extension, SENDER-ID], and [PRA], indicates that there is not a strong community deploying and using these protocols.
Six months after that RFC was published, Microsoft announced they were moving away from Sender ID. Given they were the only major implementation, this was the signal that it was a dead authentication method.
There is no reason to publish Sender ID records. It’s dead.
There was a discussion on Slack about the economics of email. It’s probably not a surprise that I have opinions (Who owns the inbox? Ownership of the Inbox). There was a discussion about this that was useful enough I’d share it.
Laura: Laura Atkins (me!)
Steve: Steve Atkins (other half of WttW)
Matt V: Director Privacy @ 250ok
Laura: Direct mail to your home the owner of the channel is paid (in the US, the USPS is the owner of your mailbox). The economics of email are different, even when I’m leasing a mailbox, say from Gmail, that mailbox is owned by Google, not the person using it as a marketing channel. Senders only pay half the transaction costs, receivers pay substantially to handle incoming email.
One of the initial drives for filtering was to stop spammers from costing the receivers so much money. Very early on some ISP users had to pay per email to/from the internet (I think that was compuserv?) so every unsolicited message had to be paid for by the recipient. (There was some pay per email at AOL early on, too, I think).
But ISPs had to increase capacity and make significant investments in hardware to cope with the unsolicited email. Handling spam costs them real money, too. All of this lead to the ISP end of the industry basically saying they’d only accept mail from opt-in senders. “We’re happy to deliver mail that recipients have asked for.”
We have expanded “opt-in” to “wanted” and “relevant” because we can measure those things better than we could previously. But marketers don’t own the email channel. At best they own half of it. Once the message touches the recipient MX, the sender isn’t paying for it. The receiver – the ISP or the end user – is.
Goodmail tried to actually make the senders pay for the channel. And they failed in part because the majority of their customers were spammers and the ISPs decided the amount of money that Goodmail was paying them to accept the mail wasn’t paying for the loss of customers.
Bonded Sender (the product that eventually became Return Path Certified) also tried to flip the economic model. You could get certified and if the mail you were sending was spam, then you would forfeit a bond to the companies you spammed.
Habeas tried a different economic model and that didn’t work, either.
Matt V: especially since spammers tend to steal the delivery resources from people as well, so they have a near zero cost and any purchase/transaction/infection/etc… is upside to the the economics of spam are crazy, and that is a good portion of why it is so problematic.
Laura: You can, of course, pay to get to the inbox. Both Verizon Media and Google will let you buy advertising that looks exactly like email. It’s not email, it’s a display ad, but it looks like email.
Steve: “Because infected windows machines are almost free.” is the counterargument to a lot of “we could reduce spam if…” proposals.
Laura: Intrusiveness is another issue, but in this case I’m actually speaking only and directly about the economics and who owns an email box. There are a lot of different answers, depending on how you measure, but “marketers” is never the right answer.
Matt V: also very few people talk about the inbox without filtering and how unusable it would be… email would die quickly if all the filtering were turned off
Laura: Very few people have an unfiltered email box. Even I don’t, although we have a whole lot less filtering than many places for reasons.
Steve: Yeah. Aggressive spam filters are the only thing keeping email marketing – amongst other things – a viable business.
Laura: But I can’t read mail on my phone in the morning if my laptop has been shut down over night. There’s just too much spam to go through. I’ve got to let the filters built into mail.app do it’s thing and then go through and still manually delete between 20 and 50 messages.
Overall, we’re in a position where even in the face of free webmail providers, someone is paying for the inbox. In no cases are email senders covering the cost of their email to the recipient / recipient ISP. This is fine and good. We’ve all opted in to mail we want and enjoy and like. But it’s disingenuous to pretend that email is a channel like bill boards or television or magazine or even direct mail. The economics are just too different.
There are times when I hesitate to call what marketers do “spam.” I can use the euphemisms with the best of ’em. “Cold emails” “Targeted Marketing” “B2B marketing.”
I’ll say it here and now: cold emails are spam. Sales people who are sending enough email that they require automation to actually send the mail are spamming.
Look at this message that just showed up in one of my mailboxes.
In the screenshot, I’m really mad because they’re sending mail to my Women of Email address. That’s not the right place to send WttW email. It will never be the right place to send WttW email.
These emails are just so insulting. I have yet to receive one of these messages where the spammer reads the blog post they’re using to advertise. Spam isn’t going away anytime soon looks at all the software and online services that support this type of spam.
Another example is from earlier this week. The spammer found one of our SORBS posts and identified a link to mcafee.com. The pitch? “You review anti-virus software, so you should link to our whitepaper reviewing 10 different anti-virus programs.”
Yeah, spam is not going away any time soon. And a lot of what hits our inboxes these days is exactly this type of spam. The big companies are reasonably good at filtering the garbage. What we actually get is this kind of spam. It is a problem. It is spam. If you, or your company, sends this kind of mail you are spammers. If you sell software that makes it easy to scrape addresses or automatically followup to mail, you’re selling spamware.
Good article. My question about Gmail engagement is how would I reach someone who has not been opening my emails? Say I want to do a re-engagement campaign. If I temporarily suppress a contact from my list for a period of time and only send to my engaged contacts, will that contact potentially get an email in the future if my reputation improves? Or is the contact essentially lost to the spam folder abyss if their emails start going there for engagement reasons?
The short answer is that yes, you can add in contacts after repairing reputation and expect them to get the mail in the inbox. There are some caveats, though.
Part of any reputation repair process is letting some of your recipients go for good. I know some folks think they can simply repair reputation and then go back to mailing the same as they did before. But that’s not how reputation works. Unless there is one precipitating incident – like a phishing page on your domain or one mailing that is clearly something unintended reputation reflects all the mail that you’re sending. If you get to a place where you have to repair reputation, then you need to make some changes to your data.
Let old subscribers who are unengaged for long periods of time go. 24 months is pretty safe, you can be more aggressive, like 12 or 18 months, but I wouldn’t advise being less aggressive.
Next trickle folks back into your active mailings slowly. Don’t take your full 2 year database and mail it. That’s the way to destroy all your hard work on reputation repair. Instead, start adding recent engagers in batches. There are different ways to structure the batches. For instance, you can increase your list by 10% a week, adding in old addresses. You may find that there is a point where you see a reputation change – like you’re adding addresses from 18 months ago and Gmail reputation falls or FBL emails increase. This is a sign to slow down, stop or change tactics.
Whatever you do, monitoring is key. Your own internal metrics – FBL numbers, Google postmaster tools, probe accounts (yours and commercially available ones), opens, clicks, bounces – will tell let you monitor how delivery is going. You can make adjustments on the fly. Try things like slowing down the addition of addresses or move the new addresses into a re-engagement stream rather than your main mail stream. Decisions are driven by data. Collect everything you can get.
Overall, the population of recipients you choose for reputation repair isn’t the only population of recipients you will ever be able to contact. Unless a recipient actually marked you as spam, you will be able to reach their inbox.
Many companies have the occasional “oops” where they send email they probably shouldn’t have. This can often cause a decrease in reputation and subsequent delivery problems. Some companies rush to fix things by changing domains.
Getting a new domain does not fix the problem!
Brand new domains, those registered less than 30 days, have really bad reputations. Blame the spammers and scammers who exploited a loophole and sent tons of untraceable spam from newly registered domains that they then abandoned without paying for them. So unless you have a domain waiting in the wings you’re not going to improve your reputation by switching.
Even if you do have a registered but unused domain in your back pocket, moving to that domain isn’t going to help. These days, domains need to be warmed just like IPs do. Depending on where you’re mailing, warmup can take 4 – 6 weeks to accomplish. Domains need to be warmed even if you’re putting them on currently warmed IPs.
Fundamentally, it’s easier to rebuild a domain reputation than it is to warm up a new domain. This is especially true when the reputation destroying incident is a one-time or short term thing. For instance, sometimes a company will need to send a legal notice to their whole database. This may hurt overall domain reputation in the short term. However, if there’s a history of good mail and the sends quickly return to that good place, then reputation won’t be damaged over the long term.
Companies that panic and switch domains are stuck warming up for weeks. They don’t have the history behind them that compensates for short term problems.
Even in the cases where there have been ongoing and long term problems, filters will often adapt faster to good practices on an established domain than they will to good practices on a new domain.
Changing domains is (almost) never the solutions to domain reputation problems.