Yes, Virginia, there is list churn

Yesterday I talked about how data collection, management, and maintenance play a crucial role in deliverability.  I mentioned, briefly, the idea that bad data can accumulate on a list that isn’t well managed. Today I’d like to dig into that a little more and talk about the non-permanence of email addresses.
A common statistic used to describe list churn is that 30% of addresses become invalid in a year.  This was research done by Return Path back in the early 2000’s. The actual research report is hard to find, but I found a couple articles and press releases discussing the info.

A new study, to be unveiled next week at The 85th Annual Direct Marketing Association conference, indicates that email addresses are changing at the rate of 31% annually, driven by ISP switching, job changes and consumer efforts to avoid SPAM.
The email survey, conducted by independent, third-party research firm NFO WorldGroup, concluded that, consequently, the majority of consumers lose touch with personal and professional contacts and with preferred websites. […] The survey, conducted in August 2002, updates a similar study by Return Path and NFO WorldGroup from September 2000, which identified a 32% annual rate of email address churn. The results are based on responses from 1,015 consumers from NFO WorldGroup’s online panel of U.S. email users over the age of 18. The panel is representative of U.S. online households.  ISP Switching and SPAM Continue to Drive Email Address Changes

While I think the address change rates are probably lower now, list churn still exists.
In 2002, NFO reported users changed personal email addresses for a number of reasons.

  • 50% changed due to an ISP switch
  • 16% changed due to spam
  • 12% changed due to a move
  • 8% changed due to a “more attractive” email address.

Work users also changed addresses, and for many of the same reasons.

  • 41% changed due to new jobs
  • 18% changed due to an ISP change
  • 8% changed due to a residential move
  • 6% change due to a name change (divorce or marriage)

Given the changes in free webmail providers since 2002, I expect address changes due to ISP switching or moving is less common than it was. But other reasons that users cited still exist, including spam levels, new jobs and name changes.
Of course, my gut feeling that these numbers are old and out of date and probably no longer accurate was crushed last week. The LA Times published an article about Hillary Rodham Clinton’s email campaigns. After her run for president in 2007, her email address database had approximately 2.5 million records. According to the article, less than 100,000 of the addresses are still valid. That’s more than 30% attrition every year.
List churn is real. While we may not know what the exact percentage of churn is, we know it happens. I expect that list churn, like most things in deliverability, is related to the actual recipient group. Some lists, like Secretary Clinton’s list, may have a very high churn rate. Other lists focused on different demographics might have a much lower churn rate.
While the LA Times article mentioned these addresses bounced (“an inbox clogged with bounce-back messages”) not all churn is so visible. There’s also “stealth” churn, where addresses are abandoned by their users but still accept mail.
What can you do? Mostly I recommend first wrapping your head around the idea that churn exists. Once you really believe churn is real then you can address how to fix it in your specific environment.
Key things to remember when planning a data management plan:

  • Email addresses are not permanent.
  • Subscriber data degrades if you don’t actively manage it.
  • Deliverability depends on data quality.
  • Maintaining data is easier than trying to clean data.
  • Using list cleaning services will remove hard bounces, but won’t address “stealth” churn, which can still affect deliverability.

If there’s anything my work with clients has taught me is that the more creative and flexible you can be in regards to list management, the more effective your overall email marketing program can be.

Related Posts

June 2015: the Month in Email

Happy July! We are back from another wonderful M3AAWG conference and enjoyed seeing many of you in Dublin. It’s always so great for us to connect with our friends, colleagues, and readers in person. I took a few notes on Michel van Eeten’s keynote on botnets, and congratulated our friend Rodney Joffe on winning the prestigious Mary Litynski Award.
In anti-spam news, June brought announcements of three ISP-initiated CAN-SPAM cases, as well as a significant fine leveled by the Canadian Radio-television and Telecommunications Commission (CRTC) against Porter Airlines. In other legal news, a UK case against Spamhaus has been settled, which continues the precedent we’ve observed that documenting a company’s practice of sending unsolicited email does not constitute libel.
In industry news, AOL started using Sender Score Certification, and Yahoo announced (and then implemented) a change to how they handle their Complaint Feedback Loop (CFL). Anyone have anything to report on how that’s working? We also noted that Google has discontinued the Google Apps for ISPs program, so we expect we might see some migration challenges along the way. I wrote a bit about some trends I’m seeing in how email programs are starting to use filtering technologies for email organization as well as fighting spam.
Steve, Josh and I all contributed some “best practices” posts this month on both technical issues and program management issues. Steve reminded us that what might seem like a universal celebration might not be a happy time for everyone, and marketers should consider more thoughtful strategies to respect that. I wrote a bit about privacy protection (and pointed to Al Iverson’s post on the topic), and Josh wrote about when senders should include a physical address, what PTR (or Reverse DNS) records are and how to use them, testing your opt-out process (do it regularly!), and advice on how to use images when many recipients view email with images blocked.

Read More

Data is the key to deliverability

Last week I had the pleasure of speaking to the Sendgrid Customer Advisory Board about email and deliverability. As usually happens when I give talks, I learned a bunch of new things that I’m now integrating into my mental model of email.
One thing that bubbled up to take over a lot of my thought processes is how important data collection and data maintenance is to deliverability. In fact, I’m reaching the conclusion that the vast majority of deliverability problems stem from data issues. How data is collected, how data is managed, how data is maintained all impact how well email is delivered.
Collecting Data
There are many pathways used to collect data for email: online purchases, in-store purchases, signups on websites, registration cards, trade shows, fishbowl drops, purchases, co-reg… the list goes on and on. In today’s world there is a big push to make data collection as frictionless as possible. Making collection processes frictionless (or low friction) often means limiting data checking and correction. In email this can result in mail going to people who never signed up. Filters are actually really good at identifying mail streams going to the wrong people.
The end result of poor data collection processes is poor delivery.
There are lots of way to collect data that incorporates some level of data checking and verifying the customer’s identity. There are ways to do this without adding any friction, even. About 8 years ago I was working with a major retailer that was dealing with a SBL listing due to bad addresses in their store signup program. What they ended up implementing was tagged coupons emailed to the user. When the user went to the store to redeem the coupons, the email address was confirmed as associated with the account. We took what the customers were doing anyway, and turned it into a way to do closed loop confirmation of their email address.
Managing Data
Data management is a major challenge for lots of senders. Data gets pulled out of the database of record and then put into silos for different marketing efforts. If the data flow isn’t managed well, the different streams can have different bounce or activity data. In a worst case scenario, bad addressees like spamtraps, can be reactivated and lead to blocking.
This isn’t theoretical. Last year I worked with a major political group that was dealing with a SBL issue directly related to poor data management. Multiple databases were used to store data and there was no central database. Because of this, unsubscribed and inactivated addresses were reactivated. This included a set of data that was inactivated to deal with a previous SBL listing. Eventually, spamtraps were mailed again and they were blocked. Working with the client data team, we clarified and improved the data flow so that inactive addresses could not get accidentally or unknowingly reactivated.
Maintaining Data
A dozen years ago few companies needed to think about any data maintenance processes other than “it bounces and we remove it.” Most mailbox accounts were tied into dialup or broadband accounts. Accounts lasted until the user stopped paying and then mail started bouncing. Additionally, mailbox accounts often had small limits on how much data they could hold. My first ISP account was limited to 10MB, and that included anything I published on my website. I would archive mail monthly to keep mail from bouncing due to a full mailbox.
But that’s not how email works today. Many people have migrated to free webmail providers for email. This means they can create (and abandon) addresses at any time. Free webmail providers have their own rules for bouncing mail, but generally accounts last for months or even years after the user has stopped logging into them. With the advent of multi gigabyte storage limits, accounts almost never fill up.
These days, companies need to address what they’re going to do with data if there’s no interaction with the recipient in a certain time period. Otherwise, bad data just keeps accumulating and lowering deliverability.
Deliverability is all about the data. Good data collection and good data management and good data maintenance results in good email delivery. Doing the wrong thing with data leads to delivery problems.
 
 

Read More

Deliverability and IP addresses

Almost 2 years ago I wrote a blog post titled The Death of IP Based Reputation. These days I’m even more sure that IP based reputation is well and truly dead for legitimate senders.
There are a lot of reasons for this continued change. Deliverability is hard when some people like the same email other people think is spam

Read More