Data Cleansing part 2


In an effort to get a blog post out yesterday before yet another doctor’s appointment I did not do nearly enough research on the company I mentioned selling list cleansing data. As Al correctly pointed out in the comments they are currently listed on the SBL. And when I actually did the research I should have done it was clear this company has a long term history of sending unsolicited email.
Poor research and a quickly written blog post led to me endorsing a company that I absolutely shouldn’t have. And I do apologize for that.
With all that being said, Justin had a great question in the comments of yesterday’s post about data cleansing.

Isn’t this contrary to the good habits we are always preaching? If we send *email people want* to an engaged, opted-in group of people who want our mail, why would there ever be a need to clean our lists?

Yes, a lot of list cleaning services are used to take non-permissioned lists and turn them into lists that don’t cause delivery problems.  But there are other reasons to clean lists and even clean permission lists.
I fully believe that mail should be sent to people who ask for the mail. I strongly believe the recipient should have some measure of control over what advertising and commercial email they receive. I also believe the recipient is the final arbiter of whether a mail is wanted or unwanted. I believe a legitimate sender must to respect the recipient’s time and attention.
With those principles clearly stated, when might list cleaning be an appropriate process? List remediation is the big one.
We’re hitting the point where some email lists or customer databases with email addresses have been around for almost a decade. There’s a lot of cruft that can accumulate in a database in 10 years. There are going to be addresses with no audit trail. Even newer databases can have a lot of entries without full audit trails.
Some databases have addresses that aren’t mailed regularly. I’ve certainly had clients that would segment enough that some addresses wouldn’t be mailed more than once or twice a year. These types of databases aren’t always kept up as well as we might hope or like.
For these databases, a list cleaning process is good and even necessary. Bad addresses accumulate on lists. One of the things I do with clients is help them separate out good addresses from bad addresses. But each case is unique and requires individualized treatment. Sure, you can run a list against a database of 300 million addresses and remove some bad ones, the ones that might get you into delivery trouble. But not all bad marketing creates delivery problems. Sometimes bad marketing is just bad. Mail gets into the inbox, sure. The source or the content isn’t blocked. But I think marketers can do more than just get mail into the inbox.
Data cleansing is not just about removing spam traps and bouncing addresses. Data cleansing should be about identifying those people who are going to buy from you. And not everyone who was interested in your product a few years ago is going to be interested in your product now. People change, their wants and needs change. They are not static, but rather fluid. Just removing problem addresses isn’t going to find those customers as effectively as searching for the good addresses in your list.

About the author


This site uses Akismet to reduce spam. Learn how your comment data is processed.

  • To be frank, I can’t think of an example where I would be proud cleaning a list with an external blacklist. Even if a list is 10 years old, and particularly for the older addresses, you should only be keeping reactives. I know this because, amongst other sources, I read it on this blog! The reality of the matter (and ESP Deliverability) is, however, a little different…

  • Laura and Justin,
    This is a great discussion– and I wanted to further share OMI’s viewpoint. As evidenced here and in general, there are many differing opinions about e-Marketing, specifically the use of the email channel for direct marketing.
    OMI’s view is that by ensuring that you are starting with the cleanest data possible, marketing respectfully and following the best practices in the direct marketing industry — that you CAN use email for acquisition purposes.
    Our view is that by cleaning your database, you are actually helping to determine if you have non-responders in your database or if that consumer has simply changed email addresses. In order to increase engagement with prospective customers, you have to get to their Inbox to begin with — and fresh, clean data is one of the ways to do this most successfully.
    We agree that you should always respect consumer wishes as to how they would like to engage with your companh. But why do so many in this industry feel that the email channel should be somehow held to a higher standard than other direct marketing channels? For marketers, the use of an integrated multi-channel marketing approach is incredibly important to the success of direct marketing campaigns.
    By using OMI’s Clean-Send service, you can rid your database of inaccurate email data, protect your sender reputation and help to keep yourself from getting listed by Spamaus. By the way, over time, Spamhaus has blacklisted many of the Fortune 500 for simply using email as a marketing channel. Those companies that we work with personally were following the policies of CAN SPAM — along with direct marketing best practices. As does OMI.
    The reason for our Spamhaus listing is due to the fact that we clean, update and refresh our database every 45-60 days. As part of that process, we received an SBL listing. We think this is wrong. We believe that this is a positive step to take to keep your marketing data clean and ensure that you are marketing to consumers who want to be marketed to — not negative.
    You can agree to disagree — but think about it . . . it’s marketing. Spamhaus came into existence to stop true spammers — not inhibit true marketers from being able to build customer relationships.
    Nancy Arter, VP Marketing
    Outward Media (OMI)

  • Except, that’s lazy data hygiene. It’s not even really data hygiene at all, it’s just maintaining and scrubbing against a “flamers list,” something most of us spamfighters observed spammers doing going all the way back to the late 1990s. It was ineffective and somewhat offensive then. Nothing here gives me any confidence that this process is any better or even different.
    My removes list is bigger than yours!
    Also, I’m counting the seconds until somebody starts spamming the scrub list.

  • “But why do so many in this industry feel that the email channel should be somehow held to a higher standard than other direct marketing channels?” Well for one in direct mail marketing the sender bears the cost of postage and receiving a piece of mail doesn’t cost anything (ok ignoring taxes). In the case of email marketing there is a substantial cost for the receiver, in bandwidth and storage and numerous other costs associated with email. Whether is Y! that receives billions of emails per day or Joe’s Bagel shop that gets a few emails a day, spam cost them money – any by prospecting with unwanted and unsolicited email to them, you are infact contribution to the problem. You talk about CAN-SPAM like its the pinnacle to strive for, when in reality its the barrier for entry. And, if you look at the act it also affords receivers the power to set it own standards on what mail to receive on their networks.
    In reality its not you or CAN-SPAM that decides what you should be sending – rather is the receivers of mail – and I think every major ISP out there has pretty much come down on the side of permission based email marketing vs opt-out marketing.

  • “The reason for our Spamhaus listing is due to the fact that we clean, update and refresh our database every 45-60 days.”
    This statement has two implications: first, that you don’t understand the actual reason for your Spamhaus listing (hint: the one you have indicated is completely wrong) and second, that if you do in fact “clean, update, and refresh your database every 45-60 days”, that you aren’t doing that well enough to avoid Spamhaus traps, which in turn implies that you yourself are losing a skills arms-race with them, and that your customers are likely to as well, which significantly devalues your offering.
    Now, I’m not anti-listwashing. I think, if someone is complaining about your mail, the first and most obvious thing you should do is to stop mailing to them. And I’m not anti-list-scrubbing either: for example, I don’t think you should let someone sign up for your newsletter with the email address of your provider’s abuse desk. Those two basic ideas strike me as simple common sense. But those two ideas alone are not the basis for a sensible email marketing strategy.
    Here’s some mildly-edited bullet points taken from a usenet post I made on 02 May 2003:
    A successful mass email campaign needs four things:
    1) permission. “The person being added to the list needs to understand why” — Gil Teriberry. And although confirmed opt-in is a best practice, it’s not always crucial. By way of example, there is currently (or was, in 2003) more than one list specifically for antispammers that does not confirm, and nobody seems to get too bent out of shape about that. The other example given was everybody’s favorite local brewpub. They have a sign-up sheet for their email newsletter. Not a lot of forged subscriptions in the bar.
    2) Identity. They need to know who it is that’s sending them the mail. It needs to be branded. When someone gets an email, they need to think “Oh yeah, I told them to send me email”.
    3) Relevance. The list mail they get needs to be relevant to them. If you have an opt-in list of people interested in your latest advances in soho routers, the people on that list may be interested in other wireless equipment, but they are probably NOT interested in your exciting new offers on Florida vacation properties.
    4) Transparent opt-outs. You presumably made it easy for the user to opt-in and confirm, make it easier for them to opt-out, and make sure that it works. You don’t want to advertise to people who have already told you that they don’t want you to. People who don’t want your ads probably won’t buy your stuff either.
    Now, none of that is new information. I posted that to the internet nine years ago. And sure enough, nine years later, most successful email marketing campaigns still include permission, branding, relevance, and working opt-outs. And yes, while you can TRY to pull off a successful campaign without one or more of those things, that’s really less of a good idea now than it was nine years ago, because in the time since then, ISPs and antispam vendors (like Spamhaus, for the obvious example) have gotten a lot better at detecting people who are bad at it.
    And if you’re currently listed by Spamhaus, that’s a pretty good sign that you’re bad at it.

  • Wow — again — great discussion points. I appreciate the spirited debate going on here!
    So, first of all . . . because you compile email databases does NOT make you a spammer. It makes you a data compiler. And, because we update our database so often, we ARE helping to protect our customer’s sender reputations. Our customers are not spamming — they are MARKETING! We follow all of the tenets of CAN SPAM and completely agree with the legislation.
    What we do not agree with is the idea that the human race is not intelligent. If they want to opt-out of an email campaign — they will and they do. As long as you are providing them the opportunity to do so, you are giving them the choice to handle their marketing communications as they wish to handle them.
    And we didn’t get listed on Spamhaus because we update our database so frequently. By doing so, we are helping marketers NOT email to inactive or invalid email addresses. So, less quantity — but better quality. Our customers are in favor of this and I would think that Spamhaus would be in favor of it too!
    Times have changed in direct marketing — marketers have to be smart about how they market, they have to honor DM recipients channel preferences and they have to provide relevant and quality content so that recipients will respond to the messaging. If you do this — market in a respectful and meaningful way — you will increase revenues for your business. And people can decide how they want to receive those messages.
    So, Huey, I think we agree on the fact that the ideas that you posted in 2003 contain the equation for successful marketing campaigns. I’m just saying, clean up your database — make your data as accurate as possible — then deploy your email campaigns.
    Finally — I disagree — I think we’re really good at what we do. And, I think we’ll convince Spamhaus to see it our way eventually. : )

  • Nancy, sending unsolicited commercial email = SPAMMING! Why in Gods good name do I have to tell you to STOP sending me something I never asked for in the first place.

  • I don’t think you’re going to convince her, Ken. She apparently thinks people who expect opt-in are not as intelligent as those who accept opt-out. That, or she’s using loaded language intentionally. (We’ve never known a spammer to do that, have we?) <wry grin>
    IMHO Spamhaus did us all a favor when they listed these jokers in the SBL. Since they list sellers of lists and providers of e-append services as spam support services frequently, they weren’t particularly targeting this organization. They simply don’t tolerate those who practice opt-out bulk email.
    Neither do I.

  • Interesting conversation…. I think when you have a blindfold on, you can’t see what’s in front of you. Email cleaning or validation is not evil, as some have stated here. [the rest of this paragraph has been removed due to offensive language — laura]
    The company I work for owns registration paths. We get approximately 20k-30k new signups every day. Whether you like co-reg advertisements or not, this is still legitimate advertising where people are signing up and requesting to join, hence giving their permission to receive emails from us. That being said, I can assure you that 15-20% of the signups are of not valid or legitimate email addresses. This is just part of the game. When you are paying affiliates to run traffic for you via text, banners, email, etc., you get bogus info signups so the affiliates can make their commission. Before we send a single email to a new signup, we use a third party email cleaning and validation company, which is not listed in this article or comments, that does a great job for us. We run a file with them 4 times per month of all of the signups we received that week. Their scrub usually nets us a loss of 25% of the email data we provide. I can tell you that since we started using this service, our complaint ratios have gone from roughly 1-2% to virtually non-existent. It got to the point that our admins thought the feedback loops and abuse email address was not working. Our IP reputation and deliverability has also been improved to the extent that the majority of our email campaigns now end up in the inbox versus being rejected or ending up in the junk folder. Let’s also not forget the fact that using this service has improved our ROI by 80%+.
    Is email list cleaning and validation worthwhile and a valuable, legitimate service? In my book it is essential and a compulsatory step for anyone who collects or aggregates email data and then markets to it.

  • Louis, if your company is legit, and has such a stellar performance, why not disclose it? I’m sure some folks would love to know where they can hire such a service.
    As to the rest of it, Nancy has her head stuck so firmly in the sand that I am not even going to bother trying to explain why what her company does is bad, aside from one remark: email is different from direct mail because in email the recipient is bearing the costs. If you don’t understand that, you won’t understand the email channel, and as Huey so eloquently put it, you will remain in the spam channel.
    I look forward to your attempts to convince Spamhaus to see it your way. I can always use a grin.

By laura

Recent Posts


Follow Us