Troubleshooting delivery problems

Everyone has their own way of troubleshooting problems. I thought I would list out the steps I take when I’m trying to troubleshoot them.

• Clarify the problem. As a consultant, folks come to me asking me to help them solve their delivery problems. My first step is to get them to clarify what symptoms they’re seeing. Something happened to make them contact me, and that’s where we start. Questions at this stage include:
  • Is mail going to bulk?
  • Is mail being rejected?
  • Where is mail rejected?
  • Are open rates steady?
  • What mail is affected?
• Once you have the answers to the question, you have your problems statement. This is a one or two sentence summary that identifies the full issue. This is a step a lot of folks avoid doing, but a good problem statement drives the troubleshooting. Example problem statements:
  • Marketing email from our ESP goes to bulk at Gmail.
  • CRM email is seeing high numbers of temp failures at Yahoo.
  • Our domain is blocked by Filter provider.
• Identify what are likely causes of the problem. Filters are pretty specific, so you want to change things that will actually make the filters do something different. Not just apply random best practices you’ve found on a website or blog (even this one). Common techniques, like sending to engaged users or removing bounced addresses don’t work everywhere, so you may end up weeks down the line with no improvement to show for it.
  • “Improving engagement” only works if the problem is bulk foldering at the three big mailbox providers. It does nothing anywhere else.
  • “List hygiene” only works if the problem is your list isn’t bounce handled (which is never, if you’re using a real ESP).
• Implement specific changes to address the reason for the delivery problems. What was the problem and how do you back it out? “
  • Our sales dude decided to harvest addresses of a social networking site and add them to our newsletter list. We purged all those addresses (or all the addresses that clicked on a link that wasn’t unsubscribe) from our list.
  • There was a flaw in our data handling process and we reactivated addresses that had bounced off in the past We fixed the flaw and have removed the previously bounced addresses.
  • We changed our frequency and ended up sending too much mail, causing recipients to report the mail as spam. We backed down to our old volume and are being more selective about who gets the new, higher volume mailings.
  • Some of our sales folks have not been abiding by company directives and are using addresses they’ve ‘saved’ from previous campaigns but that should be suppressed. We’ve implemented technical steps to prevent them sending mail to these addresses again.

All of these are actual solutions clients have implemented over the years. They’re all specific and required a clear understanding of mail processes and data flow. Once we identified the problems, the solutions fell out the end, really.

Too often, though, delivery folks don’t actually ask the right question and they don’t actually take the time to identify the problem. Instead, they “implement best practices” and try and make random changes hoping something will change. Some of the time it works, often the best practice change will randomly hit on the underlying cause of the problem. But when the random best practice changes don’t work, you absolutely need to step back and start from the beginning.