Troubleshooting: the questions

In my post earlier this week, katie asks:

what do you do next when the problem statement is as non-specific as “open rates are falling”? how would you go about getting from there to that next level of marketing email from our ESP goes to bulk?

That’s a great question, and will help me explain pieces I didn’t in the initial post.

The observation here is: “open rates are falling.”

We know that open rates are what happens when someone loads a 1×1 pixel in their mail client. There are delivery reasons why this will fail and there are non-delivery reasons why this will fail. We want to figure out why pixels aren’t loading.

What are common reasons pixel loads fail?

1. This message is too long and gmail is cutting off the 1×1 pixel.
2. There is something wrong with our tracking server
3. The mail isn’t making it to the inbox
4. The mail isn’t interesting enough for folks to open

(I really need a flow chart to make this pretty, but I’ll write it out here for now. Basically, I ask a question and then take next steps based on the answer. The questions are sorta ordered, but they don’t have to be asked and answered in this order).

Where are open rates falling? Is this happening everywhere or just at a few places? The answer here gives me another pathway to follow.

• If, for instance, it’s only gmail then maybe this is just a longer message than normal and it’s being truncated. Not a delivery problem, but we should do a test and see if gmail is truncating the message.
• If, for instance, it’s across all my recipients and it’s not isolated to a specific domain, then maybe my tracking server fell over. Not a delivery problem, but we should go talk to the web folks.
• If, for instance, it’s at Yahoo, some cable companies, and AOL, then that may be a problem at the Yahoo domains. Let’s look deeper into those domains.

Are the open rates at the affected domains zero or just smaller than normal?

• An open rate of zero suggests mail may be fully blocked or going to spam.
  • We can look at our SMTP logs to see if there are active blocks visible
  • We can do some tests to our own freemail accounts to see where mail is going for those addresses
• A lower open rate suggests some mail is going to bulk.
  • Do we have an inbox monitoring tool available? Let’s add that to the next mailing so we can see where mail is delivered.
  • Can we run some tests to our own mailboxes to see where mail is delivered?

The idea here is we’re trying to determine what pathway to go down. Was this is something technical that caused the reporting to be prevented the pixel from loading and caused a false lower reading? Or was there something about the message that caused fewer recipients to actually open it? Or

The next set of questions aren’t so much about troubleshooting, but about other things I’d think about.

• Is there any deviation from normal related to the mail client?
• Could there be something about this template that corrupted the subject line and caused folks not to open it?
• Are my tests going to the inbox?
• Did I put any new domains or content in the email?
• Was there a DNS problem that caused a temporary failure in authentication?

There are a multitude of reasons that open rates may fall. Fixing delivery problems, be they blocks or spamfoldering, can take extensive work and time to resolve. Before jumping too the conclusion that delivery is poor, figure out if there are other, easier to solve reasons, that explain a low open rate.