Return Path released their global delivery report for the second half of 2009. To put together the report, they look at mail delivery to the Mailbox Monitor accounts at 131 different ISPs for 600,000+ sends. In the US, 20% of the email sent by Mailbox Monitor customers to Return Path seed accounts doesn’t make it to the inbox. In fact, 16% of the email just disappears.
I’ve blogged in the past about previous Return Path deliverability studies. The recommendations and comments in those previous posts still apply. Senders must pay attention to engagement, permission, complaints and other policy issues. But none of those things really explain why email is missing.
Why is so much mail disappearing? It doesn’t match with the philosophy of the ISPs. Most ISPs do their best to deliver email that they accept and I don’t really expect that ISPs are starting to hard block so many Return Path customers in the middle of a send. The real clue came looking at the Yahoo numbers. Yahoo is one of those ISPs that does not delete mail they have accepted, but does slow down senders. Other ISPs are following Yahoo’s lead and using temporary failures as a way to regulate and limit email sent by senders with poor to inadequate reputations. They aren’t blocking the senders outright, but they are issuing lots of 4xx “come back later” messages.
What is supposed to happen when an ISP issues a 4xx message during the SMTP transaction is that email should be queued and retried. Modern bulk MTAs (MessageSystems, Port25, Strongmail) allow senders to fine tune bounce handling, and designate how many times an email is retried, even allowing no retries on a temporary failure.
What if the missing mail is a result of senders aggressively handling 4xx messages? Some of the companies I’ve consulted for delete email addresses from mailing lists after 2 or 3 4xx responses. Other companies only retry for 12 – 24 hours and then the email is treated as hard bounced.
Return Path is reporting this as a delivery failure, and the tone of discussion I’m seeing seems to be blaming ISPs for overly aggressive spamfiltering. I don’t really think it’s entirely an ISP problem, though. I think it is indicative of poor practices on the part of senders. Not just the obvious permission and engagement issues that many senders deal with, but also poor policy on handling bounces. Perhaps the policy is fine, but the implementation doesn’t reflect the stated policy. Maybe they’re relying on defaults from their MTA vendor.
In any case, this is yet another example of how senders are in control of their delivery problems. Better bounce handling for temporary failures would lower the amount of email that never makes it to the ISP. This isn’t sufficient for 100% inbox placement, but if the email is never handed off to the ISP it is impossible for that email to make it to the inbox.
20% of email doesn't make it to the inbox
2
RSS - Posts
Thanks for posting this, Laura. We completely agree that senders are largely in control of deliverability. Hopefully the tone of our report and release were consistent with that theme — we don’t blame ISPs for overly aggressive filtering and counsel our clients not to, either.
Laura, I am a little curious about the Yahoo! Strategy. We will sometimes see one of our IP addresses start receiving a steady stream of 4xx errors, usually:
421 4.7.0 [TS01] Messages from X.X.X.X temporarily deferred due to user complaints – 4.16.55.1; see http://postmaster.yahoo.com/421-ts01.html
In our experience, this seems to be pretty much as good as a permanent failure. We only keep emails in our queues and retry messages for 72 hours on temp fails, and we do that on a 20min, 40, 80, 160, etc. Schedule. Again, once we start seeing these errors with Yahoo, they don’t seem to stop inside that window and as a result, all the messages perm fail with a timeout after 72 hours. We’ve tried slowing down our outbound rate and different retry schedules, but not with much success. Has anyone out there observed approximately how long it takes for yahoo to start accepting again once you’ve seen one of these messages?
For our clients that run into this issue, it is making it hard for us to work with them to better manage their lists and reputations. One of our clients that hit this issue just yesterday is a utility company sending notifications to their customers – not even a marketing message…
I think this depends on a company’s definition of an acceptable delay. For some people a delay of 24 hours might be fine, but others it could be considered a lost opportunity. A good example is an event reminder or announcement. In this case it is as good as a hard bounce.
PowertMTA has some really nice methods for handling the temp issues, like backing off connections and setting retry periods. From what is says on Yahoo’s site, they recommend trying again after four hours.
In some cases it seems like Yahoo creates delays for handling their own resource load, no matter what the sender practices are. I’d be interested to hear feedback from others on this. Yahoo is definitely an ISP that requires a lot of tweaking on the MTA.
Hi, Matt. You are correct, the tone of the discussion I was commenting on was not set by Return Path, it’s what I’m seeing and hearing elsewhere in the industry.
Once a sender has gotten to the point of a [TS01] message at Yahoo, it generally means that Yahoo has identified that the mail is not wanted by Yahoo customers. Instead of changing sending rates and retry schedules, I’d suggest you look at the underlying issue. What is the sender doing that is causing user complaints? Is it an issue with permission? Is the mail expected? How can my customer change their process to fix the lack of engagement with customers? Is this a customer that would benefit from certification?
I think some of the problem here actually has to do with sender expectations. Email, as a protocol, is not designed to be instantaneous. Practically, there is no way to guarantee that the email you send now gets in front of your recipient immediately. If nothing else, there is no guarantee that the recipient will be sitting in their chair reading mail at that very moment. Just because you send it, does not mean that the recipient will get it when you want them to. The recipient may be on vacation, or may be not checking email for other reasons, or may only check that email account once a week.
In any case, expecting email to be an instantaneous form of communication is a misunderstanding of how email works and unrealistic expectations of what email can do for you.
http://blog.wordtothewise.com/2009/03/email-is-store-and-forward/
I certainly agree on the timing – How many clients show up and say something like “I have a list of x million customers and I need to send to the entire list in 1 hour”. The flip side of course is that when you buy something at Amazon, or buy a plane ticket, you don’t expect more than a few minutes (at most) to go by before your confirmation is in your inbox, and that’s not the sender’s expectations 🙂
In general the real problem comes in when you are dealing with something like a 1 day sale, or some other type of limited time offer. A recipient is mad at the sender when they get the email the day after the sale or offer ended, but if the sale notification goes to a mailbox they didn’t check and they just didn’t see it in time, that set’s a completely different attitude towards the sender.
I think the unrealistic expectations tend to be more on the recipient side (hey, me included), and the desires of the senders to meet those expectations are simply reflected. The average email recipient wants the stuff they want ASAP, they don’t care about rate limiting, temp fails, the size of your list, etc. The real question is how do legitimate senders do that? 🙂
Good discussion!
In general the real problem comes in when you are dealing with something like a 1 day sale, or some other type of limited time offer.
I would argue email is the wrong way to advertise these types of sales. I know a lot of companies do it (the number of emails I get from a particular clothing retailer advertising their “weekend blowout” or “one day sale” is excessive. I also don’t bother to check their website most of the time because I know that come next weekend there will be another one. Because there is always another one.
The less snarky answer is that: you don’t send mail at 9 am advertising a 24 hour sale ending at 9am the next morning. You send mail on Tuesday advertising a 24 hour sale from Thursday 8 am to Friday 8 am. Give yourself a little time and give your recipient a heads up. You can then send a reminder “sale ends 8am Friday” late Wednesday or early Thursday.
Expecting instantaneous delivery is the problem here. And, yes, recipient expectations are part of the problem, too. But, the ISPs are pretty good at prioritizing email that recipients want like purchase notifications and plane tickets and hotel reservations. And, senders are the marketers, aren’t they the ones who are supposed to be good at modeling what consumers want?
[…] been an interesting discussion in the comments from yesterday’s post about temp failing. My position is that email is not a 100% reliable medium for transmitting time sensitive […]
Re. the comment about a purchase confirmation message from Amazon… Has anyone else noticed that these (Paypal as well) are plain-text messages? Does anyone have any evidence that plain-text messages will see better deliverability than HTML? It seems logical that they would but I don’t have any hard evidence.
RE: Yahoo TS-01 messages
This *does not* necessarily indicate that Yahoo has deemed the sending IP to have reputation problems. If you are experiencing TS-01 temp delays do examine your retry policy. That is the best advice I can give – based on a call with several ESPs and Yahoo Postmasters some months back.
Regards,
John Johnson