Related Posts

Just give it up already

I have a mail system totally separate from my inbox to use when I’m testing signup forms. Some of them are client, some of them are vendors my clients are thinking about using. In any case, it’s mail I’m seriously concerned won’t stop just by me opting out of it.
The server hosting that mail system has been flakey lately, and needs to be hard power cycled to make it come back. We had a major power glitch this morning and so ended up down at the colo and power cycled that box while we were there.
This box was last working February 4th. It’s been off the internet for almost 2 months now. It wasn’t answering on port 25. It was dead. No mail here. And, yet, a bunch of legitimate email marketers are still attempting to send those addresses mail.
Really. Dead for 2 months and the senders keep trying to mail to those addresses. The server came back about 2 1/2 hours ago. I already have 6 emails from two different senders.
Seriously. If you can’t deliver a mail to someone for TWO MONTHS just give it up already. I am sad that even companies that get the best advice I can give them still can’t get the simple things right.
And, really, don’t argue “but it came back! Clearly we should keep trying!” Yes, it came back. But in all the years I’ve had this disposable email system I have not opened a single image. I’ve not purchased a single thing. I’ve never shown any sign of life on any of those addresses. The mailserver has been down for months at a time. There is no value to continuing to send mail to those addresses. And, yet, people still do it.
Why? WHY!?

Read More

A Disturbing Trend

Over the last year or so we’ve been hearing some concerns about some of the blacklisting policies and decisions at Trend Micro / MAPS.
One common thread is that the ESP customers being listed aren’t the sort of sender who you’d expect to be a significant source of abuse. Real companies, gathering addresses from signup forms on their website. Not spammers who buy lists, or who harvest addresses, or who are generating high levels of complaints – rather legitimate senders who are, at worst, being a bit sloppy with their data management. When Trend blacklist an IP address due to a spamtrap hit from one of these customers the actions they are demanding before delisting seem out of proportion to the actual level of abuse seen – often requiring that the ESP terminate the customer or have the customer reconfirm the entire list.
“Reconfirming” means sending an opt-in challenge to every existing subscriber, and dropping any subscriber who doesn’t click on the confirmation link. It’s a very blunt tool. It will annoy the existing recipients and will usually lead to a lot of otherwise happy, engaged subscribers being removed from the mailing list. While reconfirmation can be a useful tool in cleaning up senders who have serious data integrity problems, it’s an overreaction in the case of a sender who doesn’t have any serious problems. “Proportionate punishment” issues aside, it often won’t do anything to improve the state of the email ecosystem. Rather than staying with their current ESP and doing some data hygiene work to fix their real problems, if any, they’re more likely to just move elsewhere. The ESP loses a customer, the sender keeps sending the same email.
If this were all that was going on, it would just mean that the MAPS blacklists are likely to block mail from senders who are sending mostly wanted email.
It’s worse than that, though.
The other thread is that we’re being told that Trend/MAPS are blocking IP addresses that only send confirmed, closed-loop opt-in email, due to spamtrap hits – and they’re not doing so accidentally, as they’re not removing those listings when told that those addresses only emit COI email. That’s something it’s hard to believe a serious blacklist would do, so we decided to dig down and look at what’s going on.
Trend/MAPS have registered upwards of 5,000 domains for use as spamtraps. Some of them are the sort of “fake” domain that people enter into a web form when they want a fake email address (“fakeaddressforyourlist.com”, “nonofyourbussiness.com”, “noneatall.com”). Some of them are the sort of domains that people will accidentally typo when entering an email address (“netvigattor.com”, “lettterbox.com”, “ahoo.es”). Some of them look like they were created automatically by flaky software or were taken from people obfuscating their email addresses to avoid spam (“notmenetvigator.com”, “nofuckinspamhotmail.com”, “nospamsprintnet.com”). And some are real domains that were used for real websites and email in the past, then acquired by Trend/MAPS (“networkembroidery.com”, “omeganetworking.com”, “sheratonforms.com”). And some are just inscrutable (“5b727e6575b89c827e8c9756076e9163.com” – it’s probably an MD5 hash of something, and is exactly the sort of domain you’d use when you wanted to be able to prove ownership after the fact, by knowing what it’s an MD5 hash of).
Some of these are good traps for detecting mail sent to old lists, but many of them (typos, fake addresses) are good traps for detecting mail sent to email addresses entered into web forms – in other words, for the sort of mail typically sent by opt-in mailers.
How are they listing sources of pure COI email, though? That’s simple – Trend/MAPS are taking email sent to the trap domains they own, then they’re clicking on the confirmation links in the email.
Yes. Really.
So if someone typos their email address in your signup form (“steve@netvigattor.com” instead of “steve@netvigator.com”) you’ll send a confirmation email to that address. Trend/MAPS will get that misdirected email, and may click on the confirmation link, and then you’ll “know” that it’s a legitimate, confirmed signup – because Trend/MAPS did confirm they wanted the email. Then at some later date, you’ll end up being blacklisted for sending that 100% COI email to a “MAPS spamtrap”. Then Trend/MAPS require you to reconfirm your entire list to get removed from their blacklist – despite the fact that it’s already COI email, and risking that Trend/MAPS may click on the confirmation links in that reconfirmation run, and blacklist you again based on the same “spamtrap hit” in the future.

Read More

20% of email doesn't make it to the inbox

Return Path released their global delivery report for the second half of 2009. To put together the report, they look at mail delivery to the Mailbox Monitor accounts at 131 different ISPs for 600,000+ sends. In the US, 20% of the email sent by Mailbox Monitor customers to Return Path seed accounts doesn’t make it to the inbox. In fact, 16% of the email just disappears.
I’ve blogged in the past about previous Return Path deliverability studies. The recommendations and comments in those previous posts still apply. Senders must pay attention to engagement, permission, complaints and other policy issues. But none of those things really explain why email is missing.
Why is so much mail disappearing? It doesn’t match with the philosophy of the ISPs. Most ISPs do their best to deliver email that they accept and I don’t really expect that ISPs are starting to hard block so many Return Path customers in the middle of a send. The real clue came looking at the Yahoo numbers. Yahoo is one of those ISPs that does not delete mail they have accepted, but does slow down senders. Other ISPs are following Yahoo’s lead and using temporary failures as a way to regulate and limit email sent by senders with poor to inadequate reputations. They aren’t blocking the senders outright, but they are issuing lots of 4xx “come back later” messages.
What is supposed to happen when an ISP issues a 4xx message during the SMTP transaction is that email should be queued and retried. Modern bulk MTAs (MessageSystems, Port25, Strongmail) allow senders to fine tune bounce handling, and designate how many times an email is retried, even allowing no retries on a temporary failure.
What if the missing mail is a result of senders aggressively handling 4xx messages? Some of the companies I’ve consulted for delete email addresses from mailing lists after 2 or 3 4xx responses. Other companies only retry for 12 – 24 hours and then the email is treated as hard bounced.
Return Path is reporting this as a delivery failure, and the tone of discussion I’m seeing seems to be blaming ISPs for overly aggressive spamfiltering. I don’t really think it’s entirely an ISP problem, though. I think it is indicative of poor practices on the part of senders. Not just the obvious permission and engagement issues that many senders deal with, but also poor policy on handling bounces. Perhaps the policy is fine, but the implementation doesn’t reflect the stated policy. Maybe they’re relying on defaults from their MTA vendor.
In any case, this is yet another example of how senders are in control of their delivery problems. Better bounce handling for temporary failures would lower the amount of email that never makes it to the ISP. This isn’t sufficient for 100% inbox placement, but if the email is never handed off to the ISP it is impossible for that email to make it to the inbox.

Read More