A number of senders are reporting that they’re getting unusual responses from AOL servers. The responses include:
421 mtain-dk10.r1000.mx.aol.com Service unavailable – try again later
554 mtain-dk03.r1000.mx.aol.com ESMTP not accepting connections
mtain-dk* are assigned reserved IP addresses. It looks like something broke inside AOL again, and lots of places are having trouble delivering mail there.
I’m hearing from lots of folks that they’re seeing some improvement in delivery to AOL accounts.
As everyone can imagine, the AOL situation has been a common thread of discussion on many delivery lists. One person even commented at how fragile the AOL mail server seems. My own thoughts are a little different. The AOL mail system is notoriously complex and integrated. Many of the folks who built it have been laid off or otherwise moved on to other companies. I know there are still smart, competent people riding herd on the AOL mail servers, but I expect they don’t have the resources to do the ongoing maintenance and the fire fighting and all the other tasks that a mailserver handling billions of emails needs.
What this means is that the AOL mail system has been suffering from bit rot for at least 2 years. It is to the original designers’ credit that it’s taken this long before there were major problems like we’ve seen over the last week.
Sometime last night, AOL managed to delete their MX records, causing mail to hard bounce for at least 3 hours, possibly more. Annalivia noticed, contacted the NOC, appropriate people were paged and the records are now functional again.
This morning AOL seems to be having more mail problems, possibly related to everyone retrying mail that was hard bounced last night after the MX record was deleted. Or the company is just finally showing the consequences of laying off so many people last year.
I think the most worrying bit about this is that the AOL NOC didn’t notice there was no mail coming in for 3 hours. I don’t get mail for an hour and I start checking to see if the mailserver has fallen over. I can’t believe no one noticed no incoming mail for 3 hours.
I suggest that anyone who had AOL bounces last night package those up and resend today. But don’t send them all at once, trickle them out over the course of the day. Remember, everyone else is trying to send their mail, too. And AOL is not having a happy day.
UPDATE: The Return Path Received blog points out some of the reasons some of you might still be seeing AOL mail fail. The fix is to flush your DNS cache or reboot your DNS server.
Some news stories and links today.
Spamhaus has announced their new domain block list (DBL). The DBL is a list of domains that have been found in spam.