Recent Posts

My holiday email prediction

I was on IRC with a group of ESP delivery specialists last week and one of them was looking for something to blog about. I suggested a list of holiday predictions. Not that I have a huge number of holiday predictions, but I did come up with one.
During the holiday season at least one retailer will decide that they have information so important that they will ignore my opt-0ut request and add me to their holiday blast list.
So what’s your holiday email prediction?

Yahoo problems

I’m seeing scattered reports today that a lot of places are seeing backed up queues to Yahoo. They’ve had some problems over the last few days and seem to be still recovering. It’s looking like it’s something internal to Yahoo. One set of error messages I’ve seen reported by numerous people is: “451
4.3.2 Internal error reading data.”
It’s not you, and it’s not spam related. But it is putting a crimp in a lot of companies attempts to send lots of email ahead of black friday and cyber monday.

Changing the email client

We’re in the thick of hiring and next week is Thanksgiving, so blogging is going to be very light for the next two weeks.
One thing I have noticed is that lately there are attempts to “change how people interact with email.” Google released their Inbox product. And today I saw a post about an IBM attempt to change email and how people use it as a tool.
I find as I juggle more and more incoming email that most email clients just don’t cope with the whole process well. For a long time I could use my inbox as a todo list and manage what needed to be done. With the company growing and changing, an inbox todo list is just not as workable as it used to be. Maybe the Verse client from IBM is one solution.
I’m glad people are looking at how to improve the email client. Fundamentally, the client I’m using now is not that much different than the GUI client I was using at MAPS back in 2000 and 2001. Sure, it’s visually different, but the functionality isn’t much different.
A few years ago I blogged that people should look at building new email interfaces. I’m glad that some companies are actually looking at the interface and rethinking how people interact with email. Who knows, maybe we’ll end up with some specialized clients that are featured around getting work done by email and other clients focused around a more casual use of email, like shopping and networking.

Cloud sending with Momentum from MessageSystems

Earlier this week MessageSystems announced a new cloud platform, SparkPost, letting smaller companies have access to the power of the MessageSystems’s Momentum platform.
MessageSystems announced this at their user conference in San Diego. There was a lot of great information from ISPs and Momentum customers presented at the conference. If you get a chance check out the conference tweet stream (#msusercon) and the tweets by their director of Industry Relations Len Schneyder.
Now everyone can use the Momentum engine to send mail and take advantage of the features designed for large companies to communication with their millions of customers.

STARTTLS and misplaced outrage

About a month ago someone posted a heavily elided screenshot that they claimed was evidence of their ISP, AT&T, sabotaging SMTP connections being sent over their network, meaning that anyone could sniff their passwords and traffic.
This is it:

Most email people looking at that saw the asterisks in the banner and went “Oh. That’s not the ISP tampering with the traffic, the person running the mailserver doesn’t know how to configure their PIX firewall.”
It’s a very, very, very, well known issue.
But some groups who should know better, such as Ars Technica and the EFF, don’t seem to understand – even when they know about PIX fixup – that this isn’t tampering by intermediate ISPs, it’s just the operator of the mailserver in question not knowing how to configure his firewall. And it’s not a general attempt by consumer ISPs to “tamper with email encryption”, it’s just the operator of one mailserver not knowing how to configure his firewall.
PIX is a simple NAT/firewall appliance from Cisco. It’s a reasonable firewall, but it has some quirks. One of them is it’s “MailGuard” or “SMTP fixup” feature. When that’s turned on, it intercepts SMTP traffic and “sanitizes” it, to protect the mailserver from hostile traffic. To do this, it does a couple of things. One is that it blocks any attempt at sending a command that’s not one of the bare basic SMTP commands, by intercepting them and rejecting them with the error “502 5.5.2 Error: command not recognized”. The other is that it hides the software that’s running on the mailserver, removing any mention of it from the banner string sent when you connect. In fact, it replaces any character other than “2” or “0” with an asterisk.
I had an old PIX that I’ve not used in years, so I thought I’d set it up to show you. Here it is, being guarded by Freddy Chimpenheimer.

I set it up as though it was protecting our mailserver.
Here’s what happens when I connect to the mailserver with the PIX configured correctly:

And here’s what happens when I configure the PIX to use “fixup protocol smtp 25” and try and connect to the mailserver again:

Looks pretty similar to the “ISP tampering with the traffic” screenshot this all started with. I’m using an older PIX firmware image (I really didn’t want to spend the time and money to upgrade my PIX) so it errors out on EHLO, rather than just on STARTTLS. And because this old firmware doesn’t support EHLO, you also don’t see it using “XXX” to block out the string “STARTTLS” in the response to EHLO – the line in the original that says “250-XXXXXXXXA” said “250-STARTTLSA” before the PIX censored it.
Now I have those screenshots I’m going to disconnect my PIX and put it back in the pile of spare networking gear.
So the whole issue is just a mailserver operator who has a badly misconfigured firewall in front of his mailserver, nothing more.

Email problems are costly

Last week Zulily released their quarterly earnings. Their earnings’ report was disappointing, resulting in a drop in their stock prices. The chairman of the company told reporters on a conference call that part of the reason for the drop in earnings were due to deliverability problems “at a large ISP.”

URL reputation and shorteners

A bit of a throwback post from Steve a few years ago. The problem has gotten a little better as some shortening companies are actually disabling spammed URLs, and blocking URLs with problematic content. I still don’t recommend using a public URL shortener in email messages, though.
Any time you put a URL in mail you send out, you’re sharing the reputation of everyone who uses URLs with that hostname. So if other people send unwanted email that has the same URL in it that can cause your mail to be blocked or sent to the bulk folder.
That has a bunch of implications. If you run an affiliate programme where your affiliates use your URLs then spam sent by your affiliates can cause your (clean, opt-in, transactional) email to be treated as spam. If you send a newsletter with advertisers URLs in it then bad behaviour by other senders with the same advertisers can cause your email to be spam foldered. And, as we discussed yesterday, if spammers use the same URL shortener you do, that can cause your mail to be marked as spam.
Even if the hostname you use for your URLs is unique to you, if it resolves to the same IP address as a URL that’s being used in spam, that can cause delivery problems for you.
What does this mean when it comes to using URL shorteners (such as bit.ly, tinyurl.com, etc.) in email you send out? That depends on why you’re using those URL shorteners.
The URLs in the text/html parts of my message are big and ugly
Unless the URL you’re using is, itself, part of your brand identity then you really don’t need to make the URL in the HTML part of the message visible at all. Instead of using ‘<a href=”long_ugly_url”> long_ugly_url </a>’ or ‘<a href=”shortened_url”> shortened_url </a>’ use ‘<a href=”long_ugly_url”> friendly phrase </a>’.
(Whatever you do, don’t use ‘<a href=”long_ugly_url”> different_url </a>’, though – that leads to you falling foul of phishing filters).
The URLs in the text/plain parts of my message are big and ugly
The best solution is to fix your web application so that the URLs are smaller and prettier. That will make you seem less dated and clunky both when you send email, and when your users copy and paste links to your site via email or IM or twitter or whatever. “Cool” or “friendly” URLs are great for a lot of reasons, and this is just one. Tim Berners-Lee has some good thoughts on this, and AListApart has two good articles on how to implement them.
If you can’t do that, then using your own, branded URL shortener is the next best thing. Your domain is part of your brand – you don’t want to hide it.
I want to use a catchy URL shortener to enhance my brand
That’s quite a good reason. But if you’re doing that, you’re probably planning to use your own domain for your URL shortener (Google uses goo.gl, Word to the Wise use wttw.me, etc). That will avoid many of the problems with using a generic URL shortener, whether you implement it yourself or use a third party service to run it.
I want to hide the destination URL from recipients and spam filters
Then you’re probably spamming. Stop doing that.
I want to be able to track clicks on the link, using bit.ly’s neat click track reporting
Bit.ly does have pretty slick reporting. But it’s very weak compared to even the most basic clickthrough reporting an ESP offers. An ESP can tell you not just how many clicks you got on a link, but also which recipients clicked and how many clicks there were for all the links in a particular email or email campaign, and how that correlates with “opens” (however you define that).
So bit.ly’s tracking is great if you’re doing ad-hoc posts to twitter, but if you’re sending bulk email you (or your ESP) can do so much better.
I want people to have a short URL to share on twitter
Almost all twitter clients will abbreviate a URL using some URL shortener automatically if it’s long. Unless you’re planning on using your own branded URL shortener, using someone else’s will just hide your brand. It’s all probably going to get rewritten as t.co/UgLy in the tweet itself anyway.
If your ESP offers their own URL shortener, integrating into their reporting system for URLs in email or on twitter that’s great – they’ll be policing users of that just the same as users of their email service, so you’re unlikely to be sharing it with bad spammers for long enough to matter.
All the cool kids are using bit.ly, so I need to to look cool
This one I can’t help with. You’ll need to decide whether bit.ly links really look cool to your recipient demographic (Spoiler: probably not) and, if so, whether it’s worth the delivery problems they risk causing.
And, remember, your domain is part of your brand. If you’re hiding your domain, you’re hiding your branding.
So… I really do need a URL shortener. Now what?
It’s cheap and easy to register a domain for just your own use as a URL shortener. Simply by having your own domain, you avoid most of the problems. You can run a URL shortener yourself – there are a bunch of freely available packages to do it, or it’s only a few hours work for a developer to create from scratch.
Or you can use a third-party provider to run it for you. (Using a third-party provider does mean that you’re sharing the same IP address as other URL shorteners – but everyone you’re sharing with are probably people like you, running a private URL shortener, so the risk is much, much smaller than using a freely available public URL shortener service.)
These are fairly simple fixes for a problem that’s here today, and is going to get worse in the future.

October 2014 – The Month in Email

October was action-packed at WttW. We wrapped up some big and interesting client projects (look for some case studies soon!), attended another great M³AAWG conference, and made an exciting announcement that we’re hiring a deliverability specialist. The combination of these frees up some more of my time for blogging, which I’ve really missed. Look for more from me in November and December.

The best time to send email

This subject comes up over and over again. Many senders are convinced that there is a best time to send email. Countless research hours have been dedicated to finding that best time to send email. Numerous blog posts discuss what the best time to send email is.
From my perspective, there are better places for senders to spend time than figuring out what the exact right time is.But, senders still ask when the best time to send mail is.
There are a lot of reasons I can come up with as to why there’s no best time to send email. But the really big one is that when you send a mail has no impact on when it gets delivered. There are multiple steps between hitting the send button and the mail being delivered to the inbox totally outside the control of the individual sender.
Email is designed as “store-and-forward.” This means there are potential delays at multiple steps inside the process.
Sending queues are called queues for a reason. Emails are sent out individually, particularly when an ESP uses VERP as part of its sending. There is actually a time overhead for making a connection to a recipient server and sending the email.
Receivers have queues, too. They can only accept so many incoming connections at a time. They have limited resources to accept all the mail their users want.
Receivers may delay mail between accepting it at the MX and delivering it to the inbox. This isn’t ideal and it’s not usual, but it can happen.
Recipients using IMAP accounts may not check mail regularly. They may only collect mail a few times a day.
These are only a few of the reasons that send time doesn’t necessarily equate with delivery time. Of course, 99% of the time email is mostly instantaneous. The internet is robust enough that a message sent is delivered seconds later. I see it happen all the time, when colleagues and I send email during calls. But, when mail fails, it sometimes fails spectacularly. Back in the dark ages (of the early 90s) I had an email that took almost a year to get to the recipients. Best I can tell, it got stuck somewhere in the depths of a machine in the middle of the university mail system. Eventually that system fell over and someone noticed and rebooted it (maybe it was walled up somewhere?). The reboot shook my message out of where ever it was stuck.

The long tail of domains

I frequently get clients telling me that they have about 15 (20, 30) major domains on their list, and then a long tail of domains with only a couple of recipients. If you sort simply by the left hand side of the @, that’s true.
When you’re sending email, it’s not just the domain in the email address that is important. Of equal importance is the MX. The MX is what actually handles the mail and where many filters are applied. Sorting by MX, instead of simply recipient domain, can identify that most of your small business clients are hosted at a particular provider. The number of subscribers behind that filter may be enough to push that filter into your top 10 or even top 5 recipient domains.
There’s a much smaller tail when grouping recipients by MX domain. It makes it much easier to understand where blocks are happening. I have even seen cases where clients didn’t realize they were blocked at a commercial provider because they only saw the “onesie twosie” domains as undeliverable. They missed a real problem with blocking because they were looking at the wrong data.
I sometimes get the side eye from some ISP folks if I use the term receiver (because, well, they’re senders as much as they are receivers). But I use receiver to help distinguish between the recipient domain and the actual domain handling the email.
When was the last time you looked at your delivery by filter or MX rather than by recipient domain? What did you find?