Superstition, correlation and reality

I’m not a huge baseball fan, probably a side effect of growing up in a city with no MLB team. SF_giants_ImageBut I do enjoy the social aspects of rooting for local teams when they’re winning big games. Last night I was following the World Series score online and switched over to watch the last inning. I posted something about the game on FB just about 30 seconds before the Giant’s outfield bobbled what should have been a single (at best). I immediately posted an apology, “Sorry about that, shouldn’t have said anything!”
Do I really think that my post somehow cursed two outfielders and caused them to bobble a simple play? No, of course not. But it is a very human response. In fact, there’s an entire advertising campaign centered around the the weird things people do while watching sports.
There is a lot of superstition in email delivery, too. I think that’s a combination of filtering necessarily being a black box, human’s built in tendency to see patterns in random data, and a need to be able to control and affect outcomes.
Figuring out cause and effect in the real world is not trivial. In my research days we set out to control as many confounding factors as possible so we could demonstrate the cause and the effect. That’s really hard to do when you’re not at a lab bench. In the real world, we can’t always control things directly. Instead, we have to rely on statistics and representative (or non-representative) samples.
Delivery isn’t even close to a science and one of the major issues is that filters are always changing. I’ve certainly seen occasions where multiple clients, or colleagues, were having problems delivering to one ISP or another. One of my clients made a change and saw their delivery improve. They patted themselves on the back for figuring out the problem. At the same time, though, other folks saw their delivery improve without making any changes. I can’t always convince people that whatever they did had nothing to do with their delivery improving.
The flip side is I can’t always convince people to stop doing somethings that they don’t need to do. I see a lot of mail with both DomainKeys and DKIM signatures. In most cases both signatures have the same selectors. DomainKeys is deprecated. No one, and I mean no one with a modern email system, is checking DomainKeys without checking DKIM. Senders can safely stop signing with DomainKeys and have nothing happen. It doesn’t matter, lots of ESPs and sender sign with both. They’re not going to change it. I’ve had multiple groups tell me they’re afraid to stop signing because it might hurt their delivery.
The reality is I didn’t make the Giant’s outfield bobble the ball because I posted to FB that I was watching the bottom of the 9th inning. The reality is that DomainKeys is deprecated and there’s no benefit to signing with both DomainKeys and DKIM. The reality is we are humans and we are inherently superstitious. Most of the times our superstitions are harmless. But sometimes they cause us more work than we need to do and provide no tangible benefits.

Disposable addresses
The long tail of domains

Related Posts

Incorrect rejection messages

At least one ESP and Spamhaus are currently investigating bounce messages at a couple ISPs incorrectly pointing to Spamhaus as the reason for the block. The bounce messages are taking the form:

Read More

Alice and Bob Sign Messages

Alice and Bob can send messages privately via a nosy postman, but how does Bob know that a message he receives is really from Alice, rather than from the postman pretending to be Alice?
If they’re using symmetric-key encryption, and Bob is sure that he was talking to Alice when they exchanged keys, then he already knows that the mail is from Alice – as only he and Alice have the keys that are used to encrypt and decrypt messages, so if Bob can decrypt the message, he knows that either he or Alice encrypted it. But that’s not always possible, especially if Alice and Bob haven’t met.
Alice’s shopping list is longer for signing messages than for encrypting them (and the cryptography to real world metaphors more strained). She buys some identical keys, and matching padlocks, some glue and a camera. The camera isn’t a great camera – funhouse mirror lens, bad instagram filters, 1970s era polaroid film – so if you take a photo of a message you can’t read the message from the photo. Bob also buys an identical camera.
sign1
Alice takes a photo of the message.
sign2
Then Alice glues the photo to one of her padlocks.
sign3
Alice sends the message, and the padlock-glued-to-photo to Bob.
sign4
Bob sees that the message claims to come from Alice, so he asks Alice for her key.
sign5
(If you’re paying attention, you’ll see a problem with this step…)
Bob uses Alice’s key to open the padlock. It opens (and, to keep things simple, breaks).
Bob then takes a photo of the message with his camera, and compares it with the one glued to the padlock. It’s identical.
sign6
Because Alice’s key opens the padlock, Bob knows that the padlock came from Alice. Because the photo is attached to the padlock, he knows that the attached photo came from Alice. And because the photo Bob took of the message is identical to the attached photo, Bob knows that the message came from Alice.
This is how real world public-key authentication is often done.

Read More

DKIM and DomainKeys, Spam and Ham

I’ve been preaching “DKIM is great! DomainKeys is obsolete, get rid of it!” for several years now. I thought I’d take a look at my mailbox and see who was using authentication.
I’ve divided this into “Ham” and “Spam”. Spam is, well, all the spam I’ve received over the past couple of years. Ham is the non-spam mail in my inbox, whether personal, business, bulk or transactional. I’ve excluded most of the discussion mailing lists I’m on (not least because many of them consist of people in the email industry or are email standards development mailing lists, so have email authentication levels that are way outside the norm).

Read More