We often want to know whether two hostnames are controlled by the same person, or not.
One case for that is cookie privacy in web browsers. We want pages at www.blighty.com and images.blighty.com and blighty.com to all be able to set and read cookies for each other – so a user only needs to log in once for pages or images on all of them to work well together. So we allow all of them to access cookies for “*.blighty.com”.
But we don’t want blighty.com and example.com to be able to access each other’s cookies. (Both for privacy reasons and for security, so a hostile page at example.com can’t steal authentication cookies for blighty.com from the user).
Loosely, we want to be able to say that images.blighty.com and www.blighty.com are sort of the same, while images.example.com and www.blighty.com aren’t.
At first that looks pretty simple to do, especially if you’re in the US. Two hostnames are the same if they’re in the same domain. The relevant domains here are obviously blighty.com and example.com. And there’s an obvious way to find the domain: it’s the last two words in the hostname, sometimes called the second level domain and top level domain.
For www.blighty.com the second-level domain would be “blighty”, the top-level domain would be “com” and the domain would be “blighty.com”. We can say that any hostname that ends in “.blighty.com” is in that domain, and reasonably assume that they’re controlled by the same person.
But … if we’re looking at www.blighty.co.uk then the domain is “blighty.co.uk”. Our simple algorithm to find the “domain” doesn’t work.
You’d think that somebody would have thought about this when designing the domain name system, wouldn’t you? But no. They didn’t.
In DNS there’s not really any such thing as a “domain” or a “subdomain”. It’s all hostnames. “www.blighty.com”? Hostname. “blighty.com”? Hostname. “com”? Hostname. “.”? Hostname. And DNS doesn’t provide any information about common ownership between hostnames, at all.
So if we want to find the “domain” for a hostname we’re going to have to come up with some other way of doing it. One obvious approach would be to manually maintain a list of all the possible “top level domains” under which people can register a domain. That’d be quite ridiculous and excrutiatingly painful to maintain. So that’s what we did.
The Public Suffix List is a list of over 8,000 “top level domains”. It has both real ones like “.com”, “.co.uk”, “.zippo” and “. k12.al.us” under which people can register domains, but also domains under which many independent customers can use their own subdomains such as “herokuapp.com” or “blogspot.com”. Using this list you can finally define a unique “domain” part of any hostname. That’ll often be the same as the intuitive idea of a domain – “blighty.com”, “losaltos.k12.ca.us” or “natwest.co.uk”. Sometimes it won’t be – “lecreuset.us.com”, “myapp.herokuapp.com” or “mynas.diskstation.me” – but in those cases it’s a better description of the subdomains that are under the control of a single user.
The public suffix list is included in every web browser, to handle cookie security. It’s useful for other things too. We use it in several of our internal data-clustering tools to canonicalize URLs and MXes. And it’s a critical part of DMARC.
One of the first new concepts DMARC exposes you to is “aligned domains”.
“DMARC passes if either the message is validly DKIM signed and the DKIM d= domain aligns with the domain in the From: field, or if the message passes SPF with a domain that aligns with the domain in the From: field.”
DMARC defines two different sorts of alignment between domains. The less interesting one is “strict alignment”, communicated via adkim=s or aspf=s fields in the DMARC record. Strict alignment just means the domains are identical. The much more interesting one is “relaxed alignment”, communicated via adkim=r or aspf=r. With relaxed alignment then two domains are aligned if they’re in the same Organizational Domain – which is the extended, formalized definition of a domain as defined by the Public Suffix List.
You knew there was going to be something email-related eventually, right?
We’re in the thick of the busiest time of the year for email. It’s been so busy, in fact, that we’ve seen some slowdowns and delivery issues across the email universe. It may be worth thinking about alternate strategies for end of year promotions beyond Black Friday and Cyber Monday.
I was delighted to chat with Julia Angwin for her ProPublica piece on subscription bombing and abuse prevention. Her piece is a good introduction to the topic, and very much worth reading.
ICYMI, I did a rough analysis of the data from our survey on Google Postmaster Tools. Stay tuned for more insights when I have a moment to explore this further.
I’ve written extensively about unsolicited B2B email, and how frustrating it is to get these messages. As a sender, if you’re reaching out to people you don’t know, you can mitigate this frustration with a few best practices.
Some major industry news this month with the Proofpoint acquisition of Cloudmark and the Sendgrid IPO. Congrats to all involved.
Steve wrote a really terrific post about interacting in online communities. So much of the work we do depends on our relationships with colleagues, and it’s good to remind ourselves of the best practices for maintaining those relationships. My post on “the blighty flag” is a great example of how relationships work in our industry.
Hope you all have happy holidays. I’m taking off blogging until January. See you in the new year!
From our first Christmas with the kittens.
There are no lights because someone fell out of the tree and dragged them all off.
[#INFOGRAPHIC] Email marketing trends 2018
It’s always an honor to be asked to provide quotes and thoughts with experts in the field. Sometimes the day to day gives me tunnel vision, but things like this give me the opportunity to think more globally. Hands down, though, the best part is seeing the final product and hearing what other folks have to say.
Go check out the full infographic.
A security researcher has identified a rendering flaw that allows for “perfect” phishing emails. From his website:
Mailsploit is a collection of bugs in email clients that allow effective sender spoofing and code injection attacks. The spoofing is not detected by Mail Transfer Agents (MTA) aka email servers, therefore circumventing spoofing protection mechanisms such as DMARC (DKIM/SPF) or spam filters. Mailsploit website
While this is a bit of a problem it’s mostly a problem with the email client (MUA) not the email servers involved.
The short version is that an attacker creates an email address in a domain they own. The address includes a null value encrypted in the local part of the email address. When the email client get a hold of the address it displays the address up to the null value and drops everything after that.
Due to a rendering issue different systems end up displaying simply From: firstname.lastname@example.org.
Mail displayed on clients vulnerable to this exploit will be DMARC authenticated for a domain that is different than the domain displayed to the user.
The big problem here is in the email client and how they display to the user. While this is creative, it’s not that much different than using “POTUS <email@example.com>”. Display names are a problem, but they’re a problem that has to be addressed by individual mail clients. The choice to display only the comment is a problem.
Maybe this exploit will motivate email client maintainers to rethink their decisions on what to display to users. Their current choices and implementations are vulnerable and need to be improved.
It is increasingly clear that successful email marketing programs measure and emphasize deliverability. No longer is deliverability the crisis management team called when everything breaks. They’re part and parcel of an effective email marketing team.
Today I watched a bit of the EIS livestream where acquisition marketers were discussing their processes. Everyone of them talked about things that are critical for deliverability as core to their business.
- Engagement is key. Sending mail to folks who want the mail and who will interact with the mail is vital for delivery success.
- Focus on quality of addresses not the quantity of addresses. Sending mail to every address that shows up is a recipe for failure.
- Each address represents a person. An address itself is a bunch of electrons, the value of an address is the person who reads it.
These are all delivery fundamentals.
It’s nice to hear the marketers are starting to understand how important deliverability is.
Today on MailOp it was announced that the migration of Microsoft freemail domains to the office 365 backend. Over the next week the mx*.hotmail.com mail servers will stop working. Check your settings, folks, and make sure you’re correctly querying DNS before sending.
Vodafone NZ is shutting down mail handling for the following domains as of today, Nov 30, 2017.
According to their website, this is primarily due to an elderly platform that’s not meeting the needs of their users, including long mail delays and too much spam. The FAQ they provided about this says that users who have forwarding accounts will still receive email, for a while. But it also says that they’re going to be rejecting mail for users.
In any case, unless the user has forwarding set up for some of those addresses, then they will not be receiving any messages. Vodafone has removed all ability for users to log into their webmail accounts. Because these are forwarded messages, some domains using DMARC may see an increase in bounces even when the mail is accepted by Vodafone.
If you have a significant number of users still at these domains, now is the time to break out your non-email contact methods to get them to update their email address with you.
(Thanks to Nancy Harris for bringing this to my attention and spreading the word.)
A question came up on the Women of Email Facebook page about sending cold B2B emails. This is one of those areas I have strong opinions about, mostly because I am so tired of getting deceptive and unending messages from folks.
Realistically, cold emailing isn’t going to stop just because recipients hate receiving it. We haven’t wiped out spam in 20+ years, we’re not going to manage it for this one tiny piece. But I do think there are things senders can do to minimize the amount of frustration their spam creates.
The fake stickers printed on the envelope just really make this over the top deceptive.
Here are my top things to not do when sending cold emails to business contacts.
- One and done. Do not keep sending “followups” or “circle backs” or whatever.
- Don’t pretend you’re sending a hand crafted email when you’re sending hundreds of identical messages.
- Scraping details off of LinkedIn and stuffing them into templates doesn’t make a message personalized.
- Send through your actual mailserver with your corporate domain in the from address; don’t use a gmail address.
- Follow CAN SPAM. Really, it’s not hard.
- No answer is an answer. Silence doesn’t mean your message wasn’t received. It can mean you’re being ignored.
- Don’t ask me to give you the names of other folks in my company. I am perfectly able to forward messages without leaking employee data.
- Don’t use the various bits of automation software that lets you automate spamming through Google.
- Do a little research and see if the company you’re sending to is actually a good match for you. I am not a good target for your lead selling business, for example.
Overall, just respect your recipients’ time. Don’t do the email equivalent of printing envelopes with fake stickers to get me to open the message.
Wow! Congrats to all the senders out there for sending So Much Volume that mail servers are full. I’ve even seen reports that STARTTLS connections are taking multiple seconds to establish at Gmail. The volume of mail that it takes to make Google slow down is impressive.
Of course, Gmail isn’t the only system exhibiting slow downs. Other major consumer webmail providers are also showing signs their servers are under heavy load. I’m seeing reports about both AOL and Microsoft accepting mail slowly. Oddly enough, I’ve not seen anything about Yahoo having issues. Maybe folks just never use yahoo.com addresses any more.
There may not be a fix for this. It is very possible receiving systems just do not have the capacity to handle the volume of mail folks want to send today. If senders have, collectively, decided to send more mail than max capacity there isn’t much that can be done. Maybe some very forward thinking ISPs have spare servers they can deploy, but it’s unlikely.
No major advice here, just a warning that receivers may not be able to access all the mail that’s currently being shoved at them. Nothing to do except retry, and perhaps hold off some “less urgent” sends until after normal business hours. Those of you who are sending Cyber Monday sales emails may just have to extend them to Tuesday in some cases.
EDIT: After I posted this, I saw problems with Yahoo (mail accepted but not making it to the inbox) and Earthlink as well.