Is your website up? Are you sure?

“What would you do for 25% more sales?”
It’s panicked gift-buying season, and I got mail this morning from Boutique Academia, part of their final push before Christmas.
Inbox__18_975_messages__26_unread_
They’re hoping for some Christmas sales in the next three days. They do make some lovely jewelry – ask Laura about her necklace some time – so I clicked on their mail.
Failed_to_open_page
That’s not good. I like Boutique Academia, and fixing email and dns problems is What We Do, so I took a look.
Safari isn’t quite as bad with not-exactly-truthful error messages as Internet Explorer, but I still don’t really trust it. Perhaps the problem is with the click-tracking domain in the email, rather than with boutiqueacademia.com? So I open the base page at http://boutiqueacademia.com, get redirected immediately to https://www.boutiqueacademia.com – which fails to load.
15542402_1501169219896451_6901276936993410491_n
OK, start with the basics. DNS.

~ ∙ dig www.boutiqueacaedmia.com
; <<>> DiG 9.8.3-P1 <<>> www.boutiqueacademia.com
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 19459
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;www.boutiqueacademia.com. IN A

That’s pretty authoritative (DNS joke … never mind). The response says “NXDOMAIN”, which is DNS-speak for “no such domain”. It means that there isn’t any DNS record for www.boutiqueacademia.com – no A record, nor any DNS record of any other type for that hostname. And it isn’t something that can be caused by a temporary network glitch – it means that the DNS server for the domain says authoritatively “there’s no DNS record”.
This is beginning to look familiar. Lets see what the authoritative DNS servers for boutiqueacademia.com have to say.

~$ dig +short boutiqueacademia.com ns
ns2.hostdns4u.com.
ns1.hostdns4u.com.
~$ dig @ns1.hostdns4u.com www.boutiqueacademia.com
; <<>> DiG 9.9.5-3ubuntu0.7-Ubuntu <<>> @ns1.hostdns4u.com www.boutiqueacademia.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 4587
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1
;; QUESTION SECTION:
;www.boutiqueacademia.com. IN A
;; ANSWER SECTION:
www.boutiqueacademia.com. 30 IN CNAME boutique-academia.myshopify.com.

There’s our problem. Boutique Academia are using hostdns4u.com to host the DNS for their domain – and hostdns4u.com are sending broken DNS responses. They’re responding with a CNAME to myshopify, the ecommerce site, but they’re also responding with the status “NXDOMAIN” – there’s no DNS record for this hostname.
That response violates the way DNS works. If it’s returning any matching DNS record – such as the CNAME – it should return a “NOERROR” status, not “NXDOMAIN”.
Where this gets interesting is when you ask what a DNS resolver will do with this invalid response. That’s not really defined, and different recursive resolvers will treat this response in different ways. Some will ignore the NXDOMAIN and return the CNAME record. Some will ignore the CNAME record and return NXDOMAIN.
That means that depending on which sort of DNS server your recipient is using everything may work perfectly, or any attempt to visit the website may give an error. If the DNS server you’re using is sloppy about NXDOMAINs you may never see the problem in your own web browser.
As far as I can tell this bug is specific to CNAME records, and exists at several DNS hosting companies.
The end result is that if you’re using an ecommerce or blog-hosting company that uses CNAMEs in your DNS zone to point to their servers (such as shopify or wordpress) and you’re  hosting your DNS with a company that has this bug then a significant fraction of the Internet will not be able to reach your website.
You’ll see their opens and clicks on your email. If you’re using pay-per-click advertising you’re paying for each customer. But they’ll never see your site.
I’m not sure what fraction of the Internet is using recursive resolvers that are strict about invalid CNAME responses, but I’d guess it’s at least 20%. If your website suffers from this issue then you could get 25% more sales by moving your DNS hosting to a company that doesn’t suffer from this issue.
(I have told Boutique Academia about this issue. Hostdns4u.com have a cPanel error page for their website, so I didn’t try to contact them.)

Related Posts

HE.net DNS problems

Hurricane Electric had a significant outage of their authoritative DNS servers this morning, causing them to return valid responses with no results for all(?) queries. This will have caused delivery problems for any mail going to domains using HE.net DNS – which will include some of their colocation customers, as well as users of their free services – but also will have caused reverse DNS to fail for most servers hosted by Hurricane Electric worldwide, so if any of your mail is being sent from HE hosted machines you may have seen problems.
(We’re HE customers so we noticed. Still happy with them as a vendor.)

Read More

The Internet is hard.

There are so many things that need to happen to make the Internet work. DNS entries need to be right. MXs need to be set up. Web servers need to be configured. And, let’s be honest, anyone who has ever run their own services on the Internet has flubbed a configuration.
We don’t think about it, because most of the time the configurations are handled by scripts and they do things right. But at some point someone needs to type in something and there’s a risk it will go horribly wrong. I’ve been digging into domain data for a client of mine today. I think I’m going cross-eyed over it. But I have found so many weird things that just mean someone isn’t paying attention to what they’re doing.
Like the domain that has a MX record that says:
nullmx
 
I’m pretty sure the intention of the domain owner is to publish a null MX. But they added an extraneous “0” in there and ended up publishing something really weird. Even worse, the MTA that this client is using is listing this as a “delivered” email. I’m pretty sure that mail to that domain never left the MTA.
I’ve found horribly typoed MX domains for popular spam filters. I’ve found domains that have invalid characters in them. I’ve found domains that are totally a mess.
The vast majority of us have some story or other of the time we really broke things by accident. Like the time a very large ISP deleted their MX records. Or when a different ISP changed their internal forwarding and broke SPF authentication for everyone mailing that domain. Or when another ISP accidentally blocked every IP beginning with 6.
Sometimes I’m amazed that the Internet ever works. No matter how big it gets, there are actual people writing actual code and configurations. The number of things that have to happen to get packets from A to B is pretty impressive. We rarely ever notice the breakages, the people who run things are really good at their jobs. But sometimes poking in the grotty corners reminds me how easy it is to break things. It’s sometimes a wonder things actually work.
 

Read More

TXTing

txt
On Friday I talked a bit about the history behind TXT records, their uses and abuses.
But what’s in a TXT record? How is it used? When and where should you use them?
Here’s what you get if you query for the TXT records for exacttarget.com from a unix or OS X command line with dig exacttarget.com txt

Read More