Last week the megarbl.net domain name expired. Normally this would have no affect on anyone, but their domain registrar put in a wildcard DNS entry. Because of how DNSBLs work, this had the effect of causing every IP to be listed on the blocklist. The domain is now active and the listings due to the DNS wildcard are removed.
How does a domain expiration lead to a DNSBL listing the whole internet?
Well, it’s kinda complicated and involves some of those grubby corners where different things that shouldn’t happen all occur and weird stuff happens.
But I thought there were rules and standards about this stuff.
Well, there are standards on the Internet. The IETF publishes documents “Request for Comments” (RFCs). These aren’t rules as much as they are standards. While many folks treat RFCs as rules, that’s not really what they’re for. All the RFCs do is tell operators how to do things if they want to communicate seamlessly with other entities. Not every RFC is a standard; some are informational, some are experimental, some are drafts being actively developed and some are current practices.
So how does a domain expiration lead to listing all IPv4 space on a blocklist?
Well, there are a few technical things we need to understand before we can see how we get from domain expiration to blocklisting.
How DNSBLs work
First we need to talk about how DNSBLs work. DNSBLs publish a list of IP addresses. To query a DNSBL for an IP address you take the IP address, reverse it, add the DNSBL name onto the end and query for that domain.
Let’s say I want to look up 37.49.224.172 on the SBL. I flip the IP address and add sbl.spamhaus.org to create the domain name 172.224.49.37.sbl.spamhaus.org. Then I do a DNS lookup on the domain name. If the IP is listed, then Spamhaus will return an A record / IP address. If the IP is not listed, then Spamhaus will return an NXDOMAIN and no A record.
All DNSBLs are run this way, so if you get any answer to a DNS query it means the IP is listed. And, yes, this is how blocklists are queried, even when you use a web form. It’s all DNS behind the scenes.
DNS wildcard records
Now that we know how DNSBLs are queried, we need to talk about DNS and wildcarding. Wildcarding is allowed in the DNS RFCs, but it’s complicated. So complicated entire RFCs have been written to try and explain the grubby corners. I’m certainly not going to try and explain it. If you want more details check out the Google support page, IAB commentary on wildcarding or the Wikipedia article.
The short version that works for our purpose is that a wildcard entry in DNS means that there is an IP for every domain queried. You can make up any domain name and it will return an A record.
Wildcard records and registrars
The next thing we need to know is that some registration providers take unregistered domains and publish wildcard DNS records for them. Typically this is done to collect add revenue for parked or unused domains.
OK… so how does that lead to blocking?
When a domain expires and the registrar reclaims it and publishes a wildcard DNS record, then every query to the DNSBL will come back positive. The only negative DNSBL response is “NXDOMAIN” so the presence of any A record is a listing.
That shouldn’t happen…
Well, no one is technically doing anything wrong here, except possibly the registrar by publishing wildcard record for expired domains. Or the domain owner by forgetting to renew their domain in a timely fashion. The good news is that it happens very rarely, typically once you lose a domain due to expiration you figure out how not to do that again. And not all registrars wildcard expired domains. This is just one of those things where no one is doing anything wrong or unusual, but when they all happen together unexpected things happen.
The good news is that, as of today, the domain is back and the false listings are removed.
Can a DNSBL stop this from happening?
There are number of ways a DNSBL can stop this from happening. The two big ones are not to let the domains expire and use a registrar that won’t wildcard parked domains. Other than that, though, they can’t really.
I’m a DNSBL user, how can I know this is happening?
Well run IPv4 DNSBLs should always list 127.0.0.2 and should never list 127.0.0.1. DNSBL users should query a DNSBL daily for these two addresses. If it is listing 127.0.0.1 or it is not listing 127.0.0.2 then it should be considered non functional and dropped from your filtering scheme. (RFC5782 and RFC6471, )
RSS - Posts
The responses you get from a wildcarded domain are not going to be 127.anything, too. I don’t know about other MTAs, but at least Postfix allows you to
reject_rbl_client ZONE=SPECIFIC_RESPONSE_IP
With Postfix 2.8 and newer, you can also use patterns inside “[]” that contain one or more “;”-separated numbers or number..number ranges.
That way, you will only consult valid DNSBL responses and situations such as described here will not have unexpected consequences!
What do you make f of HMRC’s claims for the success of their DMARC implementation?
http://www.itproportal.com/news/hmrc-blocked-500000-phishing-emails-in-2015/
Atro is right, but in practice all you need to check is that you get some A record for 127.0.0.2 and nothing from 127.0.0.1. I check once a week which seems to be plenty — dead DNSBL domains generally go dead for a couple of months between expiration and being picked up and wildcarded by speculators.
By the way, that spec is in RFC 5782. They just piled on in 6471.
Thanks for the RFC #, I’ll add it in
I know you can do all sorts of funky stuff with return codes and 127.0.0.2 vs .3 vs .12 or whatever. But in a lot of cases they just check for any A record. Hence people’s detectors going haywire when someone wildcards a domain. I didn’t want to get too detailed because it’s turtles all the way down and for the purposes of the post details are confusing.