I have configured static DNS records in the RouterOS DNS resolver, which all DNS requests are routing through. Yet I am seeing evidence that look-ups on those static records are still being forwarded upstream for unknown reasons.
For example, I have a static A record of with name “switch.mgmt.domain.tld” pointing to an internal IP address.
I can resolve this manually and it comes back exactly as I expected
# host switch.mgmt.domain.tld
switch.mgmt.domain.tld has address 192.168.xx.x
But my upstream DNS (NextDNS) still shows requests for switch.mgmt.domain.tld are being forwarded to it quite often. Why would RouterOS still lookup a record on the forwarder address when there is a static record in it’s database? Especially when it is actually returning the static record to me.
I had hoped to work-around this problem by doing conditional forwarding with a regex “negative lookahead” which forwarded anything except my domain to NextDNS, but it seems the regex support in RouterOS’s DNS resolver doesn’t support that advanced of a regex as it would simply fail to forward anything at all.
I used
^(?!((.*\.mydomain.tld|mydomain\.tld))$).+$
in a FWD record but it would never pass anything to it.
note: domain.tld is not my real domain, I replaced it for privacy reasons when posting this.
Are you sure it is RouterOS that is forwarding DNS requests made to its internal resolver outward to your DNS server?
Can’t it be the clients making outbound requests that are being NATted by the router so the source is still the same?
Remember that in must systems there is no such thing as a “primary” and “secondary” DNS resolver (i.e. when you specify 2 DNS resolvers they are both being tried).
One way to check what clients are bypassing router DNS is to add a firewall logging rule for UDP/TCP 53 with destination IP other than the router.
Nowadays some web browsers and smart devices by default don’t use OS (DHCP-provided) DNS address but go directly upstream. To make it even more complicated, they could use DoT or DoH, the latter being especially difficult to detect because of common HTTPS port.
What ROS version? I just tested on 6.49.*, and so far I can’t reproduce on that version at the very least.
If this is happening, you should be able to see the router transmitting the DNS request. So, have you tried running the sniffer with interface=, ip-proto=udp, port=53, direction=tx, and does it capture those packets if you try to resolve that hostname either from the router or from another host on the LAN (after you have tried clearing local host DNS cache of course)?
I’m on 7.18.2. They have made quite a few changes to the DNS resolver since 7.x so they may not act the same.
I only use DoH on ROS so I would not be able to sniff the connection on the wan port as you suggested. But since I am observing this behavior on the upstream server, which is NextDNS, I know that they were making it to the upstream and that the only connections to NextDNS were using DoH from the router’s IP so I am confident that ROS was forwarding them.
I decided to install the NextDNS proxy on a couple of my Pi’s on the LAN and direct ROS to query that instead of going strait to NextDNS.
The NextDNS proxy logs confirm that ROS is indeed sending those upstream queries as I suspected.
Thankfully I was able to configure the NextDNS Proxies to not forward domain.tld upstream and the NextDNS service on the internet no longer shows those queries being routed upstream.
It depends on how the lookups are done by the clients. I have IPv6 disabled in RouterOS also, but if I do nslookup www.google.com, I see both IPv4 and IPv6 returned. And ROS DNS cache shows those AAAA entries. Note, if I do just ping, the entries don’t appear. But when going to www.google.com via a browser, I see some AAAA records for *.google.com. I flushed cache between tests.
Many DNS stacks are asking for both A and AAAA regardless of IP version used. I actually ran into this at work where we don’t use IPv6, and our network team was asking me about the amount of DNS queries some of our larger Red Hat servers were generating. This led to discovery (for me) that for each A query there was another AAAA query, effectively doubling the traffic. Red Hat OS allows to disable AAAA lookups, but it’s on by default.
Since you have visibility into your upstream DNS, you could observe this much better than simply relying on watching ROS cache like I did. Try adding an AAAA static record and then watch your upstream to see if it ever shows up again there.
I agree, this is annoying. I actually never thought about it for my home network, now I guess my .home queries are probably forwarded upstream (not that I care much). There might be a way to block all AAAA queries with an L7 firewall rule, not sure. Or, easier, since you don’t use IPv6, I’m thinking a ROS DNS regex AAAA entry pointing to the special discard prefix 100:: (RFC6666) could be all that’s needed.
Are you trying to hide potentially sensitive internal names, or maybe your upstream services charges by the query? Just curious how you got down this rabbit hole.
Of course when that is the problem he is after, it would not only be AAAA records.
Some programs also do queries for SRV, TXT or MX records.
I think you would need to add them all, or use the “NXDOMAIN” type which would return an error when such a record is queried.
Other records are specialized and unlikely to be used in most scenarios whereas AAAA are being looked up by probably most browsers. But I agree, if the goal is to catch 100%, this is the way, basically, just adding one of each SRV, TXT, MX the same way as shown above.
I’m not sure, would NXDOMAIN work here, since he actually needs to use the domain with A records? If RouterOS checks static entries top to bottom, then I guess one could add everything needed and then NXDOMAIN with match-subdomain at the bottom. But I’m not sure this is how ROS works, it might be doing ordered lookup for each record type independently. It didn’t work this way in my tests, an AAAA record lower on the list than an NXDOMAIN record was still returned.
I could try setting up a special FWD rule for my own domain and have it forward back to itself every time for that, although I’m not sure if that would cause a infinite loop situation.
I think when you add a static record of type NXDOMAIN for your own local domain+subdomains you can still add records for individual names which then will get preference when they are queried.
This would be the best, but unfortunately, it doesn’t work like this, I just tested. Even when NXDOMAIN is the last on the list, all valid local lookups fail. Note, Windows caches DNS, so either need to flush it or use a Linux system for testing (it doesn’t cache unless a caching resolver is configured).
It probably won’t be infinite since DNS requests have a timeout. But it will slow down lookups for non-existing names. It’s also not clear in what order FWD records are processed. It might break valid lookups.
You already have static A entries for all valid appropriate IPv4 addresses, correct? Everything else should be just discarded, including AAAA lookups you don’t need, if I understand your goal correctly. Then just use a single entry I posted above to intercept and discard all AAAA lookups. No device will actually try to use these since you have IPv6 disabled.
I don’t know if the same “catch-all” for IPv4 can be achieved since ROS DNS cannot be an authoritative server. There is no “discard prefix” in IPv4. Pointing to a non-existing IP will slow down any devices/software that might be trying some invalid local name, since it will actually get resolved as opposed to NXDOMAIN.
One possible solution is so point the domain+match-subdomain A record to an unused private IP and add a forward firewall rule to reject connection attempts to that IP (as opposed to silently drop, which would cause slowdowns).
I think it might be easier to just add the AAAA records, perhaps using mapped prefix ::FFFF: like ::FFFF:192.168.88.1 as the address (match to the A). And a script/scheduler to keep them updated from A record if desired.
Or perhaps just use NextDNS for all static records, and just let MikroTik resolve to that.
Not saying some tricks MIGHT be possible, but since /ip/dns is not a real DNS server you’re fighting an uphill battle. And, especially with FQDNs since RouterOS does not have a host domain name for itself.
I tried it. It returned an IPv6 record to any query I made, which I didnt particularly like. But worst of all, as soon as that happened, I started seeing MX queries being forwarded. So I created an MX record, and then I started seeing 65 (HTTPS) queries being forwarded, which ROS doesn’t let you create.
As for using NextDNS for static records, I don’t want local records being sent across the internet. Both for privacy reasons (I mean i don’t want it being recorded in NextDNS, i’m not using unencrypted DNS) and because I don’t want my static records to fail when the internet is down.
I’m just going to keep runing NextDNS proxy containers on different boxes on the LAN and force everything to go through them. They detect when my domain is part of the query and they re-direct them to RouterOS to get the local records. Nothing ends up transmitted up to NextDNS.
That is sort of what I expected… see above.
Unfortunately it seems that my NXDOMAIN solution is not possible (I thought that more-specific entries always have priority over domain-wide entries, but apparently not).
There are some things in DNS that do not work the way you would guess, and it is often impossible to really fix.
For an advanced environment like you have, indeed I would use a different DNS service, running on a local system or maybe in a RouterOS container. MikroTik does not seem to be willing to change their DNS resolver to a reputable advanced (and debugged) solution.
Mostly curious: is that actually a side-effect of this bug, though? Will it allow its response to your request to be influenced by any responses it gets from upstream? Does it actually seem to generate delay to you getting a response back, while it waits on the upstream request to either finish or time out? Have you tried killing your internet connection for a minute and making a query to the MT for one of your local records?
