Community discussions

 
eduplant
just joined
Topic Author
Posts: 8
Joined: Tue Dec 19, 2017 9:45 am

[Feature Request] :resolve DNS Client Improvements

Sat Nov 17, 2018 8:18 am

One of the advantages of RouterOS is its scriptability and the strength of its shell syntax for getting things done. New improvements in the :system and :tool areas have given us more tools than ever, and augmenting existing features with script="" hooks have given us even more places to use those tools. However, it seems like an important scripting primitive (for a network device, at least) has been neglected for some time: :resolve.

So what needs changing? Let's look at the current behavior:

Fig. 1: Current behavior
[user@router] > :put [ :resolve www.google.com ]
216.58.219.36
[user@router] > :put [ :resolve ipv4.google.com ]
216.58.193.206
[user@router] > :put [ :resolve ipv6.google.com ]
2607:f8b0:4007:80b::200e
[user@router] > :put [ :resolve _ldap._tcp.dc._msdcs.contoso.com ]
failure: dns name exists, but no appropriate record

Some things to note:
  1. Only one answer is ever returned, even if there are multiple answers.
  2. If both A and AAAA records exist, RouterOS chooses the A record.
  3. If the record is of another type, the resolution fails.

Taken together, I would guess that the RouterOS developers chose to turn DNS resolution into a single-answer succeed/fail operation so that parsing the results of :resolve would not require much effort. Assuming this was the logic, my suggestion also tries to preserve as many of these properties as possible.

So here are the proposed changes:

1. Continue the default behavior to only return one record, but provide an option full-answer=true|false to return the full answer.

The :resolve command is used for both debugging and scripting purposes and this change benefits both purposes. Since RouterOS has DNS server functionality, it's always frustrating to debug DNS-related issues when the router itself doesn't have a good DNS client. Currently, the only recourse is to use dig or nslookup from a client device and then inspect :ip dns cache on the router to see what happened. From a scripting perspective, it would be nice to be given all of the answers for a query for round-robin connections, health checks, etc.

2. Provide an option in :ip dns called client-behavior: prefer-v4|dual-stack
  • prefer-v4 preserves the legacy behavior and will return A record(s) if both A and AAAA are available. Since some users are undoubtedly relying on this quirk, this can remain the RouterOS default for several versions to give them time to migrate.
  • dual-stack follows RFC 8305 and attempts dual-stack resolution like a standard DNS client. If both address families are present and RouterOS has a configured IPv6 address it can use as a source, the AAAA record(s) are returned. After a sufficient amount of time, this should become the RouterOS default.

The fact that RouterOS unconditionally prefers IPv4 makes it ill-suited as a modern dual-stack client. I don't fully understand why the choice was made in the first place. Although not in my ask here (because of the amount of work that would be involved), I do hope that RouterOS 7 has a proper RFC-8305-compliant control plane for any connections the router makes.

3. Provide typed answers for some common record types (CNAME, NS, MX, SRV, and TXT please!) and provide string answer dumps for unimplemented/unknown record types.
  • For backward compatibility, only provide IPv4 or IPv6 answers unless the type=a|aaaa|cname|ns|mx|srv|txt|.* option is used.
  • As an additional measure of backward compability, :resolve should fail if the record cannot be resolved to an IPv4 or IPv6 address, regardless of whether a full answer could be provided.
  • type=auto represents these behaviors and is the default.

:resolve already returns ip and ip6 typed answers, so script authors already have to type inspect if they care about the type of the returned address. What's more, RouterOS' shell language already has the right datatypes to handle more complicated answers (arrays!).

Finally, here are some specific examples of what it all looks like put together:

Fig. 3: Proposed type behavior for auto
#Get to a single answer, even if that means picking one. Actual answer depends on dual-stack setting. With the right dual-stack setting (see 2. above), this is fully backward compatible with existing scripts.
[user@router] > :put [ :typeof [ :resolve type=auto www.google.com ] ]
ip|ip6
#Actual array is {ip|ip6,[, ip|ip6...]} of the full RRset.
[user@router] > :put [ :typeof [ :resolve type=auto full-answer=true www.google.com ] ]
array

Fig. 4: Proposed type behavior for A
#Get to a single answer, even if that means picking one. Because the user asked for A specifically, dual-stack setting has no bearing here.
[user@router] > :put [ :typeof [ :resolve type=a www.google.com ] ]
ip
#Actual array is {ip,[, ip...]} of the full RRset. Because the user asked for A specifically, dual-stack setting has no bearing here.
[user@router] > :put [ :typeof [ :resolve type=a full-answer=true www.google.com ] ]
array

Fig. 5: Proposed type behavior for AAAA
#Get to a single answer, even if that means picking one. Because the user asked for AAAA specifically, dual-stack setting has no bearing here.
[user@router] > :put [ :typeof [ :resolve type=aaaa www.google.com ] ]
ip6
#Actual array is {ip6,[, ip6...]} of the full RRset. Because the user asked for AAAA specifically, dual-stack setting has no bearing here.
[user@router] > :put [ :typeof [ :resolve type=aaaa full-answer=true www.google.com ] ]
array

Fig. 6: Proposed type behavior for CNAME
#Get to a single answer, even if that means picking one. Unlike type=auto, the answer is just a resolved canonical name.
[user@router] > :put [ :typeof [ :resolve type=cname ipv4.google.com ] ]
str
#Actual array is {str,[, str...]} of the full RRset. Unlike type=auto, the answer is all resolved canonical names.
[user@router] > :put [ :typeof [ :resolve type=cname full-answer=true ipv4.google.com ] ]
array

Fig. 7: Proposed type behavior for NS
#Get to a single answer. Actual answer depends on dual-stack setting.
[user@router] > :put [ :typeof [ :resolve type=ns ns1.google.com ] ]
ip|ip6
#Actual array is {ip|ip6,[, ip|ip6...]}.
[user@router] > :put [ :typeof [ :resolve type=ns full-answer=true ns1.google.com ] ]
array

Fig. 8: Proposed type behavior for MX
#Get to a single answer, even if that means MX -> pick an answer -> lookup CNAME/A/AAAA -> pick an answer. Ignore the priority; if users want that they can parse the full answer. Actual answer depends on dual-stack setting.
[user@router] > :put [ :typeof [ :resolve type=mx google.com ] ]
ip|ip6
#Actual array is {array,[, array...]} of the full RRset.
#Each internal array is an answer and is {hostname=str, priority=num}.
[user@router] > :put [ :typeof [ :resolve type=mx full-answer=true google.com ] ]
array

Fig. 9: Proposed type behavior for SRV
#Get to a single answer, even if that means SRV -> pick an answer -> lookup CNAME/A/AAAA -> pick an answer. Ignore the priority, weight, and port; if users want that they can parse the full answer. Actual answer depends on dual-stack setting.
[user@router] > :put [ :typeof [ :resolve type=srv _ldap._tcp.dc._msdcs.contoso.com ] ]
ip|ip6
#Actual array is {array,[, array...]} of the full RRset.
#Each internal array is an answer and is {hostname=str, port=num, priority=num, weight=num}.
[user@router] > :put [ :typeof [ :resolve type=srv full-answer=true _ldap._tcp.dc._msdcs.contoso.com ] ]
array

Fig. 10: Proposed type behavior for TXT (and unimplemented/unknown records)
#Concatenate the full RRset (here TXT) into a single string.
[user@router] > :put [ :typeof [ :resolve type=txt www.google.com ] ]
str
#Actual array is {str,[, str...]} of the full RRset (here TXT).
[user@router] > :put [ :typeof [ :resolve type=txt full-answer=true www.google.com ] ]
array

Even though the explanation of the changes is lengthy, I think this preserves a balance between backward compatibility and new features and tries to be specific about the extent of the changes requested. There are a lot of "shorter" feature requests that involve entire new technologies and protocol implementations :). Many of them would require much more effort and would still be a niche feature, whereas DNS improvements are almost universally applicable.

Taken together, these changes would modernize :resolve, make it a more useful troubleshooting tool, and continue to open the door for even more scripting flexibility. What do others think?
 
msatter
Forum Guru
Forum Guru
Posts: 1235
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: [Feature Request] :resolve DNS Client Improvements

Sat Nov 17, 2018 12:58 pm

I don't see a problem with :resolve, it just resolves to which address to go. Not every one has IPv6 so IPv4 is the safest to go. If there are multiple addresses then rotation is steered by the DNS server.

Several parts in RouterOS already read the complete list, see addresslists.

I would prefer a new command for reading DNS records to do that.
Two RB760iGS (hEX S) in series. One does PPPoE and both do IKEv2.
Running:
RouterOS 6.46Beta / Winbox 3.20 / MikroTik APP 1.3.4
Having an Android device, use https://github.com/M66B/NetGuard/releases (no root required)
 
eduplant
just joined
Topic Author
Posts: 8
Joined: Tue Dec 19, 2017 9:45 am

Re: [Feature Request] :resolve DNS Client Improvements

Sat Nov 17, 2018 10:21 pm

I don't see a problem with :resolve, it just resolves to which address to go.
In a literal sense, I agree. This is why I'm suggesting that the default behavior without type= or full-answer=true be preserved for backward compatibility. Getting a single answer is a useful property of :resolve.

Not every one has IPv6 so IPv4 is the safest to go.
I have to disagree here, and so do the RFCs. If your router is dual-stacked, there is rarely a reason to prefer v4 connectivity. Of course you're right in one sense; it's important that v4-only deployments still work unmodified. Returning IPv6 answers just because they exist isn't RFC-compliant behavior either.

If there are multiple addresses then rotation is steered by the DNS server.
Entirely true for the sake of picking an answer. For troubleshooting and diagnostics (remember, RouterOS is a recursive resolver), it's really rough to not get the full answer.

Several parts in RouterOS already read the complete list, see addresslists.
This is interesting functionality that I didn't know existed, thanks for pointing it out. I use address-lists all the time but never realized that it did multiple-resolution. The documentation is a little light...

Fig. 1: Multiple resolution with address-list
[user@router] /ip firewall address-list> add list=test4only address=cnn.com
[user@router] /ip firewall address-list> pr where list=test4only
Flags: X - disabled, D - dynamic
 #   LIST                                        ADDRESS                                                         CREATION-TIME        TIMEOUT
 0   test4only                                   cnn.com                                                         nov/17/2018 11:52:00
 1 D ;;; cnn.com
     test4only                                   151.101.193.67                                                  nov/17/2018 11:52:00
 2 D ;;; cnn.com
     test4only                                   151.101.129.67                                                  nov/17/2018 11:52:00
 3 D ;;; cnn.com
     test4only                                   151.101.1.67                                                    nov/17/2018 11:52:00
 4 D ;;; cnn.com
     test4only                                   151.101.65.67                                                   nov/17/2018 11:52:00

I guess that also means that you can resolve both address families separately because of the explicit split between :ip firewall address-list and :ipv6 firewall address-list

Fig. 2: Hacky dual-stack resolution with address-list
[user@router] /ip firewall address-list> add list=test64 address=www.google.com
[user@router] /ip firewall address-list> pr where list=test64
Flags: X - disabled, D - dynamic
 #   LIST                                        ADDRESS                                                         CREATION-TIME        TIMEOUT
 0   test64                                      www.google.com                                                  nov/17/2018 11:52:43
 1 D ;;; www.google.com
     test64                                      216.58.217.196                                                  nov/17/2018 11:52:43

[user@router] /ipv6 firewall address-list> add list=test64 address=www.google.com
[user@router] /ipv6 firewall address-list> pr where list=test64
Flags: X - disabled, D - dynamic
 #   LIST                                            ADDRESS                                                                          TIMEOUT
 0   test64                                          www.google.com
 1 D ;;; www.google.com
     test64                                          2607:f8b0:4007:808::2004/128

If you wanted to use this, though, you'd have to:
  1. Create a dummy IPv4 address-list
  2. Create a dummy IPv6 address-list
  3. Extract the contents with a loop, find, and get into a temporary array
  4. Destroy both address-lists
  5. Return the array

This isn't exactly convenient for debugging a DNS resolution problem unless you packaged it into a function and brought it with you on all of your routers. I mean, let's face it --- this functionality is in there for FQDN firewall rules. If anything, it's a good sign that the DNS resolver code that underlies both :resolve, :ip firewall address-list, and the rest of RouterOS is already written to process multiple records. Why not just expose the functionality within :resolve?

I would prefer a new command for reading DNS records to do that.
I wouldn't mind it being a separate command but, in my mind, :resolve is a perfectly natural place to put this functionality. After all, it IS the interactive DNS client for RouterOS, right?
 
cthil
just joined
Posts: 13
Joined: Sun Jan 10, 2016 7:29 pm

Re: [Feature Request] :resolve DNS Client Improvements

Wed Jan 16, 2019 10:49 am

I fully support your proposals to enhance the DNS resolver. At current, I use an external resolver via HTTP/CGI interface to circumvent the limitations of the internal one and would be VERY happy to abandon that path. Resolving arbitrary types also gives the ability for predictable IPv4 A/IPv6 AAAA discrimination, which is a big problem at current.
 
User avatar
Cha0s
Forum Veteran
Forum Veteran
Posts: 901
Joined: Tue Oct 11, 2005 4:53 pm

Re: [Feature Request] :resolve DNS Client Improvements

Fri Jan 18, 2019 4:24 pm

+1 by me as well.
 
eduplant
just joined
Topic Author
Posts: 8
Joined: Tue Dec 19, 2017 9:45 am

Re: [Feature Request] :resolve DNS Client Improvements

Sun Jan 20, 2019 12:36 pm

Thanks @cthil and @Cha0s for the feedback.

At current, I use an external resolver via HTTP/CGI interface to circumvent the limitations of the internal one and would be VERY happy to abandon that path.

Truth be told, I'm trying desperately to avoid taking this path in the first place. I'm working on a RouterOS clone of Cisco's DMVPN using BGP + unnumbered OSPF + IPSec. Basically a bunch of spoke routers with dynamic addresses (think branch offices with commodity internet) can tunnel to a set of known hub routers (think HQ/datacenter), learn the addresses of all of the other spoke routers via BGP, and automatically build spoke-to-spoke tunnels via scripting. So far it's working well in the lab, but a major pain point is the initial bootstrapping phase where a new spoke router needs the current list of available hub routers (preferably DNS names and both v4/v6). Doing it with DNS SRV records would be the most elegant, followed by even just a DNS TXT record containing the list. Or even further still, having multiple A/AAAA records for a given name like hubs.dmvpn.contoso.com where you could use any or all of the multiple answers.

Regrettably, none of this works because :resolve is the way it is. I might have to host a static webserver that hosts a plain text file or resort to a DNS-over-HTTP answer like what you're describing just so that DNS can be made into an HTTP nail since all RouterOS has is a :tool fetch hammer.

Ironically, folks have been complaining about some of the dual-stack issues since at least 2010 and the thread still isn't dead.
 
colin
newbie
Posts: 35
Joined: Mon May 11, 2015 11:11 am

Re: [Feature Request] :resolve DNS Client Improvements

Mon Jan 21, 2019 7:13 pm

I fully support your proposals to enhance the DNS resolver.

And i think use the address list to get all of the address for a domain is a good idea, but if i want to resolve the domain by different dns server, how to do it?

So i think the best answer is to enhance the DNS resolver, i think.
 
eduplant
just joined
Topic Author
Posts: 8
Joined: Tue Dec 19, 2017 9:45 am

Re: [Feature Request] :resolve DNS Client Improvements

Mon Jan 21, 2019 7:55 pm

Oh good question @colin, I don’t know of a way. Perhaps adding :resolve server=8.8.8.8 might be handy.
 
colin
newbie
Posts: 35
Joined: Mon May 11, 2015 11:11 am

Re: [Feature Request] :resolve DNS Client Improvements

Tue Jan 22, 2019 4:46 pm

Oh good question @colin, I don’t know of a way. Perhaps adding :resolve server=8.8.8.8 might be handy.
Yes, the resolve command has the server parameter, but it seems that ip firewall address-list always resolve the domain address list by the default dns server.

Who is online

Users browsing this forum: No registered users and 77 guests