Community discussions

MikroTik App
 
axe3
just joined
Topic Author
Posts: 6
Joined: Wed Mar 08, 2023 1:54 pm

DNS not working in containers with DNS over HTTPS setup on router

Wed Mar 08, 2023 3:05 pm

Hey, forum!

I'm experimenting with some containers running on my router, specifically, D53G-5HacD2HnD-TC&RG502Q-EA.

I presently have set up DNS over HTTPS on the router per instructions here: https://www.youtube.com/watch?v=w4erB0VzyIE and it has worked great for me for all devices on the network.

The issue I have now is that if I run a container on the router, then DNS doesn't work with the current DoH setup. Minimal example is running the image 'alpine:latest' with a CMD 'ping google.com' and enabling logging. This results in the container stopping, logging "ping: bad address 'google.com'". If I change the CMD to 'ping 8.8.8.8' at this point, it works, so I know the container has access to the internet. If I disable DoH, then DNS works as expected. How do I get containers to work with the DoH setup?

For getting started with containers I followed the containers series of videos: https://www.youtube.com/watch?v=8u1PVouAGnk, https://www.youtube.com/watch?v=UMcJs4oyHDk, https://www.youtube.com/watch?v=i9GcFEx_Ois.

Potentially relevant info:
/ip/dns print 
                      servers: 
              dynamic-servers: 
               use-doh-server: https://cloudflare-dns.com/dns-query
              verify-doh-cert: yes
   doh-max-server-connections: 5
   doh-max-concurrent-queries: 50
                  doh-timeout: 5s
        allow-remote-requests: yes
          max-udp-packet-size: 4096
         query-server-timeout: 2s
          query-total-timeout: 10s
       max-concurrent-queries: 100
  max-concurrent-tcp-sessions: 20
                   cache-size: 2048KiB
                cache-max-ttl: 1w
                   cache-used: 78KiB

/ip/firewall/nat print 
Flags: X - disabled, I - invalid; D - dynamic 
 0    ;;; defconf: masquerade
      chain=srcnat action=masquerade out-interface-list=WAN ipsec-policy=out,none 

 1    chain=dstnat action=redirect protocol=tcp dst-port=53 log=no log-prefix="" 

 2    chain=dstnat action=redirect protocol=udp dst-port=53 log=no log-prefix="" 

 3    ;;; Enable containers to access the internet
      chain=srcnat action=masquerade src-address=172.17.0.0/24 log=no log-prefix=""

Here the firewall filter rules beside the defaults:
/ip/firewall/filter print 
Flags: X - disabled, I - invalid; D - dynamic 
13    ;;; Custom made rule - drop DNS queries
      chain=output action=drop protocol=tcp dst-port=53 log=no log-prefix="" 

14    ;;; Custom made rule - drop DNS queries
      chain=output action=drop protocol=udp dst-port=53 log=no log-prefix="" 
 
User avatar
antonsb
MikroTik Support
MikroTik Support
Posts: 385
Joined: Sun Jul 24, 2016 3:12 pm
Location: Riga, Latvia

Re: DNS not working in containers with DNS over HTTPS setup on router

Thu Mar 09, 2023 11:05 am

Thanks for report. This is known issue - automatically "servers" field is passed to container, but in case of DOH this field does not exist, for quick fix set/change DNS in container setup to Routers DNS IP address -
/container/set <your container> dns=192.168.88.1
 
axe3
just joined
Topic Author
Posts: 6
Joined: Wed Mar 08, 2023 1:54 pm

Re: DNS not working in containers with DNS over HTTPS setup on router

Thu Mar 09, 2023 11:51 pm

Thanks for the acknowledgement of the issue. I tried the suggested workaround, but unfortunately it didn't work - I still get the same log message: "ping: bad address 'google.com'". Any other ideas or suggestions to try?
/container print 
 0 ;;; test ping google.com
   name="40398a4c-a9d1-4dda-8d29-98a856360dc1" tag="library/alpine:latest" os="linux" arch="arm" interface=veth1 cmd="ping google.com" mounts="" dns=192.168.8.1 logging=yes status=stopped
Note: the IP of my router is 192.168.8.1 (not 192.168.88.1):
/ip/address print 
Flags: D - DYNAMIC
Columns: ADDRESS, NETWORK, INTERFACE
#   ADDRESS          NETWORK       INTERFACE 
;;; defconf
0   192.168.8.1/24   192.168.8.0   bridge
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 3169
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: DNS not working in containers with DNS over HTTPS setup on router

Fri Mar 10, 2023 12:10 am

Thanks for the acknowledgement of the issue. I tried the suggested workaround, but unfortunately it didn't work - I still get the same log message: "ping: bad address 'google.com'". Any other ideas or suggestions to try?
Could be your firewall is blocking the traffic from the container IP to the DNS IP. If you add the container's VETH to the LAN interface list, that might be quick test.

Also, presume you have "allow remote requests" enabled in /ip/dns, but that's needed too for this to work.

Lastly, it could also be Alpine isn't picking up the DNS IP. If you "tail -f /dev/null" as run and get into the /container [find] shell, see do a "cat /etc/resolv.conf", and see if it's actually picking up the DNS server from RouterOS.
 
axe3
just joined
Topic Author
Posts: 6
Joined: Wed Mar 08, 2023 1:54 pm

Re: DNS not working in containers with DNS over HTTPS setup on router

Mon Mar 13, 2023 2:18 pm

Also, presume you have "allow remote requests" enabled in /ip/dns, but that's needed too for this to work.
I already had this.

Could be your firewall is blocking the traffic from the container IP to the DNS IP. If you add the container's VETH to the LAN interface list, that might be quick test.
Thanks for the suggestion, I stumbled upon the solution that works. I tried to do what you suggested, but by mistake added the containers Bridge (instead of the containers VETH) to the LAN interface list, which enabled the container to use the DNS specified in the container setup, so DoH now works for the container too!
Once I realized my mistake I also tried having the container VETH instead of the Bridge in the LAN interface list, but this did not work - same DNS failure logged as I described in the original problem.

So it was the firewall blocking the DNS reqeusts from the containers like you suggested Amm0 - I checked it using logging, specifically the default rule to drop all non-LAN traffic:
/ip/firewall/filter print 
Flags: X - disabled, I - invalid; D - dynamic
 6    ;;; defconf: drop all not coming from LAN
      chain=input action=drop in-interface-list=!LAN
No need to change this.


So, to summarize to anyone else bumping into this - both antonsb and Amm0 suggestions were relevant to make this setup work (assuming you already have DoH setup and containers enabled according to Mikrotik's youtube tutorials):
1. Set DNS in your container setup to the routers DNS IP - looking forward to this issue getting addressed in an upcoming release per antonsb suggestion
2. Add the containers Bridge interface to the LAN interface list, as it's likely that the default firewall settings block traffic from the container IP to the routers DNS IP

Here's how the interface lists look like (in my case I named the containers bridge "containers" instead of "dockers" in contrast to the videos as this naming seemed clearer to me):
/interface/list/member print
Columns: LIST, INTERFACE
# LIST  INTERFACE 
;;; defconf
0 LAN   bridge    
;;; defconf
1 WAN   lte1      
;;; Containers: have the containers Bridge be on the LAN interface list so firewall allows DNS requests from containers; as suggested on forum 
2 LAN   containers
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 3169
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: DNS not working in containers with DNS over HTTPS setup on router

Mon Mar 13, 2023 4:27 pm

Yeah, seemed like a firewall problem. /container's dns/hostname/domainname settings seem to just update /etc/hosts and /etc/resolve.conf. Alpine's built-in DNS resolve is pretty limited (e.g. it use only UDP) to keep Alpine small/efficent. So @antonsb is totally right to have DoH run back through the Mikrotik DNS – the code needed for DoH only needs to be one place.

FWIW, you often don't need a "container" bridge for VETH. It can live on it's own, and for some L3/IP server apps in a containers that may be easier since you can refer to that container instance by it's VETH in port forwarding and other firewall rules. With multiple containers that need to communicate among themself, or you have some Layer2 thing (e.g. the nifty netinstall container someone built), a bridge is needed.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: DNS not working in containers with DNS over HTTPS setup on router

Fri Mar 17, 2023 3:39 am

Hi there, so you use containers for some functionality, but use the DOH on the router itself.

Q1. Did the solution you found to your issue, mean that the Container bypasses DOH for DNS and goes to the router to DND and then out to the internet?
Q2. If not, how did you get the containers traffic to through DOH?

Q3. Why not run docker container for adguard for example instead of DOH?
Q4. Can you run DOH and adguard from within a docker/container IS THERE such a combo beast?

I guess what I am asking if both are good things to have why cannot we use both tools??
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 3169
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: DNS not working in containers with DNS over HTTPS setup on router

Fri Mar 17, 2023 4:22 pm

@anvu, short version:
Container here has nothing to do with being a DNS Server. If you think that when RouterOS is going to start a container, it needs to set the IP address/subnet mask/gateway and a DNS server. Typically containers don't use DHCP to get their network settings – so the VETH setting are what's inside container for ip-address/netmask/gateway. But how the DNS works inside a container, see the long version to answer the questions:


Q1. Did the solution you found to your issue, mean that the Container bypasses DOH for DNS and goes to the router to DND and then out to the internet?
The OP isn't running a DNS Server in the container. But has a Linux application inside the container that needs to do DNS lookups. Like any server/desktop, it needs a DNS server to use. The DNS servers a container initially uses are set by RouterOS settings for the container. This can be set explicitly using "DNS Server" in winbox for a container; or if empty, the container will uses the router's DNS IP address. If you're not using DoH, this all works dandy.

The issue here is what if you don't set DNS for the container, and the Mikrotik DNS has not IP addresses for a DNS server? This happens if RouterOS's DNS is using DoH since it wouldn't have IP addresses for DNS – as setting a DoH server supersedes any IP ones. Since the container DNS settings can only be "regular" DNS (e.g. using port 53, and generally UDP) IP addresss, a DNS Server IP to be used by the container's code needs to be set explicitly in the container's settings.

But a container can also modify its own configuration, including DNS, after install. So for something like Pi-Hole, you'd typically set the upstream DNS in those container's web GUI (or via environment variables to container).

Q2. If not, how did you get the containers traffic to through DOH?
The idea is the regular DNS is fine internally. It's actually faster since than DoH. But with regular DNS on port 53, the domain names being looked up aren't encrypted. This isn't a problem for DNS lookup from a container to the RouterOS running it. So why the container uses port 53 DNS to RouterOS's DNS server here. And the RouterOS's DNS does the DoH over the internet – where you'd care about hiding the domain name being looked up.

(Alpine container) --port 53/udp--> (Mikrotik DNS) --port 443/tcp--> (Internet) --> DoH DNS Server

Q3. Why not run docker container for adguard for example instead of DOH?
OP's case wasn't about the container to be a DNS Server – it just needed an upstream one to work. But nothing would stop anyone from running a 2nd container that run Pi-Hole or Ad-Guard for ad-blocking. And inside those containers web GUI you can set the upstream DNS to use a DoH server. So the Alpine container (or ThingsBoard) could use a DNS server that lives in another container as the DNS server. e.g.

(Alpine container) --port 53/udp--> (Pi-Hole DNS) --port 443/tcp--> (Internet) --> DoH DNS Server

Or more indirectly, RouterOS DNS could be used, as shown in the OP's case with Pi-Hole/etc container upstream of the Mikrotik DNS

(Alpine container) --port 53/udp--> (Mikrotik DNS) --port 53/udp--> (Pi-Hole DNS) --port 443/tcp--> (Internet) --> DoH DNS Server

Q4. Can you run DOH and adguard from within a docker/container IS THERE such a combo beast?
Basically you set DoH on the adguard/pi-hole admin web pages. So those will take care of the DoH.

Alpine is very minimum, so it doesn't have "direct" DoH support. But other containers based on "regular Linux" like Ubuntu or Debian, you can install a package inside the container that does DoH resolution directly from the container. This is essentially what Pi-Hole (and other ad-blocking containers) do:

(Pi-Hole container) --port 443/tcp--> (Internet) --> DoH DNS Server

Now if you want the rest of your network so using the IP address of the Ad-blocking Ubuntu container as the DNS Server, you the container's IP address as DNS in your DHCP server's network settings.

(LAN client doing DNS) --port 53/udp--> (RouterOS's DNS) --port 53/udp--> (Pi-Hole container) --port 443/tcp--> (Internet) --> DoH DNS Server

But in this case, you do have the firewall to configure for sure. And since Pi-Hole/etc may need DNS initially, you may need to explicit set the DNS Server for Pi-Hole/etc in the container setting to a real DNS server like 8.8.8.8, 1.1.1.1, 208.67.222.222.222, or 9.9.9.9 depending on your religion – you can't have Pi-Hole/etc pointing to Mikrotik DNS and the Mikroitk DNS pointing to Pi-Hole.


Overall... Like the OP saw, the firewall for container's IP/VETH/Bridge can get confusing & easily dropped since in default container's bridge/VETHs are by default "!LAN" in /ip/firewall/filter.

Who is online

Users browsing this forum: No registered users and 2 guests