I saw many times few examples here but still I don’t understand what is going on. Quick view of what it looks like in my network. Without chart.
On Mikrotik
MGMT VLAN (id: 10): 10.255.0.0/27 (with DHCP on MikroTik) / IP: 10.255.0.30
InterVLAN (id: 4040): 10.255.253.0/24 / (required by Unifi) / IP: 10.255.253.1
SRV VLAN (id: 11): 10.0.1.0/27 / (with DHCP on MikroTik) / IP: 10.0.1.30
Unifi Switch Layer3
MGMT VLAN IP: 10.255.0.29
InterVLAN IP: 10.255.253.3
USER_WIFI VLAN (id: 91): 192.168.1.0/24 (with DHCP on Unifi)
I will now try to make it easier as possible
And now In SRV VLAN there is RaspberryPie with IP 10.0.1.20 having K3s connected to Unifi Switch
That K3s host a PiHole and MetalLB. MetalLB Has it’s own addressPool 10.0.100.x
Service of PiHole for both UDP and TCP receives 10.0.100.1 ip address.
Next, this address is advertised with MetalLB BGP peering with MikroTik ( [MikroTik] 10.0.1.30 ↔ [Raspberry Pie] 10.0.1.20)
[konrad@rtpoz1] /tool/sniffer> /ip route/pr where bgp
Flags: D - DYNAMIC; A - ACTIVE; b - BGP
Columns: DST-ADDRESS, GATEWAY, DISTANCE
DST-ADDRESS GATEWAY DISTANCE
DAb 10.0.100.1/32 10.0.1.20 200
I hope everything make sense still
So let’s dive into the problem.
Problem is simple. DHCP for USER_WIFI on Unifi has 10.0.100.1 and 1.1.1.3. That Cloudflare added just in case when 10.0.100.1 fails somehow (I’m preparing another Pie that will have second Pihole in future for HA). What is happening with two DNS on client? Round Robin that happens. And for example all sponsored links in Google are opening although I want to avoid that in normal scenario when PiHole (10.0.100.1) works. So what can I do to next? Let’s force on router and capture every traffic from 192.168.1.0/24 and DST-NAT it to 10.0.100.1 to port 53 both for UDP and TCP. So let’s get into it and here it is
/ip firewall nat
add action=dst-nat chain=dstnat comment="Highway TV DNS" dst-port=53 protocol=udp src-address=192.168.0.20 to-addresses=62.21.99.94 to-ports=53
add action=dst-nat chain=dstnat comment="Redirect DNS UDP traffic in LAN to PiHole" dst-port=53 protocol=udp src-address=192.168.0.0/23 to-addresses=10.0.100.1 to-ports=53
add action=dst-nat chain=dstnat comment="Redirect DNS TCP traffic in LAN to PiHole" dst-port=53 protocol=tcp src-address=192.168.0.0/23 to-addresses=10.0.100.1 to-ports=53
Those last two are the one. I will explain why I intentionally added the one with 192.168.0.20
The problem is when I have those two DNS servers (the one from K3s Pihole & Cloudflare) and those rules to force the traffic going through 10.0.100.1 it still goes through 1.1.1.3. How I know? Sponsored links, remember ? They are still opening.
Ok Tried to sniff and see what I got there
❯ nslookup google.com
Server: 10.0.100.1
Address: 10.0.100.1#53
Non-authoritative answer:
Name: google.com
Address: 172.217.16.14
[konrad@rtpoz1] /tool/sniffer> /tool/sniffer/quick ip-protocol=udp port=53 src-ip-address=192.168.1.1
Columns: INTERFACE, TIME, NUM, DIR, SRC-MAC, DST-MAC, VLAN, SRC-ADDRESS, DST-ADDRESS, PROTOCOL, SIZE, CPU
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU
ether10 1.921 1 <- AC:8B:A9:39:65:31 64:D1:54:7F:B4:D7 4040 192.168.1.1:59943 10.0.100.1:53 (dns) ip:udp 74 0
bridge_trunk 1.921 2 <- AC:8B:A9:39:65:31 64:D1:54:7F:B4:D7 4040 192.168.1.1:59943 10.0.100.1:53 (dns) ip:udp 74 0
InterVLAN 1.921 3 <- AC:8B:A9:39:65:31 64:D1:54:7F:B4:D7 192.168.1.1:59943 10.0.100.1:53 (dns) ip:udp 70 0
SRV_11 1.921 4 -> 64:D1:54:7F:B4:D7 E4:5F:01:47:11:10 192.168.1.1:59943 10.0.100.1:53 (dns) ip:udp 70 0
bridge_trunk 1.921 5 -> 64:D1:54:7F:B4:D7 E4:5F:01:47:11:10 11 192.168.1.1:59943 10.0.100.1:53 (dns) ip:udp 74 0
ether10 1.921 6 -> 64:D1:54:7F:B4:D7 E4:5F:01:47:11:10 11 192.168.1.1:59943 10.0.100.1:53 (dns) ip:udp 74 0
Awesome! Those ← and → indicates that the firewall NAT works yes?
Let’s try with query 1.1.1.1 for example and see what sniff will show us.
❯ nslookup google.com 1.1.1.1
Server: 1.1.1.1
Address: 1.1.1.1#53
Non-authoritative answer:
Name: google.com
Address: 216.58.215.110
[konrad@rtpoz1] /tool/sniffer> /tool/sniffer/quick ip-protocol=udp port=53 src-ip-address=192.168.1.1
Columns: INTERFACE, TIME, NUM, DIR, SRC-MAC, DST-MAC, VLAN, SRC-ADDRESS, DST-ADDRESS, PROTOCOL, SIZE, CPU
INTERFACE TIME NUM DIR SRC-MAC DST-MAC VLAN SRC-ADDRESS DST-ADDRESS PROTOCOL SIZE CPU
ether10 3.343 1 <- AC:8B:A9:39:65:31 64:D1:54:7F:B4:D7 4040 192.168.1.1:50952 1.1.1.1:53 (dns) ip:udp 74 0
bridge_trunk 3.343 2 <- AC:8B:A9:39:65:31 64:D1:54:7F:B4:D7 4040 192.168.1.1:50952 1.1.1.1:53 (dns) ip:udp 74 0
InterVLAN 3.343 3 <- AC:8B:A9:39:65:31 64:D1:54:7F:B4:D7 192.168.1.1:50952 1.1.1.1:53 (dns) ip:udp 70 0
SRV_11 3.343 4 -> 64:D1:54:7F:B4:D7 E4:5F:01:47:11:10 192.168.1.1:50952 10.0.100.1:53 (dns) ip:udp 70 0
bridge_trunk 3.343 5 -> 64:D1:54:7F:B4:D7 E4:5F:01:47:11:10 11 192.168.1.1:50952 10.0.100.1:53 (dns) ip:udp 74 0
ether10 3.343 6 -> 64:D1:54:7F:B4:D7 E4:5F:01:47:11:10 11 192.168.1.1:50952 10.0.100.1:53 (dns) ip:udp 74 0
Damn! Looks like it is still working.
But why the sponsored links work?
Is the DNAT rule not working at all?
I doubt that because remember that first rule with 192.168.0.20 ? This is my TVoIP from my ISP operator. To have it work it needs special DNS provided by the operator because it seeks for special FQDN that is only resolved on that DNS. Then TV is working.
Although I didn’t checked if I set on the VLAN_ETH, where TV is connected, providing on the DHCP additional DNS just like I did on the WIFI DHCP server will work. But to be honest I don’t have to because All devices on VLAN_ETH are with 10.0.100.1 and to make exception only for one host from that network which is the mentioned TV (192.168.0.20) I added the first rule before those two and TV works. Which means although TV decoder has 10.0.100.1 it forwards DNS outside. So what I’m missing ?