Tailscale now NAT-PMP problem

RouterOS General
1

----please see later posts-----

# 2024-10-09 20:58:15 by RouterOS 7.14.2
# software id = 4PWK-AH73
#
# model = CCR2004-1G-12S+2XS
# serial number = XXXXXXXXXXXX
/interface bridge
add admin-mac=D4:01:C3:C2:36:AA auto-mac=no comment=defconf name=bridgeLocal
/interface list
add name=WAN
add name=LAN
/ip pool
add name=dhcp ranges=192.168.8.10-192.168.8.250
/ip dhcp-server
add address-pool=dhcp interface=bridgeLocal name=dhcp1
/port
set 0 name=serial0
set 1 name=serial1
/interface bridge port
add bridge=bridgeLocal comment=defconf interface=ether1
add bridge=bridgeLocal comment=defconf disabled=yes interface=sfp-sfpplus1
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus2
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus3
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus4
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus5
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus6
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus7
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus8
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus9
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus10
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus11
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus12
add bridge=bridgeLocal comment=defconf interface=sfp28-1
add bridge=bridgeLocal comment=defconf interface=sfp28-2
/interface list member
add interface=sfp-sfpplus1 list=WAN
add interface=bridgeLocal list=LAN
/ip address
add address=192.168.8.1/24 interface=bridgeLocal network=192.168.8.0
/ip dhcp-client
add comment=defconf interface=bridgeLocal
add interface=sfp-sfpplus1
/ip dhcp-server network
add address=0.0.0.0/24 dns-server=0.0.0.0 gateway=0.0.0.0 netmask=24
add address=192.168.8.0/24 dns-server=192.168.8.1 gateway=192.168.8.1 netmask=24
/ip dns
set allow-remote-requests=yes
/ip firewall nat
add action=masquerade chain=srcnat out-interface-list=WAN
/ip upnp
set enabled=yes show-dummy-rule=no
/ip upnp interfaces
add interface=bridgeLocal type=internal
add interface=sfp-sfpplus1 type=external
/system clock
set time-zone-name=Asia/Shanghai
/system identity
set name=MikroTikRT
/system note
set show-at-login=no
/system routerboard settings
set enter-setup-on=delete-key

It seems upnp works fine:

Flags: X - disabled, I - invalid; D - dynamic 
 0    chain=srcnat action=masquerade out-interface-list=WAN 

 1  D ;;; upnp 192.168.8.196: tailscale-portmap
      chain=dstnat action=dst-nat to-addresses=192.168.8.196 to-ports=58605 protocol=udp dst-address=192.168.1.3 in-interface=sfp-sfpplus1 dst-port=62714 

 2  D ;;; upnp 192.168.8.195: tailscale-portmap
      chain=dstnat action=dst-nat to-addresses=192.168.8.195 to-ports=65223 protocol=udp dst-address=192.168.1.3 in-interface=sfp-sfpplus1 dst-port=16358 

 3  D ;;; upnp 192.168.8.196: tailscale-portmap
      chain=dstnat action=dst-nat to-addresses=192.168.8.196 to-ports=58605 protocol=udp dst-address=192.168.1.3 in-interface=sfp-sfpplus1 dst-port=2038 

 4  D ;;; upnp 192.168.8.198: tailscale-portmap
      chain=dstnat action=dst-nat to-addresses=192.168.8.198 to-ports=41641 protocol=udp dst-address=192.168.1.3 in-interface=sfp-sfpplus1 dst-port=36987 

 5  D ;;; upnp 192.168.8.198: slcc
      chain=dstnat action=dst-nat to-addresses=192.168.8.198 to-ports=15692 protocol=udp dst-address=192.168.1.3 in-interface=sfp-sfpplus1 dst-port=1631 

 6  D ;;; upnp 192.168.8.196: tailscale-portmap
      chain=dstnat action=dst-nat to-addresses=192.168.8.196 to-ports=51825 protocol=udp dst-address=192.168.1.3 in-interface=sfp-sfpplus1 dst-port=19102 

 7  D ;;; upnp 192.168.8.196: tailscale-portmap
      chain=dstnat action=dst-nat to-addresses=192.168.8.196 to-ports=54233 protocol=udp dst-address=192.168.1.3 in-interface=sfp-sfpplus1 dst-port=35661 

 8  D ;;; upnp 192.168.8.196: tailscale-portmap
      chain=dstnat action=dst-nat to-addresses=192.168.8.196 to-ports=55447 protocol=udp dst-address=192.168.1.3 in-interface=sfp-sfpplus1 dst-port=16218

but direct connect cant be made, and i see no network activity for those upnp connections.

2

The explanation of your WAN is confusing.
You state an ISP modem is feeding the MT router but then you state the MT is getting a DMZ IP…??
If its a modem, the MT router would get a public IP, would it not??

If its a modem/router then getting a private IP for WAN would make more sense…

What does tailscale have to do with UPNP?
Why is UPNP required at all on your router?

3

modem pppoe-out, mt router connects to the modem has 192.168.1.2, on modem sets dmz to it.

i need upnp for tailscale. it comes with randomize port (I set for a specific reason) so any forwarding needs to be done automatically

4

i sense it is some upnp protocol issues, i swap the MT with a OpenWrt it also works just fine.

i need some troubleshoot steps to start with.

5

This is all very mysterious.

/ip dhcp-client
add comment=defconf interface=bridgeLocal
add interface=sfp-sfpplus1

Why also on bridge?

And generally: why is there srcnat rule as it is clearly not the gateway/edge router? Why should there be UPNP do anything? That is not the intention of upnp. This cant work behind a modem that is apparently not in bridge mode but more likely acting as gateway itself.

6

i guess it is confusing.

my isp wont allow the modem to run in bridge mode, but on modem it has DMZ, so i set the DMZ their to MT.

all my LAN are on bridge, the sfp1 is WAN connects to modem, has a local IP 192.168.1.2.

MT is the gateway for my PCs, my pc has 192.168.8.x

other router like openwrt or ASUS works fine if I point the DMZ on modem to the router.

7

/ip dhcp-client
add comment=defconf interface=bridgeLocal

it is by default but i delete it now, there isn’t any other dhcp server on the local net.

8

edit out

9

22:48:56 upnp add mapping: proto:UDP, ports:2564->15570, internal ip:192.168.8.195
22:49:02 upnp set mapping: proto:UDP, ports:2564->15570, internal ip:192.168.8.195
22:49:07 upnp set mapping: proto:UDP, ports:2564->15570, internal ip:192.168.8.195
22:49:12 upnp set mapping: proto:UDP, ports:2564->15570, internal ip:192.168.8.195
22:49:18 upnp set mapping: proto:UDP, ports:2564->15570, internal ip:192.168.8.195
22:49:23 upnp set mapping: proto:UDP, ports:2564->15570, internal ip:192.168.8.195
22:49:36 upnp add mapping: proto:UDP, ports:33189->61125, internal ip:192.168.8.195
22:50:08 upnp add mapping: proto:UDP, ports:24493->50469, internal ip:192.168.8.195
22:50:08 upnp remove mapping: proto:UDP, ports:24493->50469, internal ip:192.168.8.195

it seems right with whats in the nat:

D ;;; upnp 192.168.8.195: tailscale-portmap
chain=dstnat action=dst-nat to-addresses=192.168.8.195 to-ports=61125 protocol=udp dst-address=192.168.1.3 in-interface=sfp-sfpplus1 dst-port=33189


that being said, i dont know what to troubleshoot, i didn’t have PMP on, and PCP seems not support on MT.





above is the log of upnp.

10

i add a manual nat rule,


chain=dstnat action=dst-nat to-addresses=192.168.8.195 to-ports=61125 protocol=udp in-interface=sfp-sfpplus1 dst-port=61125 log=no log-prefix=“”

and it just works, i figure the upnp on MT sets dst-port and to-port differently is the problem (ASUS router is the same), how can i change that?


i mean i can always add NAT myself, but everytime tailscale starts it just changes port..

11

if anyone is interested, i figure out tailscale actually uses NAT-PMP, which seems no go for MT.

no log output in topics, and tailscale debug portmap just flat out nothing useful.

12

https://help.mikrotik.com/docs/display/ROS/NAT-PMP

13

it now reads:

unable to allocate port for mapping *:0 → localip:port, timeout 7200s

i understand tailscale is requesting 0 on the external port? it is really odd, if i swap in another router it works fine, external and internal ports are the same 51xxx

14

It works with default configuration on OpenWrt? Or did you have to configure anything additionally?

15

I turn on nat-pmp it just works.


I look deeper into this:

I can use 3rd tool to create NAT via NAT PMP w/o problem, but tailscale can’t.

What I can read from RouterOS log:

unable to allocate port for mapping *:0 → localip-machineIP:62519, timeout 7200s

It seems tailscale requests a remote port 0 via NAT PMP, which leads to failure.

when I do tailscale debug portmap, it shows the same error like above in routerOS.

tailscale debug portmap:

monitor: monitor: gateway and self IP changed: gw=192.168.8.1 self=192.168.8.196
gw=192.168.8.1; self=192.168.8.196
portmapper: [v1] Got PMP response; IP: WAN IP, epoch: 15133
Probe: {PCP:false PMP:true UPnP:false}
no mapping
serveDebugPortmap: context done: context deadline exceeded

*I have randomize port sets true in ACL, so the port is not regular 41613.


I believe tailscale just behaves this way, and other router like openwrt or ASUS just handles it well.

16

Can you please share an export of your current configuration? (remove serial and other sensitive data if any)

17

[admin@MikroTikRT] > export

2024-10-11 22:21:13 by RouterOS 7.16

software id =

model = CCR2004-1G-12S+2XS

/interface bridge
add admin-mac=D4:01:C3:C2:36:AA auto-mac=no comment=defconf name=bridgeLocal
/interface list
add name=WAN
add name=LAN
/ip pool
add name=dhcp ranges=192.168.8.10-192.168.8.250
/ip dhcp-server
add address-pool=dhcp interface=bridgeLocal name=dhcp1
/port
set 0 name=serial0
set 1 name=serial1
/interface bridge port
add bridge=bridgeLocal comment=defconf interface=ether1
add bridge=bridgeLocal comment=defconf disabled=yes interface=sfp-sfpplus1
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus2
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus3
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus4
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus5
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus6
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus7
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus8
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus9
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus10
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus11
add bridge=bridgeLocal comment=defconf interface=sfp-sfpplus12
add bridge=bridgeLocal comment=defconf interface=sfp28-1
add bridge=bridgeLocal comment=defconf interface=sfp28-2
/interface list member
add interface=sfp-sfpplus1 list=WAN
add interface=bridgeLocal list=LAN
/ip address
add address=192.168.8.1/24 interface=bridgeLocal network=192.168.8.0
/ip dhcp-client
add comment=defconf disabled=yes interface=bridgeLocal
add interface=sfp-sfpplus1
/ip dhcp-server network
add address=0.0.0.0/24 dns-server=0.0.0.0 gateway=0.0.0.0 netmask=24
add address=192.168.8.0/24 dns-server=192.168.8.1 gateway=192.168.8.1 netmask=24
/ip dns
set allow-remote-requests=yes
/ip firewall filter
add action=accept chain=forward connection-nat-state=dstnat
/ip firewall nat
add action=masquerade chain=srcnat out-interface-list=WAN
add action=dst-nat chain=dstnat comment="Tailscale N100" dst-port=61793 in-interface=sfp-sfpplus1 protocol=udp to-addresses=192.168.8.195 to-ports=61793
add action=dst-nat chain=dstnat comment="Tailscale Mac mini" dst-port=61152 in-interface=sfp-sfpplus1 protocol=udp to-addresses=192.168.8.196 to-ports=61152
/ip ipsec profile
set [ find default=yes ] dpd-interval=2m dpd-maximum-failures=5
/ip nat-pmp
set enabled=yes
/ip nat-pmp interfaces
add interface=bridgeLocal type=internal
add interface=sfp-sfpplus1 type=external
/ip upnp
set show-dummy-rule=no
/ip upnp interfaces
add interface=bridgeLocal type=internal
add interface=sfp-sfpplus1 type=external
/system clock
set time-zone-name=Asia/Shanghai
/system identity
set name=MikroTikRT
/system logging
add topics=natpmp
add topics=upnp
/system note
set show-at-login=no
/system routerboard settings


\

frankly after a few days troubleshoot, i think it is a weird tailscale and routerOS issues, hard to say who's to blame.

I use py-natpmp to test, and routerOS creates the nat via natpmp w/o issues, meanwhile tailscale works with other router in the same network envirnment.

18

i spend sometime researching and realize it is tailscale request external port as 0, a way to indicate it does care what external port NAT PMP assigns.

I do this with my asus router, it just assign the same private / local port as external.

I tested this with py-natpmp - https://github.com/jaraco/nat-pmp

python3 natpmp_client.py -u -l 1800 0 60010
PortMapResponse: version 0, opcode 129 (129), result 0, ssec 172158, private_port 60010, public port 60010, lifetime 1800

ASUS just goes ahead and assign it.

while RouterOS takes the “0” as is, ofc port 0 isnt going to work.

I don’t know who should resolve this, it seems “request external port as 0, a way to indicate it does care what external port NAT PMP assigns.” a wide use practice…

19

But if that manual rule worked, it remapped port to itself.
I.e. it should be perfectly equivalent to:
chain=dstnat action=dst-nat to-addresses=192.168.8.195 protocol=udp in-interface=sfp-sfpplus1 log=no log-prefix=“”

20

disable upnp. nat-pmp and upnp may not be active same time I guess.