ZeroTier on Mikrotik – a rosetta stone [v7.1.1+]

RouterOS Useful user articles
21

Let me reiterate one point first: If you can enable IPv6 on your WAN, ZT will preferably use IPv6. This avoids any hole punching on the Mikrotik for IPv4 ZT VL1 tunnels. Obviously not always possible, so to answer the questions…


The point was about if you have another router between the Mikrotik with ZT and the internet. Say an ISP router in-between you can’t remove, and it’s that route that may support uPnP or NAT-PMP – and on that non-Mikrotik router, you might want to make sure either one of those protocols is enabled to help ZeroTier enable a more direct path though the intermediate router. e.g. sometimes other routers have some UI/checkbox to enable UPnP and/or NAT-PMP support, which be good to enable to aid ZT path discovery.

If the Mikrotik has a direct route to the internet (even if behind a NAT), no additional steps are needed (beyond the firewall filter and/or NAT rules discussed in docs and above).



That should be unnecessary if the goal is allow Mikrotik ZT tunnel to make a tunnel to the ZT network.

What the above rules allow… and if the Mikrotik is the default gateway for a ZT network e.g. the ZT network at my.zerotier.com has a 0.0.0.0/0 route to the Mikrotik’s ZT IP address. Is if say you have some desktop/PC/laptop/smartphone that is a member of the same ZT network…and the client has “Allow Default Router Override” set so the Mikrotik is this desktop route to the internet… Then the above UPnP rule will allow that desktop, connect to same ZT network as Mikrotik, to punch holes in the Mikroitk firewall. UPnP is typically done by games and SIP clients, so that allow it happen from members of the ZeroTier network, who were using the Mikrotik as the default gateway to internet.

A different use for uPnP being enable on the Mikrotik is if the Mikrotik LAN if it has users that also have ZeroTier clients.

/ip upnp interfaces add interface=WAN type=external
/ip upnp interfaces add interface=LAN type=internal

Since these non-Mikrotik ZT client on the local LAN may use additional/different ZT networks than the Mikrotik, those may have different paths needed. If uPnP is enabled for the LAN interface list, then those desktop/smartphone ZeroTier client will attempt UPnP to form their own paths to various ZT networks. Since UPnP is deterministic on the assigned port it likely slightly quicker to establish tunnels since it doesn’t have to test using various port. Basically UPnP will short-circuit more complex hole punching techniques. As a bonus, if LAN ZT client’s tunnels will show up as dynamic entires in the firewall with comments – so the ZT tunnels from LAN become more “visible” to the admin. Otherwise, an admin finding VL1 tunnels in a network is non-determistic (see the complex script in #2 above to find peers in firewall on the Mikrotik, and it’s impossible to do for LAN client’s ZT tunnels).


No. But the ZeroTier client on Mikrotik may try it when establishes paths to next-hops beyond the current Mikrotik running ZT. Similar to above, it just an option ZeroTier’s ZL1 tunnels may try when establishing paths.


Hmm. I’m of the thought that if action=endpoint-independent-nat is needed for something, someone else made some poor design decision someplace. So not thought about its interaction with ZT. But ZT has its own complex logic to deal with NAT hole punching, so thinking it’s best not mess with their logic by remapping things. It really shouldn’t be needed since ZT will likely do something reasonable without ANY kind of port forwarding etc.

Now since Mikrotik didn’t support being a ZT controller when I wrote this article, I cannot say how rules involving “action=endpoint-independent-nat” would interact with ZT controller side… But since ideally ZT use one port (default 9993), just making sure that port was available on a controller should be all that’s need. So worse case, you might need some dst-nat upstream in more complex topology. But the how ZT controller work be another article…

22

Excellent as always, @Amm0!

  • I was referring to an MT router with a direct route to the internet (no NAT, Public IP assigned), assigning ZT interface to LAN interface-list was enough for me
  • Unfortunately no IPv6 (still…)
  • I can confirm that UPnP enabled on ZT interface is creating a dynamic roule in NAT, when using a device with Allow Default route enabled
23

Hi all, trying to learn more about ZT, read couple times the very well written guide by @Ammo (thanks again!), what I would like to achieve is in the middle between use case C and D, bridge single VLAN to another MikroTik device, to be able to reach an IoT device connected to Router B (behind NAT) from main router A.
This is what I did;

  • both devices with a same configuration and same VLANs
  • ZT as bridge port with VLAN ID
  • ZT untagged in bridge VLANS
  • added correct route in ZT portal
  • selected Bridged on both router and not assigning any IP
  • disabled DHCP on router B

Following all the instructions from use case C and, with ZT client on Windows, I’m able to ping the IoT device connected to Router B that got IP from DHCP on Router A.
What’s strange and probably due to a misconfiguration is, doing a speedtest on router B is very slow and is using public IP from Router A.
Allow-Managed=yes|no makes no difference.
Without ZT added to the bridge on Router B I was not able to ping device IoT from router A.

Spent couple days without success, should be nice to have the same behavior that I find on Windows client on Router B.

Make sense? Any suggestion?
Many thanks

24

The “middle” starts getting complex :wink:…

If I get the basic setup… you have some VLANs on one router…and different set of VLANs on another router… For simplicity, let assuming “MAIN”, “GUEST”, “IOT” on each. And MAIN on Router A wants to access IOT on Router B.

My first question is do you want the IOT on Router A to be the same network Router B? OR, do you want to keep Router A and Router B’s IOT VLAN as different networks?

If it the later… and you just one of those VLAN to be same on both side, Option C right – IOT VLAN is same on both side (bridged) & any device can “see” any other device across the WAN. The access from your MAIN VLAN is really up to the firewall rules… but assuming MAIN is allowed to access local IOT VLAN, and ZT bridge that network to other side’s IOT VLAN (using Option C). But essentially you end up one “IOT” VLAN that same network on both sides.

If you want the former (e.g. different IOT LANs on both sides, just Router A can access Router B IOT VLAN only)… In this case, you essentially might want to create a new VLAN interface called “IOT-B” and add it to Router A & untagged the ZeroTier to that IOT-B VLAN (e.g. set PVID). On Router B just need to set the PVID= on the ZeroTier bridge port to the IOT VLAN on Router B. This essentially add a new VLAN to Router A. Still same rough steps as Option C…

“Option D” describes a VLAN “trunk”, but that means that non-Mikrotik ZeroTier clients cannot connect in this mode – the client don’t untag VLANs, so that’s approach isn’t quite right.

But you may have gotten the “Option E” :wink:…

“I guess you can use both untagged traffic and tagged over same ZeroTier network…e.g. a hybrid port BUT…
This is more complex since you now DO still have to with the IPv4 auto-assign/DHCP stuff that’s been described at length above…”

It’s actually not that complex… but it is context specific & the more you understand how Bridge VLANs work, the more sense it makes…

Anyway, if you can provide your configs that’s help. And better explanation of what VLANs should map where…

25

OK so, an example of config (I don’t have the export with me now but it’s pretty much based on MT KB and pcunite’s guide :

/interface bridge add name=Bridge vlan-filtering=yes frame-types=admit-only-vlan-tagged

/interface vlan
add interface=Bridge name=LAN vlan-id=10
add interface=Bridge name=GUEST vlan-id=20
add interface=Bridge name=IoT vlan-id=30
add interface=Bridge name=MGMT vlan-id=40

/ip address
add address=192.168.1.0/24 interface=LAN
add address=172.16.1.0/25 interface=GUEST
add address=10.0.0.1/24 interface=IoT
add address=192.168.0.1/26 interface=MGMT

/interface bridge port
add bridge=Bridge interface=ether2 pvid=10 frame-types=admit-only-untagged-and-priority-tagged
add bridge=Bridge interface=ether3 pvid=20 frame-types=admit-only-untagged-and-priority-tagged
add bridge=Bridge interface=ether4 pvid=30 frame-types=admit-only-untagged-and-priority-tagged
add bridge=Bridge interface=ether5 pvid=40 frame-types=admit-only-untagged-and-priority-tagged
add bridge=Bridge interface=ZeroTier pvid=30 frame-types=admit-only-untagged-and-priority-tagged

/interface bridge vlan
add bridge=Bridge tagged=Bridge untagged=ether2 vlan-ids=10
add bridge=Bridge tagged=Bridge untagged=ether3 vlan-ids=20
add bridge=Bridge tagged=Bridge untagged=ether4,ZeroTier vlan-ids=30
add bridge=Bridge tagged=Bridge untagged=ether5 vlan-ids=40

Only difference between two Routers is Router A is using PPPoE with public IP and Router B is using DHCP-Client with NATted IP for WAN.


Don’t want to create confusion but, at the moment they share the same network for semplicity, Router A has devices connected only on LAN and GUEST, Router B is using IoT.


This is what I did, following your instructions with the only difference that ROUTER-A setup is also replicated to Router B, bridging ZeroTier as untagged port.
It’s working, what I can’t understand is why all the traffic from Router B is “routed?” to Router A (like a VPN tunnel), where I mostly intrested accessing IoT devices connected to Router B from LAN on Router A.


As soon as everything works I don’t mind having tagged or untagged traffic as ZeroTier clients are not needed (tested also “Option D” and finding the same behaviour of “Option C”), just want to connect those MikroTik Routers :slight_smile:

Hope everything is clear, thanks @Ammo!

26

I got lazy writing this towards the end is also part of the problem here :wink: – I was going to take the @pcunite’s VLAN as an example, but it was already pretty long.

I think your problem is that you’re using “Allow Managed” and/or the bridged VLAN’s IP address are duplicates on both side…once bridged. But rather than guess perhaps a more simple “checklist” for ZeroTier bridging be useful for how it works with vlan-filtering=yes (i.e. “@pcunite-style”) Bridge VLAN scheme…


On the Mikrotik side,

  • make sure “Allow Managed” addres is unchecked/=“no” on both routers – this is critical… Otherwise, it will create a “connected route” for any ZT assigned address (same as RouterOS does for any other interface when an IP address is added to interface). You don’t want this since RouterOS already has a routing table & the bridged VLAN already has an IP for each router (e.g. the one assigned the IOT VLAN) – so you don’t need ZT adding a 2nd IP to same interface (which is what happens with “Allow Managed Address” is enabled/checked)

  • make sure both routers have a unique IP address assigned to the VLAN on each side (e.g. 10.0.0.2/24 on RouterA and 10.0.0.1/24 on RouterB) & the associated DHCP server network for IOT should use that particular router’s IOT VLAN’s IP as the gateway (e.g. 10.0.0.1 or 10.0.0.2). Otherwise, there be IP address conflict once ZT is connecting both side & default gateway given to clients may be wrong

  • bridge port for zerotier1 interface uses the “right” frame-types= and pvid= So for bridge one VLAN, frame-types= is “admit-only-untagged…” and pvid= match what you want as the network shared via non-Mikrotik ZeroTier clients. To bridge multiple VLANs…whether you use frame-types=admit-all or frame-types=admit-only-vlan-tagged depend on if you want non-Mikrotik clients, i.e. need a hybrid port (frame-types=admit-all) if desktop/smartphone will connect & you want have VLAN tagged traffic over ZT network. Or, if ZT is used ONLY for trunking VLANs across a WAN, then frame-types=admit-only-vlan-tagged is what you’d want — in either hybird or truck case...just like any other hybrid/trunk bridge port for ethernet…you’d need to define the zerotier1 port as tagged= in /interface/bridge/vlan for the tagged VLAN(s) & adjust the flow rules as described in Option D in my main writeup

  • only one side should have dhcp-server enabled since with ZT bridging. Otherwise, you’d end up with multiple DHCP servers run on same LAN…not necessity going to break anything, but potentially confusing/error-prone — e.g. DHCP server configuration have to be identical & you’d want to ensure “Conflict Detection” is enabled in /ip/dhcp-server on both.

  • dhcp address pool (e.g. /ip/pool) for ZT-bridged VLAN should be adjusted so that there is some range for ZT clients to use (that be assigned by ZT to desktop/smartphone). In theory, it doesn’t matter if DHCP server use the “Conflict Detection” but better just to prevent ZT from using same IP address in the same subnet – the main writeup at top describes how to adjust the pool in more detail

  • beyond the scope here… but VRRP could used so that if the WAN is lost, each RouterOS’s IOT VLAN could operate independently during WAN outage (since dhcp-server listen on the VRRP, and use VRRP address as the gateway).


    On the ZeroTier controller side (my.zerotier.com),

  • in the routes, remove any existing ones & add one for the the bridge LAN subnet by using just 10.0.0.0/24 with no gateway – this essentially defines the scope for non-Mikrotik clients. For bridging, you don’t need any other routes. Although you can add them if you wanted to allow ZeroTier desktop/smartphone clients access other networks, as they need routes defined in ZT controller/portal) – again you do NOT want “Allow Managed” on RouterOS since it [should] knows these routes already.

  • whether it’s an access port or hybrid port, the auto-assign IP address range should be same subnet as what’s bridged – but critical it uses the same subnet as the VLAN that’s being bridged (e.g. say 10.0.0.201-10.0.0.249 but whatever as long as different/non-overlapping range with the /ip/pool). Since a true VLAN trunk (e.g. nothing untagged) will not allow non-VLAN aware ZT clients, the auto-assignment should set to something that’s NOT any subnet you used, to avoid routing issues – but it doesn’t matter since there be only tagged traffic & ZT will passes those along in Layer-2, but it has NO VLAN helpers or anything to “untag” any of them

  • both Mikrotik should have the bridging enabled under the “Members” section using the gear (as again described more fully at top of thread) – nothing else should have that defined

  • if you have no tagged[/i] VLANs passing over ZT (e.g. frame-types=admit-only-untagged-and-priority on the bridge port, and correct pvid= set), then you can ignore the “Flow Rules”. But for Hybrid or Trunk ports (no untagged traffic), you’ll need to make sure you allow the VLAN “ethertype” in the flow rules – this is not the default & will block any tagged VLANs from flowing through the ZeroTier network.

27

Followed your instructions a small step forward…
In my specific case VLAN’s IP address were duplicates on both side…(I feel so dumb!)

1st “Bridged Test”:
[on Routers]
✓ Created ZeroTier interface on both Routers (A - B) with Allow Managed unchecked
/zerotier interface add allow-default=no allow-global=no allow-managed=no instance=zt1 name=ZeroTier network=“”
✓ Created a Bridge with all ports as my last post assigning PVID / VLAN-ID and frame-types=admit-only-untagged-and-priority-tagged
✓ Unique IP address assigned to the VLAN on each side
on Router B /ip address set IoT address=10.0.0.2/24 and …dhcp-server gateway=10.0.0.2
✓ I ended up enabling DHCP Snooping /interface bridge set Bridge dhcp-snooping=yes (losing Bridge Hardware Offload) instead of having DHCP anabled only on Router B (where the IoT devices are connected, in my case), this is probably not correct but was the solution to avoid all traffic passing trough the other Router.
✓ “Conflict Detection” was enabled by default in /ip/dhcp-server, DHCP Address Pool adjusted as suggested, just in case…
[on ZeroTier portal]
✓ Checked “Bridged” on both Routers ID under “Members”
✓ Removed any “Route”
✓ Disable assigning IP
X I’m able to ping IoT device from Router A connected to Router B via ZeroTier network with bridged interface only from the Router console… probably due to picky firewall rules, this must be figured out, at lest is something, better than nothing :slight_smile:

2nd “Unbridged Test”:
[on Routers]
✓ Created ZeroTier interface on both Routers (A - B)
/zerotier interface add allow-default=no allow-global=no allow-managed=yes instance=zt1 name=ZeroTier network=“”
✓ Added ZeroTier as LAN in interface list
[on ZeroTier portal]
✓ Created a new Newtork, defining Route [ 10.0.10.0/24 via (LAN) =Empty ] and Auto-Assign IPv4 Range [ 10.0.10.3 - 10.0.10.10 ]
✓ Assigned “Managed IPs” for both router just for convenience ( Router A=10.0.10.1 | Router B=10.0.10.2 )
X Still able to ping IoT device connected to Router B from Router A, only from console, using ZT PC clients is working as expected (previous ZT interface created for test was disabled)

I’ll continue trying to solve the problem to be able to reach device on Router B from PC connected to router A via ZeroTier network, what are the advantages from using “Use Case C” (my 1st Test) over “Use Case A” (my 2nd Test)?

Thanks again.

28

On 1st “Bridged Test”…

If both routers can ping the other route’s VLAN 30 IP from the console, ZeroTier should be working.
Assuming it bridged correctly (seem like it is), then should just follow firewall rules for VLAN 30…
So not sure it’s the firewall – benefit of bridging… it should follow the VLAN of it’s assigned pvid=, so NEW firewall rules should be needed.

Re DHCP, you may have to the zerotier1 bridge port as “Trusted” for DHCP to pass, that might be simpler.

If you posted a sanitized version of both router’s config, that help here. And perhaps output of /routing/route/print.


On 2nd “Nonbridged Test”… (aka “Routed” approach)
This one I’m not sure be is useful IF one is coming from “@pcunite VLAN” setup… Since it involves using IPv4 routing (“subnets”/“prefixes”), vs thinking in VLANs and Layer-2 “interfaces”… And this approach also involves creating new firewall rules, too…

The basic theory is using a SINGLE ZeroTier network that’s connected all other routers. That gets ALL router on the same IP subnet – e.g. so the routers can “ping” each other. But that doesn’t help any other traffic like VLAN – since you need have /ip/routes to that use the fact all routers are connected via ZT… One way is do it statically — that mean using “/ip route add” with the other router’s ZT address as the gateway= and dst-address= the IP range/prefix associated with remote VLAN – for any subnet you wanted.

Instead, you actually define routes to some/all your VLANs in my.zerotier.com. Setting the routes there gets more tricky. But these ZT defined routes will be added by checking the “Managed Address” (which includes any routes at my.zerotier.com). Useful, but by default problematic… since you have to set the “Route Distance” property of zt1 instance to at least 2 (or higher). If you don’t set route-distance=2 on “zt1” instance… problems can happen. More specifically, very easy to creat ECMP/load balance route to a VLAN when route-distance=1 and your routing an existing VLAN…

And while ZeroTier will allow multicast (e.g. mDNS/SSDP/etc) over its network (unlike ZeroTier), /ip/route isn’t going get deal with multicast without using IGMP-Proxy/PIM/etc. (IF multicast is needed) e.g. LAN traffic is NOT “bridged”, but “routed” via ZeroTier’s switch to another router on the “ZeroTier Cloud Switch” (e.g. what’s defined at my.zerotier.com).

My thought is when you start with VLANs…ZeroTier just extends these – so “non bridged” case changes the model you’re using in a routed approach, when the firewall/etc is designed around VLAN and interface-list.

29

Appreciated your assistance, thanks @Amm0!
Spent days trying to figure out what’s wrong, had mixed and inconsistent results.
Today I noticed that the ARP table was not populated with IP and MAC from device at the other side (Router B) - Bridge Test, adding it helped to succeed with Ping instead of “unreachable”.
It may need some sleep and start from scratch following each step carefully.

Your guide and all institutions to Bridge (as Untagged VLAN or Trunk) are pretty clear, just thinking in case of a second remote device (Router C) will be added in the future, bridging the same IoT VLAN, having just one subnet (10.0.0.1 R-A | .2 R-B | .3 R-C) /24 is suggested or a smaller one (e.g. /26 + adding Routes) will help avoiding conflicts?

30

Zerotier bridging works great.

But as soon as i join two MK-routers within the same local lan to the same bridged ZT Network,
i loose connection to both!
When I DENY one of those routers on the ZT-controller, ping comes back as usual.
There must be somthing extra needed…

What i wanted to achieve ist if my main routers Internet goes down, the (second) LTE router should allow ZT access.
It works if i tic the “Authorize” in the ZT-controller then, but id would be nice to be up all the time

31

That can work. On both Mikrotik routers bridged to ZeroTier, do you have “Managed Address” unchecked on the ZertoTier? And at my.zerotier.com BOTH router are marked as “Bridged” in the options next to the router in “Members” section (exposed by wrench icon)?

Another possibility is the bridge has loop detection…so possible RSTP might be tripping…and it generally harmless to disable. To do this, you go your Bridge interface on each router, and select “None” as the “Protocol Type” in the STP tab. Alternatively, you can try using a higher path cost (say path-cost=20) for the zerotier bridge ports in each router (in Bridge > Ports > STP).

32

One question, are the routers physically connected via ethernet? That changes the story a bit…since it’s be even more likely STP is coming into play.

One thing you can do is look at the bridge and see the forwarding state (“role”) of the port. The zerotier1’s bridge port role should not be “disabled”, when you have both ZT interface up… if it is STP is kicking in…

33

Several points are not entirely clear. 1. About the compatibility of Mikrotik and LTE versions. If Mikrotik is not an Arm architecture, but MMIPS, but an LTE modem is inserted into it, will such a combination work?
2. If I understand correctly, without the central ZeroTier cloud it is impossible to create your own network. For example, if I created such a network by registering on their website, and then their website stopped working, or for some reason the Internet fell apart into pieces in several countries, will the ZeroTier network I created continue to work without the main cloud?
3. Without a special application, the device will not be visible on the ZeroTier network. Does this mean my IoT devices will remain unavailable?

34

1- no. ARM/ARM64 devices only. That point is quite clear.

  1. It may, it may not. Depends if a path can be found using the parts still left operational :laughing:
    But you can always host your own zerotier cloud controller …

  2. Yes and no. ONE part on that network needs to be zerotier-capable. From there on you can hop to other devices within that same network segment.
    I use as example an RB5009 with my customer to ROMON into devices on 28 locations all connected via EOIP over MPLS to that RB5009.
    Only that RB5009 has Zerotier installed (and Wireguard, I like a backup plan).
    With that same customer I have an AC3 LTE, also using Zerotier (it’s not part of that MPLS network), so I can always reach it, even when VDSL line is down (also there, wireguard backup).

35

This is true & the feature come out AFTER I wrote his article. The story is bit more complex — since there are still root servers — but it’s pretty well described in Mikrotik’s docs:
https://help.mikrotik.com/docs/display/ROS/ZeroTier#ZeroTier-Controller


It’s the term “without special attention” that confuses the question.

If ZT interface is bridged to desired VLAN/etc, then IoT/etc devices can span physical sites and operate the same.

The main caveat (and what’s described in article) is the some attention MAY be need in the “Flow Rules” — but ONLY IF your devices use non IP/IPv6 protocols. For example, RoMON does use a different ether-type, so by default ZT controller would block…but small change to flow rules would allow RoMON, then it work just fine.

Also, it’s also fair to mention that latency sensitive things may not be ideal to run across ZeroTier Network, so that be another area of “special attention”. And ZeroTier likely add more latency than some point-to-point VPNs (e.g. WireGuard). Essentially ZT trades latency (using potentially slower indirect path) to work across more diverse/restricted networks (e.g. CGNAT’s).

36

Hi,

I am running ZeroTier on a MikroTik ARM router. The router is behind a T-Mobile 5G gateway with CGNAT - no IP6 provided. I added a basic ZT Network with a managed route added for my internal network 192.168.0.X via the MT ROuter address on the ZT network. I added the ZT interface to the interface list (LAN) as recommended in the documentation. I then turned on a cell phone hotspot and connected my laptop. I am running a windows ZT Client. I can ping but I am getting many dropped packets. In fact sometimes I have to try the ping 5 or 6 time before it works and then it will work for the next 5 tries. If I try to use Remote Desktop to a server on my internal network - it start to work but then times out very quickly.
My T-Mobile connection is very solid and it is not a latency issue. My speedtest pings are about 32ms.

Any thoughts. If you have any ideas please provide detailed instructions - I am not super skilled on MT or ZT.

Thanks

37

You might want to try change the “route distance” on the “zt1” instance under zerotier > instance tab to “10”. Otherwise it’s possible 192.168.0.0/24 route might be load balanced if the ZeroTier added one’s distance (default is 1) match the distance of that internal/Mikrotik version route.

But you have a double NAT situation, worse still with MULTIPLE ZeroTier client trying to use the default ports…that gets problematic… If there is any way to get IPv6 enabled on you mobile account, that be best solution – may not be possible, but that be best. Is there an options for a “DMZ” on the T-Mobile gateway, or can you map ports on the T-Mobile gateway?

38

Hi - changing the instance tab to 10 seems to have made it more reliable. It is problematic the first try or two and then just works solid. What does that do?
Regarding T-Mobile - I don’t think they will provide an IPv6 and there is no control over the router they provide to create a DMZ. I will call today to find out.

Thanks

39

If you look at /ip/route, you’ll notice that the 192.168.0.0/24 is likely repeated (one for bridge/etherX, one from “zerotier1”). When there are multiple routes for same subnet, the LOWEST distance wins. BUT if there are two subnet, at same distance…then routing is split 50%/50%…so changing the “zt1” to distance of 10 means it has a lower priority.

But it could just be port mapping through the CGNAT isn’t conflicting & not the route distance. Time will tell.

Since ZeroTier can use IPv6 to establish the tunnels to the ZeroTier planets/root servers… you wouldn’t have the same potential for port mapping issues. But the underlying ZeroTier tunnel can still use IPv4 inside, nothing else has to change IF you can get T-Mobile to enable IPv6.

40

Hello Amm0,

Great Post, very detailed and a goldmine of information.

I have a scenario where I need to create a Site to site Layer 2 tunnel, either between two mikrotiks, or a mikrotik and Linux server.
for now I tried only the MT-Linux version, but it does not really work well.

I think I got my configs right, as I can ping, get UDP broadcast traffic, and it would work as I expect but …

By default the connection without any traffic has maybe a few packets, just a little idle traffic of 100 kilobytes, then suddenly out of nowhere, boom. Tx jumps to 150Mbps, while Rx is mostly 0. I can see this on the LAN Bridge and the ZT interface, …
This goes on for a few seconds, and it drops back to idle, 4.4 kbps Tx, 3-4 p/s Tx, mostly 0 bps Rx.
Then back at 150Mbps after 10 seconds, and so on, in completely random time intervals.

This happens if there is only the WAN port connected, and Zt is active, once the Zt interface is deactivated the effect stops. …

I wonder if this might be related to some version mismatch, as the Mikrotik is reported to be ZeroTier version 1.10.3 while every other OS reports ZeroTier version 1.12.2.

It looks like MT is trying to push something through the tunnel, like crazy. On the other end of the line, this is not received.

Any ideas, or what to check, what to look at, I would appreciate it. Thanks.

I have tried with the MT-MT version, where both sides use 1.10.3, and got the same …