Zerotier - cannot ping my "other" hEX

I have 2 identical hEX refresh routers online, one is hanging off a 4G router’s LAN, the other is direct public-facing connected by pppOe on a fibre broadband connection.

each one is registered with a zerotier IP (I can see them in zt, and they both are on the same 172.24.0.0/16 subnet.

I can ping each one’s own ZT managed IP from itself, but I cannot ping across to the “other” one’s ZT managed IP.

it must be a firewall problem because I have a note that I was able to do it last week when I set up ZT from a totally blank hEX’s.

please could someone glance at my firewall settings to see why the ping won’t “traverse” the ZT switch

# 2025-10-08 02:05:03 by RouterOS 7.20
# software id = DGH9-QP2N
#
# model = E50UG
# serial number = xxxxx
/interface bridge
add name=bridge1
/interface pppoe-client
add add-default-route=yes disabled=no interface=ether1 name=pppoe-out1 \
    use-peer-dns=yes user=XXXXXX
/ip pool
add name=dhcp_pool0 ranges=172.30.30.50-172.30.30.253
add name=dhcp_pool1 ranges=172.30.30.50-172.30.30.253
/ip dhcp-server
add address-pool=dhcp_pool1 interface=bridge1 name=dhcp1
/zerotier
set zt1 disabled=no disabled=no
/zerotier interface
add allow-default=no allow-global=no allow-managed=yes disabled=no instance=\
    zt1 name=zerotier1 network=XXX
/interface bridge port
add bridge=bridge1 interface=ether2
add bridge=bridge1 interface=ether3
add bridge=bridge1 interface=ether4
add bridge=bridge1 interface=ether5
add bridge=bridge1 interface=zerotier1
/ip neighbor discovery-settings
set discover-interface-list=!dynamic
/ip address
add address=172.30.30.254/24 interface=bridge1 network=172.30.30.0
/ip dhcp-server lease
add address=172.30.30.238 comment="bla" mac-address=\
   X
<<etc, some mac binds>>

add address=172.30.30.60 client-id=1:8:0:27:2f:dc:4b comment=\
    "dah" mac-address=X server=dhcp1
/ip dhcp-server network
add address=172.30.30.0/24 comment="Tris 172-30-30-x serveup" domain=\
    trisdomain gateway=172.30.30.254
/ip firewall filter
add action=accept chain=forward comment="TT-ZT1 fwd-in" in-interface=bridge1
add action=accept chain=forward comment="TT-ZT1 fwd-out" out-interface=\
    bridge1
add action=accept chain=input comment="TT-ZT1 ipt" in-interface=bridge1
add action=accept chain=input comment=\
    "DC-accept established, related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="DC-drop invalid" connection-state=\
    invalid log=yes
add action=accept chain=input comment="DC - accept ICMP" protocol=icmp
add action=accept chain=input comment=\
    "DC-accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment=\
    "DC - drop all not coming from Bridge (==LAN/TT)" in-interface=!bridge1 \
    log=yes log-prefix="DROPPED-not traffic from bridge"
add action=accept chain=forward comment="DC-Accept in ipsec policy" \
    ipsec-policy=in,ipsec
add action=accept chain=forward comment="DC-accept out ipsec policy" \
    ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="DC - fasttrack" \
    connection-state=established,related hw-offload=yes
add action=accept chain=forward comment=\
    "DC-accept established, related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="DC-drop invalid" connection-state=\
    invalid log=yes log-prefix="DROPPED - invalid"
add action=drop chain=forward comment=\
    "DC-drop all from ether1(==WAN\?\?/TT) not DSTNATed" \
    connection-nat-state=!dstnat connection-state=new in-interface=ether1 \
    log=yes log-prefix="DROPPED-not DSTNATed"
add action=accept chain=input comment="TT Allow Winbox" disabled=yes \
    dst-port=8291 protocol=tcp
add action=accept chain=input comment="TT allow established traffic" \
    connection-state=established,related disabled=yes
add action=accept chain=input comment=\
    "TT Accept UDP DNS requests from Bridge" disabled=yes dst-port=53 \
    in-interface=bridge1 protocol=udp
add action=accept chain=input comment=\
    "TT Accept TCP DNS requests from Bridge" disabled=yes dst-port=53 \
    in-interface=bridge1 protocol=tcp
add action=drop chain=input comment="TT drop everything else" disabled=yes
/ip firewall nat
add action=masquerade chain=srcnat out-interface=pppoe-out1
/system clock
set time-zone-name=Europe/London
/system identity
set name=HV-Tik-hEX
/system ntp client servers
add address=0.uk.pool.ntp.org comment="UK NTP connexion"

Why don't you use any interface lists, especially LAN and WAN ?

You have to allow zerotier interface on forward chain in your firewall.
Usually this is done with interface lists (and then the easiest is to directly add zerotier to LAN). I have an interface list VPN which contains wireguard and zerotier and work from there.

But since you selected another approach in your firewall, you have to do it otherwise.
Easiest is to simply add an accept rule on forward chain for zerotier and put that before any drop rule in that same chain.

However, cleanest way (and easier to maintain) is to use interface lists.

Personal suggestion:
for yourself, it might be easier to order the firewall rules block by block.
First input, then forward, then output (if any).
Makes it also easier to see what happens where.

Performance wise you should put the most used rules more at the top (while still observing proper sequence of rules) but for most this will be a non-issue.

Why did you add the zerotier1 interface to the bridge???

had a brief go at doing this but couldnt work out how in 10seconds and didnt think it was that important for a small project but I clearly dont understand the benefits of lists. my “other” hEX which was built up from a defconfig DOES have lists of course.

made sense in my head:

I visualised the zt service as a switch that I am trying to connect 2 physical switches together with

in both of the hEXes, I make a bridge including eth2-5 which makes those physical ports transparent. so it followed to add each hEX’s zt1 i/f onto the bridge.

It's a layer 3 interface with it's own IP address, not something to be added to a bridge. You also wouldn't add a wireguard1 interface to a bridge for the same reason.

Or, if you look at normal ethernet interfaces such as ether2. If you don't add it to any bridge, then you can configure IP address directly on that standalone ether2 interface (with /ip address). But if you add ether2 to a bridge, you'll should no longer give that interface it's own IP address anymore, it's only acts as a pure layer 2 interface. In fact, ether2 no longer has an IP address, only the bridge has.

zerotier1 is a layer 3 interface with its own IP address in the range 172.24.0.0/16. Use it like a standalone interface.

Also, the word bridge appears 0 times in the MikroTik documentation: ZeroTier - RouterOS - MikroTik Documentation.

1 Like

ok. yeah, clearly big gaps in my understanding here.

removed the zt1 i/f from the bridge, and I can now ping “across” the zt service, I can get a response from the managed-IP of either hEX.

as soon as I include the zt1 if into the bridge, the ping response fails. as you say, this is an obvious L3 conflict. mea culpa.

1 Like

Next step is to attempt to make DHCP operate through the zerotier “service”. (I think)

Ultimately I would like to use the zerotier like a long ethernet cable, to bridge my two mikrotik bridges as transparently as a cable would do this between two small switches on the workbench. so that I can have 1 subnet spread across 2 distant sites. linked by an IP tunnel

now that I write that down, I wonder why I am concerned with DHCP at all. if I can bridge the bridges transparently, then I just need to make sure I have only 1 dhcp server….. right?

ZT we think cannot really offer a Layer-2 connection.

went back up the ZT road a bit, forked and ultimately achieved my objective with OpenVPN in Ethernet mode (sometimes called TAP)

ZT does L2 just fine.

Sorry to be blunt, but this is simply not true.

Wireguard is really an L3 interface, however Zerotier is L2. To be exact it’s a strange amalgamation: it’s what I refer to as “routed Ethernet” which sound like something that shouldn’t exist. This means that peers announce their mac addresses, and keep tables on where to send packets. So it’s L2 with tricks around mitigating unknown unicast.

Of course an L2 interface can be used as a routed interface, and that’s the common usage, and Mikrotik’s docs show how to do this (and only this.)

There are some consideration when more than one address is present behind one peer, and in that case “bridge mode” has to be enabled for the peer. Mikrotik’s implementation fully supports this, both as client and controller.

A particular point is that Zerotier includes a full blown policy system - in essence the peers do not trust each other. This actually makes a lot of sense for ad-hoc type networks, such as someone on the other side of the world running a game server secured by ZT (and yes this happens with some frequency.)

Policies are applied on the client side, and dhcp is filtered - so no, dhcp will not work.

really interesting argument.- what is L2? I thought simply Frame-level.

Frames of data are agnostic of what is in them. so: dhcp (or any protocol) should pass. ??

Ive never really understood OSI model. and - no apology needed - this is all learning.

Determining that it’s L2 is sort of easy. An ip (L3) packet has an ip header, at L2 an additional ethernet header is added. L2 also implies bridging (switching) while IP packets are routed.

Being “transparent” for any/all traffic is a different property. What you’re saying that a switch only looks at the mac address (which is a fine principle) but for anything but the simplest dumb switch is not the case. Bridges have bridge firewalls, switches have switch rules. These are usually able to match into the packets and understand ip/tcp/udp headers just fine. Networking equipment has simply evolved from the initial principles.

Blocking dhcp is simply a security consideration, and zt provides alternative addressing allocation. It’s also not in the least illegal to run a dhcp server on the remote end. The subnets are fine to overlap, it’s only necessary to have non-overlapping address ranges.

Re-edit: Perhaps I miss understood the above post. I will remove this to try to keep down the confusion level in this thread.

I

This is a kind of common misconception. E.g. when you capture a packet on a genuine L3 interface like Wireguard, IPIP or PPPoE, you simply don’t have an ethernet header (or MAC address); just the naked IP packet.

When this packet has to be sent over Ethernet (an L2 interface) that’s when it’s equipped with the header, and this is the point that ARP or ND is performed.

EDIT: I can only agree that most packets go through various steps during its journey, understanding these steps meticulously is key to designing networking solutions.

The essential question here is about contrasting Wireguard which is purely L3, no doubt is possible: e.g. Mikrotik’s ui simply doesn’t allow a wg tunnel to become part of a bridge. Zerotier is L2, and it shouldn’t be surprising that it can be made part of a bridge. Wireguard is similar to ppp, Zerotier is ethernet.