Zerotier interfaces stops working after a PPPoE wan flap (7.2rc4-7.2rc7, maybe 7.2)

RouterOS RouterOS beta
1

I have noted that every time that my ISP resets my PPPoE connection, all my ZT interfaces stops passing traffic. No errors shown anyway, it silent stop working.

The only way to restore it is manually disabling and enabling the zt1 instance.
This happens on every 7.2rc and may also affect the 7.2 stable, as I did not saw any mention of it on changelog and my Support ticket is untouched and unanswered by a month.

one important info, this doesn’t seem to happen if I terminate the connection on my side, only when the ISP does.

2

what interface do you have zerotier bound to?

3

I have the exact same issue and I reported it before. And it’s occurred ever since Zerotier was added as a package.

If you leave it without restarting the zerotier instance it will eventually come back which makes me believe it’s related to connection tracking in the firewall.

Although I haven’t bothered to check into it in more details as I just restart the zerotier instance.

My previous report: http://forum.mikrotik.com/t/zerotier-added-to-routeros-v7-1rc2/151475/246

4

I was using the default config, that means, all. since them I switched to only the ones marked as WAN.
as my PPPoE resets once in a week, I need to wait…

5

Can’t you trigger the symptom by rebooting the ISP modem?

6

I don’t have a modem, it is PPPoE over GPON, so, I have only a fiber ONT, but no, happen only when terminated by the other side.

7

I have the exact same issue and I can’t get internet out from MT RB4011 so I have to resort from backup to let it work again
my MT router connect to Huawei N5368X 5G Outdoor CPE

8

Hi,

Same issue with me. Zt connection drops after a week or so. Can’t restart as the site is remote. WAN connection is PPPoe… Any more thoughts on this one?

9

We have been having same issue of lost ZeroTier connectivity with several Starlink remotes using RB5009 systems. Sometimes after a router reboot or Starlink offline - no ZeroTier access for varying time periods - sometimes ZT comes back in less than an hour, sometimes instantly, sometimes what seems like forever. Same drop of ZeroTier connectivity issue can be duplicated in lab fairly consistently.

If only concerned about ZT loss after a MT reboot - The fix being as mentioned > disable zt1 (instance) > delay 60s > enable zt1 (instance)
Then after Scheduler reboots and ZeroTier instance has time to request configuration - all is well.

/system scheduler
add name=schedule-ZT-dis-enb-delay on-event=":delay 5s;\r\
\n/zerotier disable zt1;\r\
\n:delay 60s;\r\
\n/zerotier enable zt1" policy=\
reboot,read,write,policy,test,password,sniff,sensitive start-time=startup

credit to @sergejs for clear help writing script http://forum.mikrotik.com/t/disable-interface-on-startup-and-enable-1-minute-later/85541/1

To use MT Watchdog in conjunction with the above script to cover times when a WAN flap or Starlink offline has caused loss of ZT connectivity, you will need to have another ZT instance online 24/7. Spin up a ZT instance on same ZT network in a dependable server to allow the MT in question to ping a specific ZT IP on same network. Then when ZT ping fails, MT Watchdog will reboot and the above Scheduler script will run.

/system watchdog
set automatic-supout=no watch-address=xxx.xxx.xxx.xxx (IP of dependable ZeroTier device in same network)

Hope this helps with ZeroTier interface not working after - MT router reboot, WAN flap, WAN drop, Starlink reboot

10

I’d make sure that ZeroTier instance (“zt1”) is NOT listening on raw interface that PPPoE is using. ZT does a lot of probing to find path and punch holes, so it doing that on the “real” interface, not PPPoE on top, seems like a bad idea. And the default is “/zerotier/set zt1 interfaces=all” so if PPPoE is using ether1, then make sure ether1 is NOT in the zt1 interface=.

11

Same problem still exists in 7.16.1 I use a net watch script to get around it as my zerotier is inband