Winbox loosing connection

RouterOS General
1

Hello.

I experience problems with my setup @ home.
Winbox looses connection regularly when connecting via IP, but not via MAC.

Maybe I have something done wrong in my configuration.
I’d really appreciate if someone could help or point me in the right direction.

# 2025-05-10 09:59:29 by RouterOS 7.18.2
# software id = 2W2R-GYTV
#
# model = CRS328-24P-4S+
/interface bridge
add admin-mac=18:FD:74:88:60:7D auto-mac=no comment=defconf ingress-filtering=no name=bridge port-cost-mode=short vlan-filtering=yes
add name=loopback port-cost-mode=short
/interface ethernet
set [ find default-name=ether1 ] l2mtu=10218
set [ find default-name=ether2 ] l2mtu=10218
set [ find default-name=ether3 ] l2mtu=10218
set [ find default-name=ether4 ] l2mtu=10218
set [ find default-name=ether5 ] comment=Uplink-Modem l2mtu=10218
set [ find default-name=ether6 ] comment=VA-AP-K l2mtu=10218
set [ find default-name=ether7 ] l2mtu=10218
set [ find default-name=ether8 ] l2mtu=10218
set [ find default-name=ether9 ] comment=FFHE-ERX l2mtu=10218
set [ find default-name=ether10 ] comment=FFHE-Client l2mtu=10218
set [ find default-name=ether11 ] comment=FFHE-Mesh l2mtu=10218
set [ find default-name=ether12 ] l2mtu=10218
set [ find default-name=ether13 ] l2mtu=10218
set [ find default-name=ether14 ] l2mtu=10218
set [ find default-name=ether15 ] l2mtu=10218
set [ find default-name=ether16 ] l2mtu=10218
set [ find default-name=ether17 ] l2mtu=10218
set [ find default-name=ether18 ] l2mtu=10218
set [ find default-name=ether19 ] l2mtu=10218
set [ find default-name=ether20 ] l2mtu=10218
set [ find default-name=ether21 ] l2mtu=10218
set [ find default-name=ether22 ] l2mtu=10218
set [ find default-name=ether23 ] l2mtu=10218
set [ find default-name=ether24 ] l2mtu=10218
set [ find default-name=sfp-sfpplus1 ] l2mtu=10218
set [ find default-name=sfp-sfpplus2 ] l2mtu=10218
set [ find default-name=sfp-sfpplus3 ] l2mtu=10218
set [ find default-name=sfp-sfpplus4 ] l2mtu=10218
/interface vlan
add interface=bridge name=CCTV vlan-id=2200
add interface=bridge name=DMZ vlan-id=2206
add interface=bridge name=HAUSAUTO vlan-id=1104
add interface=bridge name=IoT vlan-id=2201
add interface=bridge name=LAN vlan-id=2205
add interface=bridge name=MGMT vlan-id=1103
/interface bonding
add mode=802.3ad name=bond-buero slaves=ether19,ether20 transmit-hash-policy=layer-3-and-4
add mode=802.3ad name=bond-flur slaves=ether17,ether18 transmit-hash-policy=layer-3-and-4
add mode=802.3ad name=bond-fw slaves=ether1,ether2 transmit-hash-policy=layer-3-and-4
add mode=802.3ad name=bond-nas slaves=sfp-sfpplus1,sfp-sfpplus2 transmit-hash-policy=layer-3-and-4
add mode=802.3ad name=bond-pve slaves=sfp-sfpplus3,sfp-sfpplus4 transmit-hash-policy=layer-3-and-4
add mode=802.3ad name=bond-rb5009 slaves=ether3,ether4 transmit-hash-policy=layer-3-and-4
/interface lte apn
set [ find default=yes ] ip-type=ipv4 use-network-apn=no
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=LAN ranges=10.29.10.201-10.29.10.250
add name=DMZ ranges=10.29.13.201-10.29.13.250
add name=MGMT ranges=10.29.11.201-10.29.11.250
add name=HAUSAUTO ranges=10.29.14.201-10.29.14.250
add name=IoT ranges=10.29.15.201-10.29.15.250
add name=ROUTER ranges=10.29.1.201-10.29.1.250
add name=CCTV ranges=10.29.12.201-10.29.12.250
/ip dhcp-server
add address-pool=IoT interface=IoT lease-time=1w name=IoT
add address-pool=CCTV interface=CCTV lease-time=1w name=CCTV
add address-pool=HAUSAUTO interface=HAUSAUTO lease-time=1w name=HAUSAUTO
add address-pool=DMZ interface=DMZ lease-time=1w name=DMZ
add address-pool=LAN interface=LAN lease-time=1w name=LAN
add address-pool=MGMT interface=MGMT lease-time=1w name=MGMT
/port
set 0 name=serial0
/snmp community
set [ find default=yes ] disabled=yes
add addresses=10.29.11.0/24 name=CHAOSTRUPPE write-access=yes
/interface bridge port
add bridge=bridge comment=defconf ingress-filtering=no interface=ether5 internal-path-cost=10 path-cost=10 pvid=1100
add bridge=bridge comment=defconf ingress-filtering=no interface=ether6 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether7 internal-path-cost=10 path-cost=10 pvid=1103
add bridge=bridge comment=defconf ingress-filtering=no interface=ether8 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged ingress-filtering=no interface=ether9 internal-path-cost=10 path-cost=10 pvid=2201
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged ingress-filtering=no interface=ether10 internal-path-cost=10 path-cost=10 pvid=332
add bridge=bridge comment=defconf frame-types=admit-only-untagged-and-priority-tagged ingress-filtering=no interface=ether11 internal-path-cost=10 path-cost=10 pvid=336
add bridge=bridge comment=defconf ingress-filtering=no interface=ether12 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether13 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether14 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether15 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether21 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether22 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether23 internal-path-cost=10 path-cost=10 pvid=1103
add bridge=bridge comment=defconf ingress-filtering=no interface=ether24 internal-path-cost=10 path-cost=10 pvid=1103
add bridge=bridge ingress-filtering=no interface=bond-nas internal-path-cost=10 path-cost=10 pvid=2206
add bridge=bridge ingress-filtering=no interface=bond-pve internal-path-cost=10 path-cost=10 pvid=1103
add bridge=bridge ingress-filtering=no interface=bond-flur internal-path-cost=10 path-cost=10
add bridge=bridge ingress-filtering=no interface=bond-buero internal-path-cost=10 path-cost=10
add bridge=bridge ingress-filtering=no interface=bond-rb5009 internal-path-cost=10 path-cost=10
add bridge=bridge ingress-filtering=no interface=bond-fw internal-path-cost=10 path-cost=10
/ip firewall connection tracking
set udp-timeout=10s
/ip settings
set max-neighbor-entries=8192
/ipv6 settings
set disable-ipv6=yes
/interface bridge vlan
add bridge=bridge comment=MGMT tagged=bridge,bond-rb5009,bond-nas,bond-fw,bond-buero,bond-flur,ether6 untagged=ether7,bond-pve,ether23,ether24 vlan-ids=1103
add bridge=bridge comment=LAN tagged=bond-rb5009,bond-flur,bond-buero,bond-fw,bond-pve,ether6 vlan-ids=2205
add bridge=bridge comment=DMZ tagged=bond-buero,bond-rb5009,bond-fw,bond-pve,bond-flur untagged=bond-nas vlan-ids=2206
add bridge=bridge comment=IoT tagged=bond-buero,bond-rb5009,bond-flur,bond-fw,bond-pve,ether6 untagged=ether9 vlan-ids=2201
add bridge=bridge comment=CCTV tagged=bond-pve,bond-rb5009 vlan-ids=2200
add bridge=bridge comment=Automatisierung tagged=bond-rb5009,bond-buero,bond-fw,bond-pve,bond-flur,ether6 vlan-ids=1104
add bridge=bridge comment=FFHE-Client tagged=bond-flur,bond-pve,bond-rb5009 untagged=ether10 vlan-ids=332
add bridge=bridge comment=FFHE-Mesh tagged=bond-rb5009 untagged=ether11 vlan-ids=336
add bridge=bridge comment=TRANSPORT tagged=bond-flur,bond-rb5009 untagged=ether5 vlan-ids=1100
add bridge=bridge comment=k3s tagged=bond-pve,bond-rb5009 vlan-ids=3000
add bridge=bridge comment=RANDOM tagged=bond-buero,bond-rb5009 vlan-ids=1337
/interface ovpn-server server
add auth=sha1,md5 mac-address=FE:F5:94:78:DA:FB name=ovpn-server1
/ip address
add address=192.168.88.1/24 comment=defconf interface=bridge network=192.168.88.0
add address=10.29.11.100/24 comment=MGMT interface=MGMT network=10.29.11.0
add address=10.255.29.1 interface=loopback network=10.255.29.1
add address=10.29.15.100/24 comment=IoT interface=IoT network=10.29.15.0
add address=10.29.12.100/24 comment=CCTV interface=CCTV network=10.29.12.0
add address=10.29.14.100/24 comment=HAUSAUTO interface=HAUSAUTO network=10.29.14.0
add address=10.29.13.100/24 comment=DMZ interface=DMZ network=10.29.13.0
add address=10.29.10.100/24 comment=LAN interface=LAN network=10.29.10.0
/ip dhcp-server network
add address=10.29.10.0/24 comment=LAN dns-server=10.29.11.53 domain=chaos.lan gateway=10.29.10.254 netmask=24 ntp-server=10.29.11.53
add address=10.29.11.0/24 comment=MGMT dns-server=10.29.11.53 domain=chaos.lan gateway=10.29.11.254 netmask=24 ntp-server=10.29.11.53
add address=10.29.12.0/24 comment=CCTV dns-none=yes gateway=0.0.0.0 netmask=24
add address=10.29.13.0/24 comment=DMZ dns-server=10.29.11.53 domain=chaos.lan gateway=10.29.13.254 netmask=24 ntp-server=10.29.11.53
add address=10.29.14.0/24 comment=HAUSAUTO dns-server=10.29.11.53 domain=chaos.lan gateway=10.29.14.254 netmask=24 ntp-server=10.29.11.53
add address=10.29.15.0/24 comment=IoT dns-server=10.29.11.53 domain=chaos.lan gateway=10.29.15.254 netmask=24 ntp-server=10.29.11.53
/ip dns
set mdns-repeat-ifaces=LAN,DMZ,IoT,HAUSAUTO servers=10.29.11.53
/ip ipsec profile
set [ find default=yes ] dpd-interval=2m dpd-maximum-failures=5
/ip route
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=10.29.11.254 routing-table=main scope=30 suppress-hw-offload=no target-scope=10
/routing bfd configuration
add disabled=no interfaces=all min-rx=200ms min-tx=200ms multiplier=5
/system clock
set time-zone-name=Europe/Berlin
/system identity
set name=VA-RSWS01
/system note
set show-at-login=no

Thanks in advance :slight_smile:

2

Hey

That’s like “needle in the haystack” …

Please be more specific: how do you connect, from to, any additional observations about the system? what is happing on crs when you do loos connectivity? is the loss at random or some specific situations? and so on…

3

Is the winbox flowing over any of the bonded links? While winbox protocol should be fine with transmit-hash-policy=layer-3-and-4 (at least IMO)…it’s also not hard imagine something could “go wrong” in hashing winbox traffic. And since winbox via MAC works, that would use L2 hashing only (regardless of the policy).

4

The CRS is connected to a pfSense via a Bond.
The hashing policy of every bonded interface is Layer 3+4.
From the CRS, there is another Bond going to a Unifi switch in my office, where my PC is plugged in to a access port.
As far as I remember, everything worked up until a point, and I can’t exactly isolate what changed. I personally can say, that nothing was changed on the configuration side except Updates to Software and Firmware. And maybe Winbox 4 got an update.
I did check if Windox 3 has the same behavior (just now), and sure enough, it does.
At the same time Winbox 3 experiences the disconnect, Winbox 4 does also.
The logs on the CRS only says:

user admin logged out from 10.29.10.101 via winbox
user admin logged in from 10.29.10.101 via winbox

RX or TX Errors/Drops aren’t rising before or after the disconnect incident.
The CRS has no firewall rules, it’s acting as a switch.

Firewall rules on the pfSense are allowing access no matter what.
There is no traffic shaping on the LAN side.

I am connecting from my PC (LAN VLAN), via the pfSense to the MGMT VLAN where the CRS is located at.

Everything else (my home network contains just under 50 connected clients, wireless and wired) is working fine.
The pfSense is the Router, and the CRS is doing DHCP (moved it from the pfSense).


Is there anything else, I can provide for a clearer picture of my setup?
Thank you so far, for you help :slight_smile:

5

Good afternoon :smiley:
It seems I’ll have to admit, that today it is working fine.

I went to recheck the settings on the pfSense again, because as I stated I thought every policy was set to Layer 3+4, but unfortunately I found that the pfSense bond was set to Layer 2+3+4.
After I changed this setting to only Layer 3+4, and after a state table flush (to close every possible connection) it works flawlessly now.

Just to make sure I am having a correct experience, I’d like to ask if this really could have been the root cause for the disconnects I had experienced.
I can’t remember any time in the past, where I have had Problems like this. And the Hash policy settings wasn’t changed by me in a looooong time.

Sorry for having had errors in my last post…
And thank you all still for trying to help me.

6

Part of the issue is the winbox protocol is not described/documented. So if some packet looks “different” than rest of sessions, it may drop session. And you got stuff like ARP and bond caches in between. Flip side is that packets should look same since it’s going through the bond. But IDK exactly why… more that bonding generally speaking can be tricky (and MikroTik has been doing a lot of “bug fixes”… so possible something changed someplace that broken whatever was going on previously )

7

I’ll close the thread.
This seems to have been the root cause.

Thank you all for your engagement and your patience.