bridge stops forwarding traffic

Hi Guys,

my hEX-S stopps forwarding traffic on a bridge. I bridge 2 interfaces (sfp and ether1) or in other situations (sfp, ether1, ether2, ether3).

Here is my configuration:

[rack@routeros] > export
# 2024-05-01 20:31:49 by RouterOS 7.14.3
# software id = QPCN-XXXX
#
# model = RB760iGS
# serial number = E1F10XXXXXXXXX
/interface bridge
add fast-forward=no ingress-filtering=no name=bridge-internet port-cost-mode=short protocol-mode=none pvid=1470 vlan-filtering=yes
/interface ethernet
set [ find default-name=ether1 ] name=ether1
set [ find default-name=ether2 ] name=ether2
set [ find default-name=ether3 ] name=ether3
set [ find default-name=sfp1 ] auto-negotiation=no name=sfp1-internet speed=1G-baseX
/interface lte apn
set [ find default=yes ] ip-type=ipv4 use-network-apn=no
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip smb users
set [ find default=yes ] disabled=yes
/port
set 0 name=serial0
/queue simple
add max-limit=257786k/257786k name=queue2 queue=ethernet-default/ethernet-default target=ether2
add max-limit=257786k/257786k name=queue1 queue=ethernet-default/ethernet-default target=ether3
/routing bgp template
set default disabled=no output.network=bgp-networks
/interface bridge filter
add action=drop chain=forward in-interface=ether2 src-mac-address=!D4:24:DD:83:23:92/FF:FF:FF:FF:FF:FF
add action=drop chain=forward in-interface=ether3 src-mac-address=!1C:ED:6F:9F:E9:34/FF:FF:FF:FF:FF:FF
add action=drop chain=forward in-interface=ether1 src-mac-address=!1C:ED:6F:6F:01:21/FF:FF:FF:FF:FF:FF
/interface bridge port
add bridge=bridge-internet frame-types=admit-only-untagged-and-priority-tagged interface=ether1 internal-path-cost=10 path-cost=10 pvid=1470
add bridge=bridge-internet ingress-filtering=no interface=sfp1-internet internal-path-cost=10 path-cost=10 trusted=yes
add bridge=bridge-internet frame-types=admit-only-untagged-and-priority-tagged interface=ether2 internal-path-cost=10 path-cost=10 pvid=1470
add bridge=bridge-internet frame-types=admit-only-untagged-and-priority-tagged interface=ether3 internal-path-cost=10 path-cost=10 pvid=1470
/interface bridge settings
set allow-fast-path=no use-ip-firewall=yes use-ip-firewall-for-vlan=yes
/ip firewall connection tracking
set udp-timeout=10s
/ip neighbor discovery-settings
set discover-interface-list=!dynamic
/ipv6 settings
set max-neighbor-entries=8192
/interface bridge vlan
add bridge=bridge-internet tagged=sfp1-internet vlan-ids=1470
/interface ovpn-server server
set auth=sha1,md5
/ip address
add address=172.16.4.154/26 interface=bridge-internet network=172.16.4.128
/ip dns
set servers=91.205.12.0
/ip route
add disabled=no dst-address=0.0.0.0/0 gateway=172.16.4.129
/ip smb shares
set [ find default=yes ] directory=/flash/pub
/ip ssh
set forwarding-enabled=both
/routing bfd configuration
add disabled=no interfaces=all min-rx=200ms min-tx=200ms multiplier=5
/system clock
set time-zone-name=Europe/Berlin
/system identity
set name=routeros
/system note
set show-at-login=no

Is a simple bridge configuration.

What i have tried and had no resolved the issue:

  • Disable / Enable bridge
  • Disable / Enable slave-interfaces
  • Disable / Enable Bridge-Port
  • Remove Bridge-Port and readding

Only solution was ROUTER REBOOT.

We have this issue not on all hEX S at the same time. We have about 1.500 hEX S in production use. Sometimes it is only one device discovering this issue a day, and another day it hits up to 10 routers. Also the issue is not reproducable for me.

Why would you do anything on the bridge setup other than turn on vlan-filtering. Once you get cute you run into problems.
I see your routers are not for allowing traffic they are designed to block mac addresses primarily.

Hello, we have the same error but can’t find a solution. The HEX S no longer forwards traffic - only a restart will help. It’s not an STP problem, all ports are clean.

How did you solve the problem?

… here some more Informations:

  1. The Config we use:
# 2024-10-04 15:53:06 by RouterOS 7.15.3
# software id = SVIN-TPCP
#
# model = RB760iGS
# serial number = HF009xxxxx
/interface bridge
add comment=defconf name="Bridge VLAN 2018" port-cost-mode=short pvid=2018 \
    vlan-filtering=yes
add name="Bridge VLAN 2101" port-cost-mode=short pvid=2101 vlan-filtering=yes
/interface ethernet
set [ find default-name=ether5 ] comment="local Management Port"
set [ find default-name=sfp1 ] comment="WAN Port"
/interface vlan
add comment="xxxx" interface=sfp1 name="SFP1 vlan2018" \
    vlan-id=2018
add comment="ISP Management" interface=sfp1 name="SPF1 vlan2101" vlan-id=2101
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface lte apn
set [ find default=yes ] ip-type=ipv4 use-network-apn=no
/ip pool
add name=dhcp ranges=192.168.88.10-192.168.88.254
/ip smb users
set [ find default=yes ] disabled=yes
/port
set 0 name=serial0
/queue simple
add max-limit=210M/210M name=InetSpeed target=ether1 time=\
    0s-1d,sun,mon,tue,wed,thu,fri,sat
/snmp community
set [ find default=yes ] disabled=yes
add addresses=xx.x.x.x/xx,0.0.0.0/0 authentication-protocol=SHA1 \
    encryption-protocol=AES name=monitoring security=authorized
/interface bridge filter
add action=accept chain=forward in-interface=ether1 mac-protocol=ip \
    src-address=xxx.xxx.xxx.xxx/32
add action=accept chain=forward dst-address=xxx.xxx.xxx.xxx/32 mac-protocol=\
    ip out-interface=ether1
add action=drop chain=forward in-interface=ether1 mac-protocol=ip
add action=drop chain=forward mac-protocol=ip out-interface=ether1
/interface bridge port
add bridge="Bridge VLAN 2018" comment=defconf disabled=yes ingress-filtering=\
    no interface=ether2 internal-path-cost=10 path-cost=10
add bridge="Bridge VLAN 2018" comment=defconf disabled=yes ingress-filtering=\
    no interface=ether3 internal-path-cost=10 path-cost=10
add bridge="Bridge VLAN 2018" comment=defconf disabled=yes ingress-filtering=\
    no interface=ether4 internal-path-cost=10 path-cost=10
add auto-isolate=yes bridge="Bridge VLAN 2018" hw=no interface=ether1 \
    internal-path-cost=10 path-cost=10 pvid=2018
add bridge="Bridge VLAN 2101" interface="SPF1 vlan2101" internal-path-cost=10 \
    path-cost=10 pvid=2101
add bridge="Bridge VLAN 2018" interface="SFP1 vlan2018" internal-path-cost=10 \
    path-cost=10 pvid=2018
/interface bridge settings
set use-ip-firewall=yes use-ip-firewall-for-vlan=yes
/ip firewall connection tracking
set udp-timeout=10s
/ip neighbor discovery-settings
set discover-interface-list=LAN
/ipv6 settings
set max-neighbor-entries=8192
/interface list member
add comment=defconf interface=ether1 list=LAN
add interface=ether2 list=LAN
add interface=ether3 list=LAN
add interface=ether4 list=LAN
add interface=ether5 list=LAN
add comment="to FTTH P2P" interface=sfp1 list=WAN
add interface="Bridge VLAN 2018" list=LAN
/interface ovpn-server server
set auth=sha1,md5
/ip address
add address=192.168.88.1/24 comment=defconf interface=ether5 network=\
    192.168.88.0
add address=xx.xxx.x.xx/24 interface="Bridge VLAN 2101" network=xx.xxx.x.x
/ip dhcp-server network
add address=192.168.88.0/24 comment=defconf gateway=192.168.88.1
/ip dns
set allow-remote-requests=yes
/ip dns static
add address=192.168.88.1 comment=defconf name=router.lan
/ip route
add disabled=no dst-address=0.0.0.0/0 gateway=xx.xxx.x.x routing-table=main \
    suppress-hw-offload=no
/ip smb shares
set [ find default=yes ] directory=/flash/pub
/ipv6 firewall filter
add action=accept chain=forward connection-state=\
    invalid,established,related,new,untracked src-address=xxxx:xxxx:x::/48
add action=accept chain=forward connection-state=\
    established,related,new,untracked dst-address=xxxx:xxxx:x::/48
add action=drop chain=forward connection-state=\
    invalid,established,related,new
/ipv6 nd
set [ find default=yes ] disabled=yes
/routing bfd configuration
add disabled=no interfaces=all min-rx=200ms min-tx=200ms multiplier=5
/snmp
set contact=isp@buerodata.de enabled=yes engine-id-suffix=xxxxxxxxxxxx \
    location="xxxx" trap-community=monitoring
/system clock
set time-zone-name=Europe/Berlin
/system identity
set name="xxxx"
/system logging
add topics=stp
/system note
set show-at-login=no
/system ntp client
set enabled=yes
/system ntp client servers
add address=xx.xx.x.x
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN

2 changing Hardware with different Production/Version of HEX-S >> same failure
3 searching arround the Forum and find one more Thread:
http://forum.mikrotik.com/t/hex-s-set-to-bridge-mode-needs-constant-rebooting/159935/1
4 activate LOG / Debug (STP) >> NOTHING
5 send supout to mikrotik >> NOTHING found

The HEX-S works fine radomize for weeks or 1-2 days, but if traffic is stopping forwarding ONLY reboot is helping.
There is no STP Problem, there is no CPU Burst and no Error in LOG to see

I belive this is a strange Firware BUG!

Can anybody help here?

Today i was able to fix the issue by seeing strange things to happen. That was pure luck that i was on the right device at right time to find the culprit.

Short-Solution: Disable HARDWARE-OFFLOAD on the Bridge-Port

I was changing something on a device (vlan from 1340 to 1341) all was configured correctly but the device was not able to communicate with the network. There was no RX-Traffic. So when doing /tools/sniffer/quick interface=ether1 direction=rx i was not able to see any traffic. But on RX-Traffic-Stats on Interface i was able to see bytes counter increase.

I’ve opened /logs and saw, that seconds after changing PVID of Bridge-Port there was a message that tells that Hardware-Offload was activated for ether1.

hardware offloading activated on bridge “bridge1” ports: ether1

I did some research on other hEX S and found that none of the bridge-ports have the Flag “H”. But settings on Bridge-Ports says “hardware-offload=yes”. So rOS knows, that enabling HW-Offload will screw things up and don’t activate HW-Offload on that port.

But it seems that the unit runs into a race condition and enables HW-Offload on the port. And when it does, traffic stops forwarding. In my case ether1 is not able to forward traffic to SFP1 anymore.

Today: Completely other customer raised a ticket, internet is down. I was able to get onto the hEX S and at first look, i saw that H-Flag is on the Bridge-Port for ether1. I was changing the settings to hardware-offload=no and voila, traffic starts passing again, ping works, internet works. In the log i saw that randomly (for my point of view) rOS added Interface ether1 to HW-Offload. But why?

So i can confirm, that is a Firmware-Bug. And the bug exists also on 7.19.3.

Where exactly in the configuration is that setting?

Sorry, its hw=no

/interface bridge port
add bridge=bridge1 hw=no ingress-filtering=yes interface=ether1 trusted=no
add bridge=bridge1 hw=no ingress-filtering=yes interface=ether2 trusted=no
add bridge=bridge1 hw=no ingress-filtering=yes interface=sfp1-internet trusted=yes```

Good, thanks. :smile:

Might be advisable to create a formal support ticket, just in case they are not aware (anymore) about this problem.