Switch IP Packet Loss w/ HW Offload or Two Uplinks

Hello,

I figured I would turn here in hopes somebody smarter than I can assist with my headache. I have a switch that is on 10.0.2.240/24 (VLAN2) and a router that houses the VLAN on 10.0.2.1/24. I have two connections (SFP+ 10G and 1G Ethernet) that I would like to utilize for fail over. My router is the root RSTP bridge at priority 1000 and the switch is at 3000. As far as I can tell STP isn’t related but not entirely sure. The priority and root cost is lower for the SFP+ on both devices (I want it to prioritize the 10G connection).

So the strange part of this is that I get significant packet loss on ONLY the switch IP address (dropped pings, SSH won’t work etc.) when either:
A: HW offload is disabled on my SFP+ port on the switch (HW offload is already disabled on the router side because I believe that it cannot use HW offload)
B: The bridge port is disabled for ether1 on the router.

All other packets work perfectly fine. I have 5 or so VLANs, they all function, and all traffic on the 10.0.2.0/24 subnet work fine except for the one address (10.0.2.240).

Other VLANs have the same issue when trying to ping / access the switch IP as well. I used the packet sniffer and noticed that HW offloading is disabled or the ether1 is disabled, the router does not get any of the traffic for the IP addresses on the same subnet (as it should be). But when either of those is turned on, the router starts getting a portion of the traffic and that is where it seems my packet loss is.

Hopefully somebody has seen this before and knows if the behavior is a bug or something related to hardware? I am not sure why only this one address would be having this issue. I will try to include some configuration but feel free to let me know if there are any other sections that are needed:

Router:

 
[admin@MikroTik-Router] > /system resource print
                   uptime: 4d4h21m49s
                  version: 7.16.1 (stable)
               build-time: 2024-10-10 14:03:32
         factory-software: 7.9.2
              free-memory: 902.6MiB
             total-memory: 1024.0MiB
                      cpu: ARM
                cpu-count: 4
            cpu-frequency: 533MHz
                 cpu-load: 3%
           free-hdd-space: 478.4MiB
          total-hdd-space: 512.0MiB
  write-sect-since-reboot: 2446
         write-sect-total: 935535
               bad-blocks: 0%
        architecture-name: arm
               board-name: RB4011iGS+
                 platform: MikroTik
                  
[admin@MikroTik-Router] > /interface bridge print detail
Flags: X - disabled, R - running
 0 R name="backup-bridge" mtu=auto actual-mtu=1500 l2mtu=1592 arp=enabled
     arp-timeout=auto mac-address= protocol-mode=rstp
     fast-forward=yes igmp-snooping=no auto-mac=yes ageing-time=5m
     priority=0x8000 max-message-age=20s forward-delay=15s
     transmit-hold-count=6 vlan-filtering=no dhcp-snooping=no
     port-cost-mode=long mvrp=no max-learned-entries=auto


[admin@MikroTik-Router] > /interface vlan print detail
Flags: X - disabled, R - running
 0 R ;;; LAN VLAN
     name="VLAN 2" mtu=1500 l2mtu=1588 mac-address=
     arp=enabled arp-timeout=auto loop-protect=default
     loop-protect-status=off loop-protect-send-interval=5s
     loop-protect-disable-time=5m vlan-id=2 interface=bridge
     use-service-tag=no mvrp=no

[admin@MikroTik-Router] > /ip route print detail
Flags: D - dynamic; X - disabled, I - inactive, A - active;
c - connect, s - static, r - rip, b - bgp, o - ospf, i - is-is, d - dhcp, v - >
H - hw-offloaded; + - ecmp
 1   s   dst-address=0.0.0.0/0 routing-table=main gateway=192.168.255.1
         immediate-gw=192.168.255.1%ether3 distance=2 scope=30
         target-scope=10 suppress-hw-offload=no

   DAd   dst-address=0.0.0.0/0 routing-table=main gateway=X.X.X.X
         immediate-gw=X.X.X.X%ether2 distance=1 scope=30 target-scope=10
         vrf-interface=ether2

   DAc   dst-address=10.0.2.0/24 routing-table=main gateway=VLAN 2
         immediate-gw=VLAN 2 distance=0 scope=10 target-scope=5
         local-address=10.0.2.1%VLAN 2

[admin@MikroTik-Router] > /interface bridge vlan print detail
Flags: X - disabled, D - dynamic
 0   bridge=bridge vlan-ids=2,5,10,20,25,30,250
     tagged=bridge,ether1,veth1-reflector,sfp-sfpplus1 untagged=""
     mvrp-forbidden=""
     current-tagged=bridge,sfp-sfpplus1,veth1-reflector,ether1
     current-untagged=""

 1 D ;;; added by pvid
     bridge=bridge vlan-ids=1 tagged="" untagged=bridge mvrp-forbidden=""
     current-tagged="" current-untagged=bridge
	 
[admin@MikroTik-Router] /interface/bridge> port/print detail
Flags: X - disabled, I - inactive; D - dynamic; H - hw-offload
 0 I   ;;; defconf
       interface=ether4 bridge=backup-bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=1 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       mvrp-registrar-state=normal mvrp-applicant-state=normal-participant
       multicast-router=temporary-query fast-leave=no

 1 I H ;;; defconf
       interface=ether5 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=1 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       mvrp-registrar-state=normal mvrp-applicant-state=normal-participant
       multicast-router=temporary-query fast-leave=no

 2 I H ;;; defconf
       interface=ether6 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=1 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       mvrp-registrar-state=normal mvrp-applicant-state=normal-participant
       multicast-router=temporary-query fast-leave=no

 3 I H ;;; defconf
       interface=ether7 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=1 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       mvrp-registrar-state=normal mvrp-applicant-state=normal-participant
       multicast-router=temporary-query fast-leave=no

 4 I H ;;; defconf
       interface=ether8 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=1 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       mvrp-registrar-state=normal mvrp-applicant-state=normal-participant
       multicast-router=temporary-query fast-leave=no

 5 I H ;;; defconf
       interface=ether9 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=1 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       mvrp-registrar-state=normal mvrp-applicant-state=normal-participant
       multicast-router=temporary-query fast-leave=no

 6 I H ;;; defconf
       interface=ether10 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=1 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       mvrp-registrar-state=normal mvrp-applicant-state=normal-participant
       multicast-router=temporary-query fast-leave=no

 7     ;;; defconf
       interface=sfp-sfpplus1 bridge=bridge priority=0x70 path-cost=5
       internal-path-cost=5 edge=auto point-to-point=yes learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=2 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       mvrp-registrar-state=normal mvrp-applicant-state=normal-participant
       multicast-router=temporary-query fast-leave=no

 8 X   interface=ether1 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=yes learn=auto
       horizon=none auto-isolate=no restricted-role=no restricted-tcn=no
       pvid=2 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       mvrp-registrar-state=normal mvrp-applicant-state=normal-participant
       multicast-router=temporary-query fast-leave=no

 9     interface=veth1-reflector bridge=bridge priority=0x80 edge=yes
       point-to-point=yes learn=yes horizon=none auto-isolate=no
       restricted-role=no restricted-tcn=no pvid=999
       frame-types=admit-only-vlan-tagged ingress-filtering=no
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       mvrp-registrar-state=normal mvrp-applicant-state=normal-participant
       multicast-router=permanent fast-leave=no

Switch:

 [admin@MikroTik-Switch] > /system resource print
                   uptime: 11w4d23h13m14s
                  version: 6.49.17 (stable)
               build-time: Aug/07/2024 11:47:14
         factory-software: 6.47.4
              free-memory: 477.2MiB
             total-memory: 512.0MiB
                      cpu: ARMv7
                cpu-count: 1
            cpu-frequency: 800MHz
                 cpu-load: 5%
           free-hdd-space: 2016.0KiB
          total-hdd-space: 16.0MiB
  write-sect-since-reboot: 2080052
         write-sect-total: 7972827
               bad-blocks: 0%
        architecture-name: arm
               board-name: CRS328-24P-4S+
                 platform: MikroTik

[admin@MikroTik-Switch] > /interface bridge print detail
Flags: X - disabled, R - running
 1 R ;;; defconf
     name="bridge" mtu=auto actual-mtu=1500 l2mtu=1592 arp=enabled
     arp-timeout=auto mac-address= protocol-mode=rstp
     fast-forward=yes igmp-snooping=no auto-mac=no
     admin-mac= ageing-time=5m priority=0x3000
     max-message-age=20s forward-delay=15s transmit-hold-count=6
     vlan-filtering=yes ether-type=0x8100 pvid=1 frame-types=admit-all
     ingress-filtering=yes dhcp-snooping=no

[admin@MikroTik-Switch] > /interface vlan print detail
Flags: X - disabled, R - running
 0 R name="VLAN 2" mtu=1500 l2mtu=1588 mac-address=
     arp=enabled arp-timeout=auto loop-protect=default
     loop-protect-status=off loop-protect-send-interval=5s
     loop-protect-disable-time=5m vlan-id=2 interface=bridge
     use-service-tag=no


[admin@MikroTik-Switch] > /ip route print detail
Flags: X - disabled, A - active, D - dynamic,
C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme,
B - blackhole, U - unreachable, P - prohibit
 0 A S  dst-address=0.0.0.0/0 gateway=10.0.2.1
        gateway-status=10.0.2.1 reachable via  VLAN 2 distance=1 scope=30
        target-scope=10

 1 ADC  dst-address=10.0.2.0/24 pref-src=10.0.2.240 gateway=VLAN 2
        gateway-status=VLAN 2 reachable distance=0 scope=10

[admin@MikroTik-Switch] > /interface bridge vlan print detail
Flags: X - disabled, D - dynamic
 0   bridge=bridge vlan-ids=2
     tagged=bridge,ether18,ether11,ether23,ether22,ether20,ether24,ether21,
       ether15,ether1,sfp-sfpplus1
     untagged=ether16,ether12,ether9,QNAP
     current-tagged=bridge,ether11,ether15,ether20,ether21,ether23,ether24,sfp-
               sfpplus1,ether1
     current-untagged=ether9,ether12,QNAP

 1   bridge=bridge vlan-ids=5
     tagged=ether23,ether24,ether22,ether20,ether11,bridge,ether21,ether15,
       QNAP,ether1,sfp-sfpplus1
     untagged=ether7,ether8,ether10,ether2,ether3,ether4,ether5,ether6
     current-tagged=bridge,ether11,ether15,ether20,ether21,ether23,ether24,
               QNAP,sfp-sfpplus1,ether1

     current-untagged=ether2,ether3,ether4,ether5,ether6,ether7,ether8,ether10

 2   bridge=bridge vlan-ids=25
     tagged=QNAP,ether18,ether11,ether23,ether15,ether2,ether3,ether4,ether6,
       ether20,ether22,ether24,bridge,ether21,ether1,sfp-sfpplus1
     untagged="" current-tagged=bridge,ether2,ether3,ether4,ether6,ether11,
               ether15,ether20,ether21,ether23,ether24,QNAP,sfp-sfpplus1,
               ether1
     current-untagged=""

 3   bridge=bridge vlan-ids=250
     tagged=QNAP,ether18,ether11,ether23,ether24,ether2,ether3,ether4,ether6,
       ether22,ether20,bridge,ether21,ether15,ether1,sfp-sfpplus1
     untagged=ether13,ether14,ether16
     current-tagged=bridge,ether2,ether3,ether4,ether6,ether11,ether15,ether20,
               ether21,ether23,ether24,QNAP,sfp-sfpplus1,ether1
     current-untagged=""

 4   bridge=bridge vlan-ids=1 tagged="" untagged=bridge current-tagged=""
     current-untagged=bridge

 5   bridge=bridge vlan-ids=20
     tagged=ether18,ether11,ether23,bridge,QNAP,ether15,ether20,ether21,
       ether24,ether1,sfp-sfpplus1
     untagged="" current-tagged=bridge,ether11,ether15,ether20,ether21,ether23,
               ether24,QNAP,sfp-sfpplus1,ether1
     current-untagged=""

 6   bridge=bridge vlan-ids=30 tagged=QNAP,ether1,bridge,sfp-sfpplus1
     untagged="" current-tagged=bridge,QNAP,sfp-sfpplus1,ether1
     current-untagged=""

 7 X bridge=bridge vlan-ids=10 tagged=QNAP,ether1,sfp-sfpplus1,bridge
     untagged="" current-tagged="" current-untagged=""
	 
[admin@MikroTik-Switch] /interface bridge> port print detail
Flags: X - disabled, I - inactive, D - dynamic, H - hw-offload
 0   H ;;; defconf
       interface=ether2 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=5 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

 1   H ;;; defconf
       interface=ether3 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=5 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

 2   H ;;; defconf
       interface=ether4 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=5 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

 3   H ;;; defconf
       interface=ether5 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=5 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

 4   H ;;; defconf
       interface=ether6 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=5 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

 5   H ;;; defconf
       interface=ether7 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=5 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

 6   H ;;; defconf
       interface=ether8 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=5 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

 7   H ;;; defconf
       interface=ether9 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=2 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

 8   H ;;; defconf
       interface=ether10 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=5 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

 9   H ;;; defconf
       interface=ether11 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=2 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

10   H ;;; defconf
       interface=ether12 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=2 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

11 I H ;;; defconf
       interface=ether13 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=250 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

12 I H ;;; defconf
       interface=ether14 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=250 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

13   H ;;; defconf
       interface=ether15 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=2 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

14 I H ;;; defconf
       interface=ether16 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=250 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

15 I   ;;; defconf
       interface=ether17 bridge=backup-bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=1 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

16 I H ;;; defconf
       interface=ether18 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=2 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

17 I H ;;; defconf
       interface=ether19 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=1 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

18   H ;;; defconf
       interface=ether20 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=2 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

19   H ;;; defconf
       interface=ether21 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=2 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

20 I H ;;; defconf
       interface=ether22 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=2 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

21   H ;;; defconf
       interface=ether23 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=2 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

22   H ;;; defconf
       interface=ether24 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=2 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

23 I H ;;; defconf
       interface=sfp-sfpplus3 bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=1 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

24   H ;;; QNAP
       interface=QNAP bridge=bridge priority=0x80 path-cost=10
       internal-path-cost=10 edge=auto point-to-point=auto learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=2 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

25   H interface=ether1 bridge=bridge priority=0x80 path-cost=15
       internal-path-cost=15 edge=auto point-to-point=yes learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=2 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

26   H interface=sfp-sfpplus1 bridge=bridge priority=0x70 path-cost=5
       internal-path-cost=5 edge=auto point-to-point=yes learn=auto
       horizon=none hw=yes auto-isolate=no restricted-role=no
       restricted-tcn=no pvid=2 frame-types=admit-all ingress-filtering=yes
       unknown-unicast-flood=yes unknown-multicast-flood=yes
       broadcast-flood=yes tag-stacking=no bpdu-guard=no trusted=no
       multicast-router=temporary-query fast-leave=no

Screenshot 2024-10-29 152509.png

Please, post config export from both devices … run /interface export on both devices for start. It’s much more readable than output of “print detail” for one thing.

And why would you want to disable HW offload on switch? If you do, then all traffic via switch will have to pass CPU and your switch’ CPU is too weak to deal with traffic in any decently performant way. Router is another thing, there SFP port is connected directly to CPU, but router also has quite a bit faster CPU.

Thanks for the response and sorry for the confusion. I very much want HW offloading, without it on the SFP+ on my switch it can’t even handle 1G of throughput. I was just saying that disabling HW offloading or disabling the redundant connection are the only two things that seems to fix the packet loss issue. Here are the interface exports:

Router:

# 2024-10-30 09:24:01 by RouterOS 7.16.1
# software id = 5ZG1-731G
#
# model = RB4011iGS+
# serial number = XXX
/interface bridge
add name=backup-bridge
add admin-mac= auto-mac=no comment=defconf name=bridge \
    port-cost-mode=short priority=0x1000 vlan-filtering=yes
/interface ethernet
set [ find default-name=sfp-sfpplus1 ] l2mtu=1592
/interface veth
add address=127.1.0.10/32 gateway=127.1.0.1 gateway6="" name=veth1-reflector
/interface wireguard
add listen-port= mtu=1420 name=wireguard1
/interface vlan
add comment="" interface=bridge name="VLAN 2" vlan-id=2
add comment="" interface=bridge name="VLAN 5" vlan-id=5
add comment="" interface=bridge name="VLAN 10" vlan-id=10
add comment="" interface=bridge name="VLAN 20" vlan-id=20
add comment="" interface=bridge name="VLAN 25" vlan-id=25
add comment="" interface=bridge name="VLAN 30" vlan-id=30
add comment="" interface=bridge name="VLAN 250" vlan-id=250
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
add name=VLAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/interface bridge port
add bridge=backup-bridge comment=defconf interface=ether4 internal-path-cost=\
    10 path-cost=10
add bridge=bridge comment=defconf interface=ether5 internal-path-cost=10 \
    path-cost=10
add bridge=bridge comment=defconf interface=ether6 internal-path-cost=10 \
    path-cost=10
add bridge=bridge comment=defconf interface=ether7 internal-path-cost=10 \
    path-cost=10
add bridge=bridge comment=defconf interface=ether8 internal-path-cost=10 \
    path-cost=10
add bridge=bridge comment=defconf interface=ether9 internal-path-cost=10 \
    path-cost=10
add bridge=bridge comment=defconf interface=ether10 internal-path-cost=10 \
    path-cost=10
add bridge=bridge comment=defconf interface=sfp-sfpplus1 internal-path-cost=5 \
    path-cost=5 point-to-point=yes priority=0x70 pvid=2
add bridge=bridge disabled=yes interface=ether1 internal-path-cost=10 \
    path-cost=10 point-to-point=yes pvid=2
add bridge=bridge edge=yes frame-types=admit-only-vlan-tagged \
    ingress-filtering=no interface=veth1-reflector learn=yes \
    multicast-router=permanent point-to-point=yes pvid=999
/interface bridge vlan
# ether1 not a bridge port
add bridge=bridge tagged=bridge,ether1,veth1-reflector,sfp-sfpplus1 vlan-ids=\
    2,5,10,20,25,30,250
/interface detect-internet
set detect-interface-list=WAN
/interface list member
add comment=defconf interface=bridge list=LAN
add interface=ether1 list=LAN
add interface=ether2 list=WAN
add interface=ether3 list=WAN
add interface=ether4 list=LAN
add interface="VLAN 10" list=VLAN
add interface="VLAN 2" list=VLAN
add interface="VLAN 20" list=VLAN
add interface="VLAN 25" list=VLAN
add interface="VLAN 250" list=VLAN
add interface="VLAN 30" list=VLAN
add interface="VLAN 5" list=VLAN
add interface=backup-bridge list=LAN
add interface=sfp-sfpplus1 list=LAN
add interface=wireguard1 list=VLAN

Switch:

# oct/30/2024 09:29:48 by RouterOS 6.49.17
# software id = GANV-3F2U
#
# model = CRS328-24P-4S+
# serial number = XX
/interface bridge
add name=backup-bridge protocol-mode=none
add admin-mac= auto-mac=no comment=defconf \
    ingress-filtering=yes name=bridge priority=0x3000 vlan-filtering=yes
/interface ethernet
set [ find default-name=ether1 ] comment=""
set [ find default-name=ether2 ] comment=""
set [ find default-name=ether3 ] comment=""
set [ find default-name=ether4 ] comment="t"
set [ find default-name=ether5 ] comment=""
set [ find default-name=ether6 ] comment=""
set [ find default-name=ether7 ] comment=""
set [ find default-name=ether8 ] comment=""
set [ find default-name=ether9 ] comment=""
set [ find default-name=ether10 ] comment=""
set [ find default-name=ether11 ] comment=""
set [ find default-name=ether12 ] comment=""
set [ find default-name=ether13 ] comment="" disabled=yes
set [ find default-name=ether14 ] comment=""
set [ find default-name=ether15 ] comment=""
set [ find default-name=ether16 ] comment=""
set [ find default-name=ether19 ] comment=""
set [ find default-name=ether20 ] comment=""
set [ find default-name=ether21 ] comment=""
set [ find default-name=ether23 ] comment=""
set [ find default-name=ether24 ] comment=""
set [ find default-name=sfp-sfpplus1 ] advertise="10M-half,10M-full,100M-half,\
    100M-full,1000M-half,1000M-full,2500M-full,5000M-full,10000M-full" \
    comment="Uplink to Router"
set [ find default-name=sfp-sfpplus2 ] advertise="10M-half,10M-full,100M-half,\
    100M-full,1000M-half,1000M-full,2500M-full,5000M-full,10000M-full" \
    comment="QNAP Top"
set [ find default-name=sfp-sfpplus3 ] advertise="10M-half,10M-full,100M-half,\
    100M-full,1000M-half,1000M-full,2500M-full,5000M-full,10000M-full"
set [ find default-name=sfp-sfpplus4 ] advertise="10M-half,10M-full,100M-half,\
    100M-full,1000M-half,1000M-full,2500M-full,5000M-full,10000M-full" \
    comment="QNAP Bottom"
/interface vlan
add interface=bridge name="VLAN 2" vlan-id=2
/interface bonding
add mode=802.3ad name=QNAP slaves=sfp-sfpplus2,sfp-sfpplus4
/interface list
add name=VLAN
add name=Uplink
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/interface bridge port
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether2 \
    pvid=5
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether3 \
    pvid=5
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether4 \
    pvid=5
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether5 \
    pvid=5
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether6 \
    pvid=5
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether7 \
    pvid=5
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether8 \
    pvid=5
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether9 \
    pvid=2
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether10 \
    pvid=5
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether11 \
    pvid=2
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether12 \
    pvid=2
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether13 \
    pvid=250
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether14 \
    pvid=250
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether15 \
    pvid=2
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether16 \
    pvid=250
add bridge=backup-bridge comment=defconf ingress-filtering=yes interface=\
    ether17
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether18 \
    pvid=2
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether19
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether20 \
    pvid=2
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether21 \
    pvid=2
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether22 \
    pvid=2
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether23 \
    pvid=2
add bridge=bridge comment=defconf ingress-filtering=yes interface=ether24 \
    pvid=2
add bridge=bridge comment=defconf ingress-filtering=yes interface=\
    sfp-sfpplus3
add bridge=bridge comment=QNAP ingress-filtering=yes interface=QNAP pvid=2
add bridge=bridge disabled=yes ingress-filtering=yes interface=ether1 \
    internal-path-cost=15 path-cost=15 point-to-point=yes pvid=2
add bridge=bridge ingress-filtering=yes interface=sfp-sfpplus1 \
    internal-path-cost=5 path-cost=5 point-to-point=yes priority=0x70 pvid=2
/interface bridge vlan
add bridge=bridge tagged="bridge,ether18,ether11,ether23,ether22,ether20,ether\
    24,ether21,ether15,ether1,sfp-sfpplus1" untagged=\
    ether16,ether12,ether9,QNAP vlan-ids=2
add bridge=bridge tagged="ether23,ether24,ether22,ether20,ether11,bridge,ether\
    21,ether15,QNAP,ether1,sfp-sfpplus1" untagged=\
    ether7,ether8,ether10,ether2,ether3,ether4,ether5,ether6 vlan-ids=5
add bridge=bridge tagged="QNAP,ether18,ether11,ether23,ether15,ether2,ether3,e\
    ther4,ether6,ether20,ether22,ether24,bridge,ether21,ether1,sfp-sfpplus1" \
    vlan-ids=25
add bridge=bridge tagged="QNAP,ether18,ether11,ether23,ether24,ether2,ether3,e\
    ther4,ether6,ether22,ether20,bridge,ether21,ether15,ether1,sfp-sfpplus1" \
    untagged=ether13,ether14,ether16 vlan-ids=250
add bridge=bridge untagged=bridge vlan-ids=1
add bridge=bridge tagged="ether18,ether11,ether23,bridge,QNAP,ether15,ether20,\
    ether21,ether24,ether1,sfp-sfpplus1" vlan-ids=20
add bridge=bridge tagged=QNAP,ether1,bridge,sfp-sfpplus1 vlan-ids=30
add bridge=bridge disabled=yes tagged=QNAP,ether1,sfp-sfpplus1,bridge \
    vlan-ids=10
/interface detect-internet
set detect-interface-list=Uplink
/interface list member
add interface=sfp-sfpplus1 list=Uplink
add interface=ether1 list=Uplink

I suggest you to upgrade ROS to latest v7 (7.16.1 at this time) on CRS. There were some changes in how RSTP works after 7.13.

Also I wouldn’t bother with setting port properties, bridge priority setting should do just fine. Port cost is inversely proportional to port speed, even without manually setting port property the 10Gbps SFP+ will be preferred link between both bridges.

Another thing: set sensible value to admin-mac property or unset it altogether. Settibg it to empty value may be causing problems.

Hi cpeshalelimpix,

Here’s another suggestion: try disabling the “detect internet” feature on the switch, or even better, on both devices.

Thanks for replying guys, it does indeed appear to be the detect internet on the switch that was causing the flapping / packet loss.

Thanks so much for your guy’s help!