repcio
October 2, 2023, 9:39pm
1
I have CRS354-48G-4S+2Q+ connected to 2 QNAP on both I have bonding connection configured.
when bonding is enabled on CRS354 I’m getting 100% CPU usage and throughput aprox. 100-120Mbit.
Using bonding (2x10GE)
rsync:~$ speedtest
Speedtest by Ookla
Server: Korbank S.A - Wroclaw (id: 5679)
ISP: Netia SA
Idle Latency: 5.37 ms (jitter: 0.06ms, low: 5.33ms, high: 5.44ms)
Download: 106.47 Mbps (data used: 159.0 MB)
13.33 ms (jitter: 8.71ms, low: 4.73ms, high: 451.13ms)
Upload: 117.37 Mbps (data used: 85.1 MB)
5.18 ms (jitter: 0.75ms, low: 4.05ms, high: 12.19ms)
Packet Loss: 0.8%
Result URL: https://www.speedtest.net/result/c/b80d3899-101e-4fec-8638-0a4b3b59e037
Without bonding (using 1x10GE)
rsync:~$ speedtest
Speedtest by Ookla
Server: Korbank S.A - Wroclaw (id: 5679)
ISP: Netia SA
Idle Latency: 5.37 ms (jitter: 0.03ms, low: 5.31ms, high: 5.39ms)
Download: 933.21 Mbps (data used: 456.4 MB)
11.66 ms (jitter: 1.63ms, low: 4.76ms, high: 24.22ms)
Upload: 289.96 Mbps (data used: 218.3 MB)
5.92 ms (jitter: 0.92ms, low: 3.93ms, high: 21.11ms)
Packet Loss: 0.0%
Result URL: https://www.speedtest.net/result/c/9d18dfcd-599b-4372-827f-f0c73d0d0e51
since you don’t put any network diagram or configuration i’m going to assume you use this switch as your NAT router therefore use hw offloaded nat since your device support that feature
https://help.mikrotik.com/docs/display/ROS/L3+Hardware+Offloading
if the assumption is wrong please put network diagram so that others can help you, CRS354 is a switch with limited router functionality but when hw offload nat is used it will help you to achieved the traffic but still it bound to your hardware limitation, in CRS317 i can easily achieved gig speed in speedtest without breaking a sweat with hw offloaded nat but i have yet to test that along side with lacp
repcio
October 3, 2023, 12:16am
3
I have attached my network layout.
Sounds almost like the bond is not being hardware offloaded. Can you export your config so we know what your config is?
repcio
October 3, 2023, 7:59am
5
This is configuration with disabled bonding on 10Ge connection to QNAP-2
/interface bridge
add add-dhcp-option82=yes admin-mac=18:FD:74:AA:AA:AA auto-mac=no comment=defconf dhcp-snooping=yes igmp-snooping=yes name=\
bridge
/interface ethernet
set [ find default-name=ether1 ] l2mtu=9216 mtu=9000
set [ find default-name=ether2 ] l2mtu=9216 mtu=9000
set [ find default-name=ether3 ] l2mtu=9216 mtu=9000
set [ find default-name=ether4 ] disabled=yes l2mtu=9216 mtu=9000
set [ find default-name=ether5 ] l2mtu=9216 mtu=9000
set [ find default-name=ether6 ] l2mtu=9216 mtu=9000
set [ find default-name=ether7 ] l2mtu=9216 mtu=9000
set [ find default-name=ether8 ] l2mtu=9216 mtu=9000
set [ find default-name=ether9 ] l2mtu=9216 mtu=9000
set [ find default-name=ether10 ] l2mtu=9216 mtu=9000
set [ find default-name=ether11 ] l2mtu=9216 mtu=9000
set [ find default-name=ether12 ] disabled=yes l2mtu=9216 mtu=9000
set [ find default-name=ether13 ] l2mtu=9216 mtu=9000
set [ find default-name=ether14 ] comment=TVS-675-1G l2mtu=9216 mtu=9000
set [ find default-name=ether15 ] comment=QNAP-OLD l2mtu=9216 mtu=9000
set [ find default-name=ether16 ] l2mtu=9216 mtu=9000
set [ find default-name=ether17 ] l2mtu=9216 mtu=9000
set [ find default-name=ether18 ] l2mtu=9216 mtu=9000
set [ find default-name=ether19 ] comment=QNAP-OLD l2mtu=9216 mtu=9000
set [ find default-name=ether20 ] l2mtu=9216 mtu=9000
set [ find default-name=ether21 ] l2mtu=9216 mtu=9000
set [ find default-name=ether22 ] l2mtu=9216 mtu=9000
set [ find default-name=ether23 ] l2mtu=9216 mtu=9000
set [ find default-name=ether24 ] l2mtu=9216 mtu=9000
set [ find default-name=ether25 ] l2mtu=9216 mtu=9000
set [ find default-name=ether26 ] l2mtu=9216 mtu=9000
set [ find default-name=ether27 ] l2mtu=9216 mtu=9000
set [ find default-name=ether28 ] l2mtu=9216 mtu=9000
set [ find default-name=ether29 ] l2mtu=9216 mtu=9000
set [ find default-name=ether30 ] l2mtu=9216 mtu=9000
set [ find default-name=ether31 ] l2mtu=9216 mtu=9000
set [ find default-name=ether32 ] l2mtu=9216 mtu=9000
set [ find default-name=ether33 ] l2mtu=9216 mtu=9000
set [ find default-name=ether34 ] l2mtu=9216 mtu=9000
set [ find default-name=ether35 ] l2mtu=9216 mtu=9000
set [ find default-name=ether36 ] l2mtu=9216 mtu=9000
set [ find default-name=ether37 ] l2mtu=9216 mtu=9000
set [ find default-name=ether38 ] l2mtu=9216 mtu=9000
set [ find default-name=ether39 ] l2mtu=9216 mtu=9000
set [ find default-name=ether40 ] l2mtu=9216 mtu=9000
set [ find default-name=ether41 ] comment="PDU " l2mtu=9216 mtu=9000
set [ find default-name=ether42 ] l2mtu=9216 mtu=9000
set [ find default-name=ether43 ] l2mtu=9216 mtu=9000
set [ find default-name=ether44 ] l2mtu=9216 mtu=9000
set [ find default-name=ether45 ] comment=EAP670_2 l2mtu=9216 mtu=9000
set [ find default-name=ether46 ] comment=EAP670_1 l2mtu=9216 mtu=9000
set [ find default-name=ether47 ] comment=EAP655-WALL l2mtu=9216 mtu=9000
set [ find default-name=ether48 ] comment=Router l2mtu=9216 mtu=9000
set [ find default-name=ether49 ] l2mtu=2028 mtu=2028 name=ether49-mgmt
set [ find default-name=qsfpplus1-1 ] auto-negotiation=no l2mtu=9216 mtu=9000 speed=10Gbps
set [ find default-name=qsfpplus1-2 ] l2mtu=9216 mtu=9000
set [ find default-name=qsfpplus1-3 ] l2mtu=9216 mtu=9000
set [ find default-name=qsfpplus1-4 ] l2mtu=9216 mtu=9000
set [ find default-name=qsfpplus2-1 ] auto-negotiation=no l2mtu=9216 mtu=9000 speed=10Gbps
set [ find default-name=qsfpplus2-2 ] auto-negotiation=no disabled=yes l2mtu=9216 mtu=9000 speed=10Gbps
set [ find default-name=qsfpplus2-3 ] l2mtu=9216 mtu=9000
set [ find default-name=qsfpplus2-4 ] l2mtu=9216 mtu=9000
set [ find default-name=sfp-sfpplus1 ] comment=Warsztat disabled=yes l2mtu=9216 mtu=9000
set [ find default-name=sfp-sfpplus2 ] advertise=\
10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full,10000M-full,2500M-full,5000M-full comment=EAP670_2 l2mtu=9216 \
mtu=9000
set [ find default-name=sfp-sfpplus3 ] advertise=\
10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full,10000M-full,2500M-full,5000M-full comment=EAP670_1 l2mtu=9216 \
mtu=9000
set [ find default-name=sfp-sfpplus4 ] auto-negotiation=no comment=Warsztat disabled=yes l2mtu=9216 loop-protect=on mtu=9000 \
speed=10Gbps
/interface bonding
add mtu=9000 name=bonding-qnap2G slaves=ether15,ether19
add disabled=yes mtu=9000 name=bonding-qnap10G slaves=qsfpplus2-1,qsfpplus2-2
/interface list
add name=WAN
add name=LAN
/interface lte apn
set [ find default=yes ] ip-type=ipv4 use-network-apn=no
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/port
set 0 name=serial0
/snmp community
set [ find default=yes ] disabled=yes
add addresses=::/0 name=public-r0
/interface bridge port
add bridge=bridge comment=defconf ingress-filtering=no interface=ether1 trusted=yes
add bridge=bridge comment=defconf ingress-filtering=no interface=ether2
add bridge=bridge comment=defconf ingress-filtering=no interface=ether3
add bridge=bridge comment=defconf ingress-filtering=no interface=ether4
add bridge=bridge comment=defconf ingress-filtering=no interface=ether5
add bridge=bridge comment=defconf ingress-filtering=no interface=ether6
add bridge=bridge comment=defconf ingress-filtering=no interface=ether7
add bridge=bridge comment=defconf ingress-filtering=no interface=ether8
add bridge=bridge comment=defconf ingress-filtering=no interface=ether9
add bridge=bridge comment=defconf ingress-filtering=no interface=ether10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether11
add bridge=bridge comment=defconf ingress-filtering=no interface=ether12
add bridge=bridge comment=defconf ingress-filtering=no interface=ether13
add bridge=bridge comment=defconf ingress-filtering=no interface=ether14
add bridge=bridge comment=defconf disabled=yes ingress-filtering=no interface=ether15
add bridge=bridge comment=defconf ingress-filtering=no interface=ether16
add bridge=bridge comment=defconf ingress-filtering=no interface=ether17
add bridge=bridge comment=defconf ingress-filtering=no interface=ether18
add bridge=bridge comment=defconf disabled=yes ingress-filtering=no interface=ether19
add bridge=bridge comment=defconf ingress-filtering=no interface=ether20
add bridge=bridge comment=defconf ingress-filtering=no interface=ether21
add bridge=bridge comment=defconf ingress-filtering=no interface=ether22
add bridge=bridge comment=defconf ingress-filtering=no interface=ether23
add bridge=bridge comment=defconf ingress-filtering=no interface=ether24
add bridge=bridge comment=defconf ingress-filtering=no interface=ether25
add bridge=bridge comment=defconf ingress-filtering=no interface=ether26
add bridge=bridge comment=defconf ingress-filtering=no interface=ether27
add bridge=bridge comment=defconf ingress-filtering=no interface=ether28
add bridge=bridge comment=defconf ingress-filtering=no interface=ether29
add bridge=bridge comment=defconf ingress-filtering=no interface=ether30
add bridge=bridge comment=defconf ingress-filtering=no interface=ether31
add bridge=bridge comment=defconf ingress-filtering=no interface=ether32
add bridge=bridge comment=defconf ingress-filtering=no interface=ether33
add bridge=bridge comment=defconf ingress-filtering=no interface=ether34
add bridge=bridge comment=defconf ingress-filtering=no interface=ether35
add bridge=bridge comment=defconf ingress-filtering=no interface=ether36
add bridge=bridge comment=defconf ingress-filtering=no interface=ether37
add bridge=bridge comment=defconf ingress-filtering=no interface=ether38
add bridge=bridge comment=defconf ingress-filtering=no interface=ether39
add bridge=bridge comment=defconf ingress-filtering=no interface=ether40
add bridge=bridge comment=defconf ingress-filtering=no interface=ether41
add bridge=bridge comment=defconf ingress-filtering=no interface=ether42
add bridge=bridge comment=defconf ingress-filtering=no interface=ether43
add bridge=bridge comment=defconf ingress-filtering=no interface=ether44
add bridge=bridge comment=EAP670_2 ingress-filtering=no interface=ether45 trusted=yes
add bridge=bridge comment=EAP670_1 ingress-filtering=no interface=ether46 trusted=yes
add bridge=bridge comment=defconf ingress-filtering=no interface=ether47 trusted=yes
add bridge=bridge comment=Router ingress-filtering=no interface=ether48 trusted=yes
add bridge=bridge comment=defconf disabled=yes ingress-filtering=no interface=ether49-mgmt
add bridge=bridge comment=defconf ingress-filtering=no interface=qsfpplus1-1
add bridge=bridge comment=defconf ingress-filtering=no interface=qsfpplus1-2
add bridge=bridge comment=defconf ingress-filtering=no interface=qsfpplus1-3
add bridge=bridge comment=defconf ingress-filtering=no interface=qsfpplus1-4
add bridge=bridge comment=defconf ingress-filtering=no interface=qsfpplus2-1 trusted=yes
add bridge=bridge comment=defconf disabled=yes ingress-filtering=no interface=qsfpplus2-2
add bridge=bridge comment=defconf ingress-filtering=no interface=qsfpplus2-3
add bridge=bridge comment=defconf ingress-filtering=no interface=qsfpplus2-4
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus1 trusted=yes
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus2 trusted=yes
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus3 trusted=yes
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus4 trusted=yes
add bridge=bridge disabled=yes interface=bonding-qnap10G trusted=yes
add bridge=bridge interface=bonding-qnap2G
/ip neighbor discovery-settings
set discover-interface-list=all
/ip settings
set max-neighbor-entries=8192
/ipv6 settings
set disable-ipv6=yes max-neighbor-entries=8192
/interface list member
add interface=ether1 list=LAN
add interface=ether2 list=LAN
add interface=ether3 list=LAN
add interface=ether4 list=LAN
add interface=ether5 list=LAN
add interface=ether6 list=LAN
add interface=ether7 list=LAN
add interface=ether8 list=LAN
add interface=ether9 list=LAN
add interface=ether10 list=LAN
add interface=ether11 list=LAN
add interface=ether12 list=LAN
add interface=ether13 list=LAN
add interface=ether14 list=LAN
add interface=ether15 list=LAN
add interface=ether16 list=LAN
add interface=ether17 list=LAN
add interface=ether18 list=LAN
add interface=ether19 list=LAN
add interface=ether20 list=LAN
add interface=ether21 list=LAN
add interface=ether22 list=LAN
add interface=ether23 list=LAN
add interface=ether24 list=LAN
add interface=ether25 list=LAN
add interface=ether26 list=LAN
add interface=ether27 list=LAN
add interface=ether28 list=LAN
add interface=ether29 list=LAN
add interface=ether30 list=LAN
add interface=ether31 list=LAN
add interface=ether32 list=LAN
add interface=ether33 list=LAN
add interface=ether34 list=LAN
add interface=ether35 list=LAN
add interface=ether36 list=LAN
add interface=ether37 list=LAN
add interface=ether38 list=LAN
add interface=ether39 list=LAN
add interface=ether40 list=LAN
add interface=ether41 list=LAN
add interface=ether42 list=LAN
add interface=ether43 list=LAN
add interface=ether44 list=LAN
add interface=ether45 list=LAN
add interface=ether46 list=LAN
add interface=ether47 list=LAN
add interface=ether48 list=LAN
add interface=ether49-mgmt list=LAN
add interface=qsfpplus1-1 list=LAN
add interface=qsfpplus1-2 list=LAN
add interface=qsfpplus1-3 list=LAN
add interface=qsfpplus1-4 list=LAN
add interface=qsfpplus2-1 list=LAN
add interface=qsfpplus2-2 list=LAN
add interface=qsfpplus2-3 list=LAN
add interface=qsfpplus2-4 list=LAN
add interface=sfp-sfpplus1 list=LAN
add interface=sfp-sfpplus2 list=LAN
add interface=sfp-sfpplus3 list=LAN
add interface=sfp-sfpplus4 list=LAN
/interface ovpn-server server
set auth=sha1,md5
/ip address
add address=192.168.1.100/24 disabled=yes interface=sfp-sfpplus2 network=192.168.1.0
/ip dhcp-client
add interface=bridge
/ip dns
set servers=192.168.102.200
/ip service
set telnet disabled=yes
set ftp disabled=yes
/routing bfd configuration
add disabled=no
/system clock
set time-zone-name=Europe/Warsaw
/system health settings
set fan-full-speed-temp=60C fan-min-speed-percent=10% fan-target-temp=54C
/system identity
set name=MT-XXXXXX
/system note
set show-at-login=no
/system ntp client
set enabled=yes
/system routerboard settings
set boot-delay=1s boot-os=router-os
/system swos
set identity=SwOS-XXXXX static-ip-address=192.168.102.2
/tool bandwidth-server
set enabled=no
/tool sniffer
set filter-interface=ether38
You don’t specify a bonding mode, and the old docs state that the default bonding mode is not one of the supported bonding modes. (the current docs don’t state the default mode).
I would set it to 802.3ad or balance-xor manually (whatever the qnap supports) and then monitor the bond to see the status of the link / bond process. Please also note if the bond is being hardware offloaded in the status.
https://help.mikrotik.com/docs/display/ROS/Bonding
repcio
October 3, 2023, 11:29am
7
It’s strange as in Winbox it is configured and it is balance-rr.
repcio
October 4, 2023, 9:46am
8
With balance xor - it is not causing CPU usage and I’m able to pass this 100Mbit limit.
