Problem Hardware Offload on CRS326-24G-2S+

RouterOS General
1

Hello,

I ran into trouble with HW offloading on CRS326-24G-2S+.

On interface “uplink” an CRS326-24S+2Q+ is connected by 802.1ad (2 * 10GbE), this interface works properly
with HW offload.

On interface “backup-src” a FreeBSD 12.1 server is connected with 802.1ad (2 * GE). This
connection does not HW offload and therefore there is a poor performance and high CPU load.
802.1ad config on FreeBSD is ok, at least it works with a Cisco Nexus 3000 used for reference.

Config on CRS326-24G-2S+

# may/02/2020 12:01:18 by RouterOS 6.46.5
# software id = PUMZ-Z25Q
#
# model = CRS326-24G-2S+
# serial number = 94550BB498A4
/interface bridge
add name=bridge1 vlan-filtering=yes
add name=vlan2
/interface bridge port
add bridge=bridge1 interface=uplink
add bridge=vlan2 interface=VLAN2
add bridge=vlan2 interface=backup-srv
/interface bridge vlan
add bridge=bridge1 tagged=uplink,bridge1 untagged=vlan2 vlan-ids=2
/interface bonding
add lacp-rate=1sec mode=802.3ad name=backup-srv slaves=ether15,ether16
add lacp-rate=1sec mode=802.3ad name=uplink slaves=sfp-sfpplus1,sfp-sfpplus2

FreeBSD Interface Config

ifconfig_em0="up"
ifconfig_em1="up"
cloned_interfaces="lagg0"
ifconfig_lagg0="laggproto lacp laggport em0 laggport em1 192.168.1.4/24"
defaultrouter="192.168.1.254"

I have no clue, what is wrong with this configuration.

Please give me a hint.

best regards
Christian

2

You have two bridges, and currently only a single bridge can be hardware-offloaded on CRS3xx series devices. This is clearly documented here.

Why do you need two separate bridges?

3

Thank you for hint - obviously a configuration problem. I changed configuration, so CRS326-24G-2S+ works with one bridge
and I got HW offloading for all ports.
Only management port (IP address for Mikrotik switch itself) has no HW offloading for obvious reasons.

4

In case you want to segment your network, VLANs is what you should choose…

5

I am using VLANs, there was a misunderstanding regarding Routeros config, only.

This config works now:

# may/02/2020 17:43:23 by RouterOS 6.46.5
# software id = PUMZ-Z25Q
#
# model = CRS326-24G-2S+
/interface bridge
add name=bridge1 vlan-filtering=yes
/interface bridge port
add bridge=bridge1 interface=uplink
add bridge=bridge1 interface=VLAN4090 pvid=4090
add bridge=bridge1 interface=ether1 pvid=2
...
add bridge=bridge1 interface=backup-srv pvid=2
/interface bridge vlan
add bridge=bridge1 tagged=uplink,bridge1,ether24,ether23,ether22,ether21 untagged=\
    *22,backup-srv,ether3,ether5,ether6,ether7,ether8,ether9,ether10,ether11 vlan-ids=2

Now Ethernets are with HW offload now and no more CPU load problems:

Flags: X - disabled, I - inactive, D - dynamic, H - hw-offload 
 #     INTERFACE                          BRIDGE                         HW  PVID PRIORITY  PATH-COST INTERNAL-PATH-COST    HORIZON
 0   H uplink                             bridge1                        yes    1     0x80         10                 10       none
 1 I H ether2                             bridge1                        yes   2     0x80         10                 10       none
 2     VLAN4090                           bridge1                            4090     0x80         10                 10       none
...
23   H backup-srv                         bridge1                        yes    2     0x80         10                 10       none

best regards
Christian

6

i have CRS326-24G+2Q+ .

i have 8 remote sites which is connected to CRS326 ,let `say it A,B,C,D,E,F,G,H . Each remote sites connected with 2 link with 802.3ad Bonding to CRS326
A remote site is only using vlan 10,20
B remote site is only using vlan 30,40
C remote site is only using vlan 50,60
D remote site is only using vlan 70,80
E remote site is only using vlan 90,100
F remote site is only using vlan 110,120
G remote site is only using vlan 130,140
H remote site is only using vlan 150,160


IN CRS326
A remote site is connected to sfpplus1 and sfpplus2 with Bonding
B remote site is connected to sfpplus3 and sfpplus4 with Bonding
C remote site is connected to sfpplus5 and sfpplus6 with Bonding
D remote site is connected to sfpplus7 and sfpplus8 with Bonding
E remote site is connected to sfpplus9 and sfpplus10 with Bonding
F remote site is connected to sfpplus11 and sfpplus12 with Bonding
G remote site is connected to sfpplus13 and sfpplus14 with Bonding
H remote site is connected to sfpplus15 and sfpplus16 with Bonding
BRAS Server is connected to sfpplus17

At first ,i`m using simple one bridge which name is bridge1.
And all of the bonding interfaces and sfpplus17 added to bridge1 and hardware offload is enabled.
i saw H flag is on every ports in bridge1.
But i only got 4.6Gbps total throughput for download and upload got only 2mbps .
Then i checked the block diagram
Sfp1 to 8 is 10Gbps , Sfp9 to 16 is 10Gbps ,Sfp17 to 24 is 10Gbps and QSFP+ 1-2 is 40Gbps throughput for wirespeed.
I tried to create 4 bridges but HW is only working with one bridge .
how can i make it ? Please help me .

7

This is the only proper setup to have all the job done by the switch chip, so we have to find out what’s wrong. So first, how did you measure/from where did you get these speed figures? And is 2 Mbps in the upload direction a typo, and it should have been 2 Gbps, or is it really 2 Mbps?

Next, do you have a rough idea how many MAC addresses may there be in total in all the VLANs? Hundreds, tens of thousands?


I don’t get why the block diagram is drawn that way, but each group of 8 SFP+ ports is marked as “8×10 Gbps”, so there is no bandwidth sharing between the 8 ports, each has its own 10 Gbps. You can check that by transiting traffic from one SFP in the same group to another.