Hi,

in my home network I have a 3rd party router (Firewalla Golde SE) and two CRS switches daisy-chained one after another, i.e. Router -> CRS328 -> CRS326.

The two CRS switches are interconnected via 10Gbps multimode SFP+ link. I plan to use VLANs but these are currently not configured and "VLAN filtering" is off in bridge setting. All ports (including SFPs) in each respective swtich are bridged together in one bridge (i.e. one bridge per switch).

Looking at Resources the CRS328 shows typically 5% CPU load (oscilates between 4 - 6) and the CRS326 shows 1% CPU load.

When I run Bandwidth test with CRS328 being Server (with default settings) and CRS326 being client, I see a throughput nearing 400 Mbps but never reaching over it. CRS328 CPU load jumps to 100% and CRS326 CPU load is approx. 60%.

Reversing the Client and Server roles, i.e. CRS326 running server and CRS328 being client I get near 600 Mbps throughput and CPUs in both machines coming close to 100% (~98 or so).

Question: is this expectable performance and I should not worry or should I start hunting the culprit to what seems as pretty low throughput to me given the link physical capacity of 10 Gbps? In other words - is the CRS' CPU performance limiting Bwidth test result but with "normal" traffic the link capacity will be fully used or is there likely something rotten in my setup?

Thank you for opinions in advance!

please use iperf it was a known fact that bandwidth test inside mikrotik will eat your CPU

please use iperf it was a known fact that bandwidth test inside mikrotik will eat your CPU

Have checked iperf in relation to Mikrotik and that tool/protocol is not natively supported. Hence you probably suggest connecting two iperf devices (running iperf server and client) to each Mikrotik and generating (and consuming) traffic outside Mikrotik. Well, I don't have even one, let alone two devices capable of connecting over 10G interface to start with.

For my purpose it is ok to know that native inbuilt btest will peak at about 400 - 600 Mbps on CRS32x series so the reason in reaching 10G troughput is likely not in the link or switch environment setup but simply because the machine can't generate more traffic on its own. In other words, this is expected behavior (is it?) and I don't need to bother. (still the achieved throughput looks really low but ok - if you say this is normal I will stick to it).

It's expected.

Using the mikrotik bandwith test will never show the switching performance. Switching will be wirespeed and as you have no VLANs you can be reasonably certain that you will get full switching performance.

As @lolski already mentioned running iperf between two devices connected to the switches will give you a more accurate reading.