100Gbit Networkspeed maximum 25Gbit

RouterOS General
1

Hi,

I am using the following transceiver:
SPTSBP2CLCKV INTEL 100G-CWDM4 1271-1331NM OPTISCHES TRANSCEIVER-MODUL

and

SPTSBP2CLCKS INTEL 100G-CWDM4 1271-1331NM OPTISCHES TRANSCEIVER-MODUL

Both are connected to the Mikrotik crs504-4xq-in with beta RouterOS 7.14rc1

The other side is connected to a Mellanox MCX455A ECAT PCIExpeess Card 100Gbit and a Mellanox MCX555N Card.

Both OS is Debian 11. The maximum speed with iperf is 27Gbit per second. I am curious why there is a limit at 27Gbit/sec.

The link between the pc and the mikrotik switch is 100GB.

Are there more settings i have to male to achieve the full bandwidth at 100Gbit?

2

Do you have any results obtained with non-MT switch for comparision?

3

Due to the performance of the linux kernel, it is difficult for iperf to reach 100G. I only got 25G with iperf, but 39G with ntttcp on windows. Mellanox has its own performance testers, nd_write_bw.exe and nd_read_bw.exe, which use RDMA. It is very easy to reach 100G using RDMA.