Low bandwith between CRS305 mixed single/multimode SFP+

Hi, we have two CRS305 with a singlemode-Connection about 1km distance.

some10G-Switch=>192.168.x=>multimodeSFP+=CRS305==singlemodeSFP+==CRS305==multimodeSFP+==>192.168.y=>some10GSwitch

No NAT, no FW-Policy. Everything looks fine, but i can’t get over 400MBits/sec.

Which details do you need to understand what is going on?

Regards, Dirk

How do you perform network speedtest?
At what link rate are the CRS's connected?

Hi, i did all different tests from linux- to linux-system (iperf(3)) and with the internal tool.torch and tool.ping between the interfaces between the 2 routers (both SFP+4= Port5) to the other side of the canyon.
The interfaces itself said, that they are at 10G-Speed.

BR, dirk

What is the CPU usage during tests?

Signal level, power, snr, etc is okay according the modules? They report similar values or different? Try to replace the modules if you have spare ones.

RZA1: Vendor Name ALLNET
Vendor Part Number ALL4758

Wavelength 1310.00 nm
Temperature 45 C
Supply Voltage 3.223 V
Tx Bias Current 38 mA
Tx Power -2.273 dBm
Rx Power -7.744 dBm

Rate 10Gbps
Full Duplex yes

RZA2: Vendor Name ALLNET
Vendor Part Number ALL4758

Wavelength 1310.00 nm
Temperature 47 C
Supply Voltage 3.300 V
Tx Bias Current 38 mA
Tx Power -3.757 dBm
Rx Power -4.378 dBm

Rate 10Gbps
Full Duplex Yes

Because they are single-mode, i only have a few to change. It’s an hour away from me, so stay tuned.

CPU-Usage: check_mk:

RZA1: ping <40%; torch 73%

RZA2: uuh, the last 30 days around 97%; i made a reboot; comes back to 99% after a few minutes. Where to look deeper here?

Good one: v6.49.10

“Bad” one, with high CPU-Usage: v6.49.8

Both longterm.

Can you share the config of both? Did you do anything with VLAN? Or multiple bridges?

Just checked the changelogs on the V6 branch...there doesn't seem to be a lot of changes related to this switch (on contrary to the V7 branch).

Uuuh, I’d made an Update to 6.49.19 long-term and have to learn more about it. on one side there is an additional “bridge” at the interfaces-page. I will be back, after i checked what is going on. Thanx

Ok, Bridge on one side removed and all interfaces reconnected. Still max. 400Mbps.

On the CRS305.RZ2 there is a 100% CPU-Load after a few minutes; RZ1 around 33%.

How can I dig deeper here?

BR, D.

CPU should be a couple of percent maximum when just being used as a switch. Do the bridge ports have hardware offloading enabled (Hardware Offload ticked under General tab for each interface under Bridge>Ports, or yes in HW column from /interface bridge port print) and active (H flag in column before interface name under Bridge>Ports, or from /interface bridge port print)

Hi, i can’t find these items “hardware offloading” in Bridge→Ports or find interface→bridge port print.

Any hint to a manulal page o.s.e. ?

I found this, but I’m on 6.49.19:

The L3HW Settings menu has been introduced in RouterOS version 7.6.

Sub-menu: /interface ethernet switch l3hw-settings

Can’t find these.

L3 HW offloading is only available in ROS v7. But it doesn't apply if the use case is using CRS devices as switches (which are L2 entities). It is important, though, to configure bridge and ports in the way it allows switch chip to do the work.

If, OTOH, devices are used as routers (i.e. physical interfaces, involved in data transfer, are not members of same bridge), then ... CRS are routers with shitty performance unless they are properly configured for L3 HW offload (and then you're back to the beginning of my post).

A print of my crs305 ports with default config for v6.49.19 on it.
Note the H Flags.

 /interface bridge port> print
Flags: X - disabled, I - inactive, D - dynamic, H - hw-offload 
 #     INTERFACE                                     BRIDGE                                     HW  PVID PRIORITY  PATH-COST INTERNAL-PATH-COST    HORIZON
 0   H ;;; defconf
       ether1                                        bridge                                     yes    1     0x80         10                 10       none
 1 I H ;;; defconf
       sfp-sfpplus1                                  bridge                                     yes    1     0x80         10                 10       none
 2   H ;;; defconf
       sfp-sfpplus2                                  bridge                                     yes    1     0x80         10                 10       none
 3   H ;;; defconf
       sfp-sfpplus3                                  bridge                                     yes    1     0x80         10                 10       none
 4 I H ;;; defconf
       sfp-sfpplus4                                  bridge                                     yes    1     0x80         10                 10       none

Do you perhaps have multiple bridges configured?
(very) Old firmware in system/routerboard?

Yes, used as switch (L2 entity, bridge functions offloaded to ahrdware as indicated by "H" flags). And it's unclear from information, provided by @DirkK , how he is using the two devices (as switches or as routers).

Actually now I look closer at the original question, it does seem likely that he is using them as routers.

Switch=>192.168.x=>multimodeSFP+=CRS305==singlemodeSFP+==CRS305==multimodeSFP+==>192.168.y=>some10GSwitch

If just switched, you likely don't need the CRS305 at all.

So perhaps upgrade to 7.20.8 and rejig with hardware L3 offload.
I would be hesitant doing this cold without prior testing on a non-live/test system though.

On thinking about it some more (it takes a while :frowning: )
You can probably just upgrade the near end,
and making it fast is possibly as simple as then enabling l3 hw offload on it.

Make sure you have your configs backed up somewhere (not only on router)
before doing the upgrade.

When that end is working ok, visit the other end and do the same.