CRS520 not linking up with Mellanox ConnectX-4

Hardware MikroTik hardware general
1

Hi! I bought the CRS518 switch. (typo in the title)

It has 2x100G ports.

1 of the ports goes to my CRS504 for a local 100G link (working normally).

The second port goes into Mellanox CX415A MCX415A-CCAT.

These are the transceivers on both ends: https://www.cisco.com/c/en/us/products/collateral/interfaces-modules/transceiver-modules/at-a-glance-c45-740242.pdf

I’ve tried with autoneg on and fec on auto. Did not work.

I’ve tried with forced 100G on both ends with RS-FEC and fec91, no link.

I’ve tried with forced 100G on both ends with RS-FEC and fec74, no link.

Mikrotiks and the Mellanox card are on the latest FW.

Error 36 means: No Partner detected during force mode.

Any ideas would be appreciated.

2

One, maybe dumb, question:
Are you sure that you are using MMF (multimode) cable between those two transceivers ?

3

Yes.

4

Try to make loopback, plug both transceivers into CRS518 or CRS504 to test them and the cable. Or test between CRS518 and CRS504 just to make sure transceivers and cable are ok.

5

Hi! I have 518 and 504 connected at 100G with no issues.

6

You could try forcing 40Gbe and 10Gbe to see will it connect.

7

I have tried that as well. Does not link up regardless of what speed I pick.

8

I have a similar nic card with you link to CRS317 with a regular DAC cable (non-mikrotik) and it works fine for LibreQoS

9

In the past I’ve had lot of issues with non-Mellanox branded 40Gbe cards not working with certain transceivers.
If I plug them in two switches they were working. Maybe that transceiver doesn’t like that particular Mellanox card.