Community discussions

MikroTik App
 
User avatar
raistlin84
just joined
Topic Author
Posts: 1
Joined: Wed Dec 02, 2020 12:38 pm

CRS354 + Intel XL710-QDA1: 40G link sometime takes minutes

Wed Dec 02, 2020 1:47 pm

Hi guys, I have a rather noob question here:
I have widely varying durations to establish a 40G link between my Intel XL710-QDA1 cards and CRS354 switches, which I do not understand.

I recently upgraded my home network with two CRS354 switches/routers for the two 40G ports. There is one big NAS server in the basement and one powerful workstation on the ground floor, all the other network participants are rather low-powered. Basically, I have set up a high-speed 40G highway with a number of somewhat irrelevant smaller clients on the side:

Workstation --- CRS354 --- CRS354 --- NAS

The two CRS354 switches are connected via an older existing LC/LC-terminated OM3 fibre using "Mikrotik Q+31DLC2D Compatible 40GBASE-LX4 QSFP+" optical transceivers that I got from fs.com. This works perfectly, no problems here. The cable is just 50m, so not too much signal degradation. The connection/link between the two switches/routers is up within a second and is rock solid stable, just like it was before when I was using the same cable for 10G-connections. So far, so good.

In the NAS and the workstation I use "Intel XL710-QDA1" 40G-network cards, based on my very positive experience with the Intel X710-DA2 cards (it's the same chip, just different network terminals, if you will). Now, the weird issue for me is, that it sometime takes 1-2 seconds for the connection/link between the CRS354 and card to go up, but other times several minutes until the link finally turns green, and quite often something in between. This issue occurs with both computers and various different cables:
  • Intel Original Direct Attach 1m Cable
  • Intel Original AOC 10m Cable
  • Intel-compatible 40GBase-SR4 to MikroTik-original 40GBase-SR4 optical modules
Therefore, I guess it is not a "broken cable" issue. Since it is happening on both computers, I guess it is also not a hardware fault. Once the connection/link is up, everything is fine and rock-solid stable, but it is just very odd for me that there is such a huge variance in the time it takes to finally establish the connection. Both Intel network cards have the most recent NVM firmware and I use up-to-date drivers. Like I said, it's not stability issue, it's the widely varying time to establish the connection. When I reboot the workstation, sometime the machine up-and-ready on the desktop and I am out of the network connection for 2 minutes. Not a catastrophe, but odd nonetheless...

Since is the first time I am dealing with 40G hardware, I am just wondering if this is normal behavior or if I am missing some setting in the cards or the switch.

Any insights would be highly appreciated.
Best regards!

Who is online

Users browsing this forum: No registered users and 50 guests