Community discussions

MikroTik App
 
awes1
just joined
Topic Author
Posts: 10
Joined: Fri Mar 05, 2021 9:43 pm

QSFP Bonding

Fri Mar 04, 2022 5:04 pm

Hi,

CRS354, i need to create bond from two QSFP interfaces. ROS shows qsfpplus1-1 (to 1-4) and qsfpplus2-1 (to 2-4).
Do all of them should be slaves of a bond ?
 
Zacharias
Forum Guru
Forum Guru
Posts: 3459
Joined: Tue Dec 12, 2017 12:58 am
Location: Greece

Re: QSFP Bonding

Fri Mar 04, 2022 9:08 pm

I 've not used CRS354 yet, however as i can see from the manual it has 2 QSFP+ ports...
You mention 8 ?
Do all of them should be slaves of a bond ?
No.
 
awes1
just joined
Topic Author
Posts: 10
Joined: Fri Mar 05, 2021 9:43 pm

Re: QSFP Bonding

Sat Mar 05, 2022 12:12 am

Obviously this one has 2 QSFP physical ports. In routeros those are represented by 8 virtual interfaces as i described previously. So im not sure if i should bond them all in case of single port.
 
Zacharias
Forum Guru
Forum Guru
Posts: 3459
Joined: Tue Dec 12, 2017 12:58 am
Location: Greece

Re: QSFP Bonding

Sat Mar 05, 2022 9:45 am

Read this post here viewtopic.php?t=151761#p747879 from @mkx
Its about a CRS326, but i guess tha same principle applies...
If you coneect that QSFP+ port to another QSFP+ port of another switch, can you still see the 4 QSFP Virtual Interfaces ? Or now it shows as one ?
 
awes1
just joined
Topic Author
Posts: 10
Joined: Fri Mar 05, 2021 9:43 pm

Re: QSFP Bonding

Sat Mar 05, 2022 11:06 am

4 interfaces still apears after connecting.
 
Zacharias
Forum Guru
Forum Guru
Posts: 3459
Joined: Tue Dec 12, 2017 12:58 am
Location: Greece

Re: QSFP Bonding

Sat Mar 05, 2022 11:22 am

What is the version running on the CRS354 ?
Can you please provide a screenshot of the interfaces after connecting the QSFP+ port on both sides ?
 
awes1
just joined
Topic Author
Posts: 10
Joined: Fri Mar 05, 2021 9:43 pm

Re: QSFP Bonding

Sat Mar 05, 2022 11:45 am

Attached

CRS354-48G-4S+2Q+, fw 6.46.3
You do not have the required permissions to view the files attached to this post.
 
Zacharias
Forum Guru
Forum Guru
Posts: 3459
Joined: Tue Dec 12, 2017 12:58 am
Location: Greece

Re: QSFP Bonding

Sat Mar 05, 2022 11:48 am

Is there a bond created for QSFP1-1,QSFP1-2,QSFP1-3 and QSFP1-4 ?
What is the negotiotaion Rate of the QSFP interface ?
What is the module used ?

Also on versions 6.49.x there are improvements on the CRS354 regrading the QSFP interfaces... Upgrade to latest V6 ( 6.49.4 ) and test again.
https://mikrotik.com/download/changelogs
 
awes1
just joined
Topic Author
Posts: 10
Joined: Fri Mar 05, 2021 9:43 pm

Re: QSFP Bonding

Sat Mar 05, 2022 11:59 am

Right now there is no bond created.

Rate is 40Gbps.

Module is based on original DAC for Mikrotik (right now no more info about it, remote infrastructure).
You do not have the required permissions to view the files attached to this post.
 
Zacharias
Forum Guru
Forum Guru
Posts: 3459
Joined: Tue Dec 12, 2017 12:58 am
Location: Greece

Re: QSFP Bonding

Sat Mar 05, 2022 12:08 pm

Update to Latest ROS version, 6.49.4 and test again...
 
awes1
just joined
Topic Author
Posts: 10
Joined: Fri Mar 05, 2021 9:43 pm

Re: QSFP Bonding

Sat Mar 05, 2022 12:51 pm

Updated to 6.48.6 LTS, no changes.

I will schedule update to 6.49.
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11433
Joined: Thu Mar 03, 2016 10:23 pm

Re: QSFP Bonding

Sat Mar 05, 2022 9:02 pm

Rate is 40Gbps.

If rate is 40Gbps, then that's what interface can do ... not sure what you expect though ...
 
Zacharias
Forum Guru
Forum Guru
Posts: 3459
Joined: Tue Dec 12, 2017 12:58 am
Location: Greece

Re: QSFP Bonding

Mon Mar 07, 2022 7:02 pm

Rate is 40Gbps.

If rate is 40Gbps, then that's what interface can do ... not sure what you expect though ...
Yes, but the question is why the physical interface is being shown as four different ones QSFP1-1,QSFP1-2,QSFP1-3 and QSFP1-4 ...
Wouldn't it be more clear if it was shown as one single interface since the rate is 40 Gbps...
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11433
Joined: Thu Mar 03, 2016 10:23 pm

Re: QSFP Bonding

Mon Mar 07, 2022 7:10 pm

Since I don't own one of those devices, I can't say for sure. But AFAIK QSFP interface can either be used as single 40Gbps interface (if connected to matching peer) or can be split to 4x 10Gbps independent interfaces. MT could possibly introduce a (software) switch toggling between both modes but it seems they rather went with autodetecting. Since it's impossible to know in advance what kind of peer is conencted to such port, there are 4 interfaces created ... but the first (master?) will show if the rest are independent or not via negotiated speed. I'm not sure if there's way to negotiate 10Gbps when complete interface (all 4 fibres) are connected to another QSFP interface ... So I guess it's down to steep learning curve of MT again. Not recommended for everybody.
 
Zacharias
Forum Guru
Forum Guru
Posts: 3459
Joined: Tue Dec 12, 2017 12:58 am
Location: Greece

Re: QSFP Bonding

Mon Mar 07, 2022 7:35 pm

Yes, maybe...
 
changuelco
just joined
Posts: 2
Joined: Fri Oct 10, 2014 6:19 pm

Re: QSFP Bonding

Thu Mar 09, 2023 6:11 pm

I have the same issue but with a CCR2216. The QSFP28 are represented as 4 separate interfaces. Using a DAC 100G cable between 2 mikrotiks, the negotiation show 100G but the bandwidth test only passed 12G. Not even 40G. I try with a bundle adding all the interfaces qspf28 but still down. Only the first one become up.
You do not have the required permissions to view the files attached to this post.
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 2990
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: QSFP Bonding

Thu Mar 09, 2023 6:14 pm

i think when you use a 40g or 100g link (not breakout cables) you only need to consider the first qsfp interface on configurations


do not expect a TCP test to be able to saturate such a "big" link, try udp test or traffic generator
 
emunt6
Frequent Visitor
Frequent Visitor
Posts: 87
Joined: Fri Feb 02, 2018 7:00 pm

Re: QSFP Bonding

Sat Mar 18, 2023 2:48 am

I have the same issue but with a CCR2216. The QSFP28 are represented as 4 separate interfaces. Using a DAC 100G cable between 2 mikrotiks, the negotiation show 100G but the bandwidth test only passed 12G. Not even 40G. I try with a bundle adding all the interfaces qspf28 but still down. Only the first one become up.
I have never used 40G link with Mikrotik. This picture shows the "QSFP-port is in "split-mode", thats why one QSFP has 4x indexed-interface.
Any other vendor, when you issue the "split-mode", after you get indexed interface ( 1-1, 1-2, 1-3, 1-4 ).
Can you change it ? ( split-mode=no, or try to disable other indexed interface leave enabled only the 1-1 ).
Weird implementation from Mikrotik ( probably broken as hell ).

Who is online

Users browsing this forum: 4l4R1, holvoetn and 24 guests