CCR2216-1G-12XS-2XQ interconnect with CRS354-48G-4S+2Q+RM

[mx5gr]

Hello to All,

We are looking the alternatives to connect 2xCCR2216-1G-12XS-2XQ units with multiple CRS354-48G-4S+2Q+RM units.

CCR2216-1G-12XS-2XQ has 12x25G QSFP28 ports and 2x100G QSFP28 ports
CRS354-48G-4S+2Q+RM has 4x10G SFP+ ports and 2x40G QSFP+ ports

One of the 100G ports in CCR2216-1G-12XS-2XQ will be used for router interconnect for redundancy. We were looking for a Mikrotik 100G optical module, however as we could not locate one we are considering 85DMTP100D (https://www.optcore.net/product/q2885dmtp100d-mik/)

As for interconnection between the CCR2216 units and the CRS354s, we were considering the usage of Q+85MP01D modules at both ends. These are 40G QSFP+ modules.

However, as CCR2216-1G-12XS-2XQ has QSFP28 and not QSFP+ ports:

1. Can the Q+85MP01D modules be installed at both ends, albeit operating at a lower speed (10 GB)?
2. Which modules can be used at both sides to interconnect these units at maximum speed?

Distance between the routers and the switches are below 300m, MultiMode fiber cabling is already in place.

Your help is highly appreciated, as we are not familiar with the QSFP+/QSFP28 standards.

Thank you!

[mbovenka]

2. Which modules can be used at both sides to interconnect these units at maximum speed?

Distance between the routers and the switches are below 300m, MultiMode fiber cabling is already in place.

Your help is highly appreciated, as we are not familiar with the QSFP+/QSFP28 standards.

100G QSFP28 is backwards compatible with 40G QSFP+, so they should connect at 40G using the Q+85MP01D or equivalent (at 4 lanes of 10G instead of 4 lanes of 25G, same fallback as SFP28 → SFP+, which is 1 x 10G instead of 1 x 25G).

[mx5gr]

100G QSFP28 is backwards compatible with 40G QSFP+, so they should connect at 40G using the Q+85MP01D or equivalent (at 4 lanes of 10G instead of 4 lanes of 25G, same fallback as SFP28 → SFP+, which is 1 x 10G instead of 1 x 25G).

I read the first mentioned article before posting, however it mentions the QSFP28 100G port and not the QSFP28 25G ports, which are available on CCR2216-1G-12XS-2XQ.

Unfortunately, we plan to use about 8 units of CRS354-48G-4S+2Q+RM and there is only one available 100G port on either CCR2216-1G-12XS-2XQ (the other serves as a 100G router interconnect). Thus, the usage of QSFP28 ports along with (Q)SFP+ counterparts is a one-way street.

We’d like QSFP28 ports on the switch side instead of QSFP+ or QSFP+ ports instead of QSFP28 on the router side so that we can utilize the full available bandwidth (40G or 25G), however the “mix and match” between the two seems to sacrifice performance and lock the maximum available speed at 10G.

Can we use Q+85MP01D with the QSFP28 25G ports on CCR2216-1G-12XS-2XQ operating at 10G and have the same module (Q+85MP01D) on the CRS354-48G-4S+2Q+RM side?

[mbovenka]

We’d like QSFP28 ports on the switch side instead of QSFP+ or QSFP+ ports instead of QSFP28 on the router side so that we can utilize the full available bandwidth (40G or 25G), however the “mix and match” between the two seems to sacrifice performance and lock the maximum available speed at 10G.

Can we use Q+85MP01D with the QSFP28 25G ports on CCR2216-1G-12XS-2XQ operating at 10G and have the same module (Q+85MP01D) on the CRS354-48G-4S+2Q+RM side?

To recap:
SFP28 is single lane 25G, with fallback to single lane 10G. This is what the CCR2216 has as ‘low speed’ ports.
QSFP28 is 100G, 4 lanes of 25G, with fallback to 40G (4 lanes of 10G). The CCR2216 ‘high speed’ ports.
QSFP+ is 40G, 4 lanes of 10G. This is what the CRS354 has as ‘high speed’ ports.

QSFP28 ↔ QSFP+ gives you 40G (4 lanes of 10G). This you can do with the Q+85MP01D at both ends. But you need to use the 100G QSFP28 port at the CCR end; the SFP28 (not QSFP28) ports in the CCR are single lane and the Q+85MP01D will not work in them, it needs 4.

The SFP28 ports in the CCR will fall back to 10G if you put an SFP+ transceiver in them, but that’s as high as they will go connecting to a CRS354, as no port on those will do 25G.

So you can either go 40G with the 100G port you have left, or 10G with the 25G ports. That’s it.

[mx5gr]

We’d like QSFP28 ports on the switch side instead of QSFP+ or QSFP+ ports instead of QSFP28 on the router side so that we can utilize the full available bandwidth (40G or 25G), however the “mix and match” between the two seems to sacrifice performance and lock the maximum available speed at 10G.

Can we use Q+85MP01D with the QSFP28 25G ports on CCR2216-1G-12XS-2XQ operating at 10G and have the same module (Q+85MP01D) on the CRS354-48G-4S+2Q+RM side?

To recap:
SFP28 is single lane 25G, with fallback to single lane 10G. This is what the CCR2216 has as ‘low speed’ ports.
QSFP28 is 100G, 4 lanes of 25G, with fallback to 40G (4 lanes of 10G). The CCR2216 ‘high speed’ ports.
QSFP+ is 40G, 4 lanes of 10G. This is what the CRS354 has as ‘high speed’ ports.

QSFP28 ↔ QSFP+ gives you 40G (4 lanes of 10G). This you can do with the Q+85MP01D at both ends. But you need to use the 100G QSFP28 port at the CCR end; the SFP28 (not QSFP28) ports in the CCR are single lane and the Q+85MP01D will not work in them, it needs 4.

The SFP28 ports in the CCR will fall back to 10G if you put an SFP+ transceiver in them, but that’s as high as they will go connecting to a CRS354, as no port on those will do 25G.

So you can either go 40G with the 100G port you have left, or 10G with the 25G ports. That’s it.

Thank you for your detailed answer!

Thus, if we use the S+85DLC03D SFP+ modules instead of the Q+85MP01D ones, we’ll have 10G connectivity between the “low speed” CCR2216 ports and the respective “low speed” ports of the CRS354.

Otherwise, as we have only 2x100G spare ports on the CCR2216, we can accomodate only two of the CRS354 units at 40G speed using QQ+85MP01D modules at both ends.

[mbovenka]

Thank you for your detailed answer!

Thus, if we use the S+85DLC03D SFP+ modules instead of the Q+85MP01D ones, we’ll have 10G connectivity between the “low speed” CCR2216 ports and the respective “low speed” ports of the CRS354.

Otherwise, as we have only 2x100G spare ports on the CCR2216, we can accomodate only two of the CRS354 units at 40G speed using QQ+85MP01D modules at both ends.

Yes, that’s pretty much it.