10G-LR instability in CCR2116-12G-4S+

Hardware MikroTik hardware general
1

Hi,

we are currrently testing CCR2116-12G-4S+ with 4x 10G-LR SFP connecting to a Juniper QFX5120-32C through 4x10G-LR breakout.
We verified the Juniper ports to be correctly working with switches and routers from other vendors.

However, the CCR doesn’t work with either of them in a stable manner. We tried 7.16b2 and 7.15.1.
We tried Fiberstore and Flexoptix 10G-LR optics, which work fine in other devices like Juniper QFX5120-48Y.
The Flexoptix are coded to Mikrotik S+31DLC10D.

In /interface/ethernet/monitor sfp-sfpplus we observe the SFP disappearing about every 20-40s, triggering RX LOS on the other side:

Jun 21 16:30:08 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx power low alarm cleared
Jun 21 16:30:08 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx power low warning cleared
Jun 21 16:30:08 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx loss cleared
Jun 21 16:30:46 hostname mib2d[11204]: SNMP_TRAP_LINK_UP: ifIndex 625, ifAdminStatus up(1), ifOperStatus up(1), ifName xe-0/0/6:2
Jun 21 16:30:47 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx power low alarm set
Jun 21 16:30:47 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx power low warning set
Jun 21 16:30:50 hostname mib2d[11204]: SNMP_TRAP_LINK_DOWN: ifIndex 625, ifAdminStatus up(1), ifOperStatus down(2), ifName xe-0/0/6:2
Jun 21 16:30:51 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx power low alarm cleared
Jun 21 16:30:51 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx power low warning cleared
Jun 21 16:30:51 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx loss cleared
Jun 21 16:31:09 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx power low alarm set
Jun 21 16:31:09 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx power low warning set
Jun 21 16:31:28 hostname mib2d[11204]: SNMP_TRAP_LINK_UP: ifIndex 625, ifAdminStatus up(1), ifOperStatus up(1), ifName xe-0/0/6:2
Jun 21 16:31:32 hostname mib2d[11204]: SNMP_TRAP_LINK_DOWN: ifIndex 625, ifAdminStatus up(1), ifOperStatus down(2), ifName xe-0/0/6:2
Jun 21 16:31:33 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx power low alarm cleared
Jun 21 16:31:33 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx power low warning cleared
Jun 21 16:31:33 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx loss cleared
Jun 21 16:32:10 hostname mib2d[11204]: SNMP_TRAP_LINK_UP: ifIndex 625, ifAdminStatus up(1), ifOperStatus up(1), ifName xe-0/0/6:2
Jun 21 16:32:11 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx power low alarm set
Jun 21 16:32:11 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx power low warning set
Jun 21 16:32:14 hostname mib2d[11204]: SNMP_TRAP_LINK_DOWN: ifIndex 625, ifAdminStatus up(1), ifOperStatus down(2), ifName xe-0/0/6:2
Jun 21 16:32:15 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx power low alarm cleared
Jun 21 16:32:15 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx power low warning cleared
Jun 21 16:32:15 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx loss cleared
Jun 21 16:32:52 hostname mib2d[11204]: SNMP_TRAP_LINK_UP: ifIndex 625, ifAdminStatus up(1), ifOperStatus up(1), ifName xe-0/0/6:2
Jun 21 16:32:53 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx power low alarm set
Jun 21 16:32:53 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx power low warning set
Jun 21 16:32:56 hostname mib2d[11204]: SNMP_TRAP_LINK_DOWN: ifIndex 625, ifAdminStatus up(1), ifOperStatus down(2), ifName xe-0/0/6:2
Jun 21 16:33:01 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx power low alarm cleared
Jun 21 16:33:01 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx power low warning cleared
Jun 21 16:33:01 hostname fpc0 qsfp-0/0/6 Chan# 2: Rx loss cleared

Routerboard firmware has been upgraded, config is reset and in default settings, also device has been hard-rebooted by removing power.

Any idea what could cause the issue?

Kind regards,
Frank

2

There is usually no need to code SFPs to Mikrotik since Mikrotik doesn’t block non-Mikrotik SFPs.

Whit that said - I’ve do have encountered SFP that don’t work properly in Mikrotiks. For example Cisco GLC-T and some strange long-range multimode ones.

3

Try enabling “Ignore Rx LOS” and see if that helps.

If not, make a supout.rif while broken, running on 7.16b2 and send it to Mikrotik support.

They have been making many changes to the SFP code…

4

Thanks for the hints. Unfortunately Ignore Rx LOS didn’t help, the behaviour stays the same. I’ll send a support file to Mikrotik support.

5

After replacing the router, the issue was resolved. Seems to be a hardware problem. We will RMA the router.