I have one network that various 2216 and 2116s, will start port flapping off the switch chip. Any ports connected to the CPU are fine, the router is RUNNING, but any ports connected to the switch chip start flapping, up/down. This happening since 7.15ish to current 7.18.2. Note we have tried to go back to 7.15.3 and it still occurs. So don’t really know when it started.
The routers are locked down to only a few IPs, no services are available, synflooding protection is enabled. We did install a new 2116, and moved the hotspot/queues from the 2216, since then the 2116 has had this issue but NOT the 2216. So its looking like its something to do with hotspot/queuing etc.. These units are at 4 different locations, they use both 100G and 10G fiber and DAC Cables to OLTs in most cases. If its local to the site, normally a DAC cable is used.
We did open a ticket with MikroTik and they have been responsive, but so far they are unable to reproduce the issue.
ccr2116 and 2216 does not have any external interface connected to the CPU (except 1 ethernet management port)
all interfaces depend of switch chip
with this clarified
if you use some copper connections i would point out to and electrical problem (induction, potential difference, lack of isolation), if you only exclusively use fiber, check power quality and stability
just in case, be aware, some sfp dac cables are copper
Correct, the ether1 management does not flap, we can access it locally and RouterOS is running just fine, just all of the switch ports are flapping..
here is a clippt from the log
Mar/14/2025 06:47:19 route,info L3HW Offloading OFF
Mar/14/2025 06:47:39 interface,info qsfp28-2-1--100g link up
Mar/14/2025 06:47:39 interface,info qsfp28-2-1--100g link down
Mar/14/2025 06:47:39 interface,info sfp28-1-to--10g link down
Mar/14/2025 06:47:40 interface,info sfp28-2-to--10g link down
Mar/14/2025 06:47:40 interface,info sfp28-11-olt2-data link down
Mar/14/2025 06:47:41 interface,info sfp28-12-olt1-data link down
Mar/14/2025 06:47:42 interface,info sfp28-11-olt2-data link up (speed 10G, full duplex)
Mar/14/2025 06:47:42 interface,info sfp28-12-olt1-data link up (speed 10G, full duplex)
Mar/14/2025 06:47:42 interface,info sfp28-1-to--10g link up (speed 10G, full duplex)
Mar/14/2025 06:47:42 interface,info sfp28-2-to--10g link up (speed 10G, full duplex)
Mar/14/2025 06:47:42 interface,info qsfp28-2-1--100g link up
Mar/14/2025 06:47:43 interface,info qsfp28-2-1--100g link down
Mar/14/2025 06:47:43 interface,info sfp28-1-to--10g link down
Mar/14/2025 06:47:43 interface,info sfp28-2-to--10g link down
Mar/14/2025 06:47:43 interface,info sfp28-11-olt2-data link down
Mar/14/2025 06:47:44 interface,info sfp28-12-olt1-data link down
Mar/14/2025 06:47:45 interface,info sfp28-11-olt2-data link up (speed 10G, full duplex)
Mar/14/2025 06:47:45 interface,info sfp28-12-olt1-data link up (speed 10G, full duplex)
Mar/14/2025 06:47:45 interface,info sfp28-1-to--10g link up (speed 10G, full duplex)
Mar/14/2025 06:47:45 interface,info sfp28-2-to--10g link up (speed 10G, full duplex)
Mar/14/2025 06:47:45 interface,info qsfp28-2-1--100g link up (speed 100G, full duplex)
Mar/14/2025 06:47:46 route,info L3HW Offloading ON
Mar/14/2025 06:47:53 route,info L3HW Offloading OFF
Mar/14/2025 06:47:54 interface,info qsfp28-2-1--100g link down
Mar/14/2025 06:47:54 interface,info sfp28-1-to--10g link down
Mar/14/2025 06:47:55 interface,info sfp28-2-to--10g link down
Mar/14/2025 06:47:55 interface,info sfp28-11-olt2-data link down
Mar/14/2025 06:47:56 interface,info sfp28-12-olt1-data link down
Mar/14/2025 06:47:57 interface,info sfp28-11-olt2-data link up (speed 10G, full duplex)
Mar/14/2025 06:47:57 interface,info sfp28-12-olt1-data link up (speed 10G, full duplex)
Mar/14/2025 06:47:57 interface,info sfp28-1-to--10g link up (speed 10G, full duplex)
Mar/14/2025 06:47:57 interface,info sfp28-2-to--10g link up (speed 10G, full duplex)
Mar/14/2025 06:47:58 route,info L3HW Offloading ON
And that will continue, till the router is rebooted. Note that there is a 100gigE link in there as well… All of these are fiber connections. the OLTs are local, and several of them are dark fiber.
what makes me wonder is having the same issue with multiple devices at the same site, because of that i point towards an environment issue
just in case, be aware, some sfp dac cables are copper if you have some kind of grounding or isolation problem between devices one dac can be an electrical bonding between devices, some time ago i saw a dac catch fire because a grounding problem
Hey, having these issues with 2116 since its realease.
port flapping → HW replacement and new box dies within a month.
have other 2x 2116s in DC running on 7.5 with 2 years uptime. One box had one of the LACP ports flapping for a month and uptime timer reset after 1 year uptime but all good otherwise.
out of 20x CCR2116 I have seen more deaths then with 1000x CCR 1009.
We are moving to CHR heavily.
We have had discussions with MikroTik, does these have queues on them? The latest is they found the issue and the next beta version should have it in there.
Well 7.19b6 should fix it, well at least according to MikroTIk. We will see. I moved the hotspot from a 2216 to the 2116 and now the 2216 does not have the issue, but the 2116 does. Just a FYI.