Port flapping on CRS317-1G-16S+ with S+DA0003

Hardware MikroTik hardware general
1

We have next network hardware:

  • 1x main gateway CCR1009-7G-1C-1S+
  • 1x core 10G switch CRS317-1G-16S+
  • 5x managed switches CRS326-24G-2S+
  • 3x managed poe-switches CRS328-24P-4S+RM

CCR connected to CRS317 using S+DA0003.
CRS328 and CRS326 connected to CRS317 using S+DA0003.

On CCR and CRS used 3 software bridges + 3 VLANS (core + work + guest).
10G used as trunk.

All work fine, low load CPU, memory etc, last RouterOS 6.47.2, last RouterBOOT, but…periodically flapping ports (all to CRS328 / POE):

08:27:20 interface,info sfp-sfpplus7 link up (speed 10G, full duplex) 
08:27:51 interface,info sfp-sfpplus7 link down 
08:27:52 interface,info sfp-sfpplus7 link up (speed 10G, full duplex) 
08:31:14 interface,info sfp-sfpplus7 link down 
08:31:15 interface,info sfp-sfpplus7 link up (speed 10G, full duplex) 
08:40:44 interface,info sfp-sfpplus6 link down 
08:40:45 interface,info sfp-sfpplus6 link up (speed 10G, full duplex) 
08:43:11 interface,info sfp-sfpplus6 link down 
08:43:12 interface,info sfp-sfpplus6 link up (speed 10G, full duplex) 
08:46:02 interface,info sfp-sfpplus6 link down 
08:46:03 interface,info sfp-sfpplus6 link up (speed 10G, full duplex) 
08:47:19 interface,info sfp-sfpplus7 link down 
08:47:20 interface,info sfp-sfpplus7 link up (speed 10G, full duplex) 
08:49:27 interface,info sfp-sfpplus7 link down 
08:49:28 interface,info sfp-sfpplus7 link up (speed 10G, full duplex) 
08:51:21 interface,info sfp-sfpplus7 link down 
08:51:22 interface,info sfp-sfpplus7 link up (speed 10G, full duplex) 
08:52:22 interface,info sfp-sfpplus6 link down 
08:52:23 interface,info sfp-sfpplus6 link up (speed 10G, full duplex) 
08:52:55 interface,info sfp-sfpplus6 link down 
08:52:56 interface,info sfp-sfpplus6 link up (speed 10G, full duplex) 
08:53:40 interface,info sfp-sfpplus7 link down 
08:53:41 interface,info sfp-sfpplus7 link up (speed 10G, full duplex) 
08:54:30 interface,info sfp-sfpplus7 link down 
08:54:31 interface,info sfp-sfpplus7 link up (speed 10G, full duplex) 
08:55:44 interface,info sfp-sfpplus7 link down 
08:55:45 interface,info sfp-sfpplus7 link up (speed 10G, full duplex) 
08:58:19 interface,info sfp-sfpplus6 link down 
08:58:20 interface,info sfp-sfpplus6 link up (speed 10G, full duplex) 
09:02:51 interface,info sfp-sfpplus7 link down 
09:02:52 interface,info sfp-sfpplus7 link up (speed 10G, full duplex) 
09:03:41 interface,info sfp-sfpplus6 link down 
09:03:42 interface,info sfp-sfpplus6 link up (speed 10G, full duplex) 
09:06:13 interface,info sfp-sfpplus7 link down 
09:06:14 interface,info sfp-sfpplus7 link up (speed 10G, full duplex) 
09:16:29 interface,info sfp-sfpplus7 link down 
09:16:30 interface,info sfp-sfpplus7 link up (speed 10G, full duplex) 
09:19:41 interface,info sfp-sfpplus7 link down 
09:19:42 interface,info sfp-sfpplus7 link up (speed 10G, full duplex) 
09:19:48 interface,info sfp-sfpplus7 link down 
09:19:49 interface,info sfp-sfpplus7 link up (speed 10G, full duplex) 
09:20:04 interface,info sfp-sfpplus6 link down 
09:20:05 interface,info sfp-sfpplus6 link up (speed 10G, full duplex) 
09:21:43 interface,info sfp-sfpplus7 link down 
09:21:44 interface,info sfp-sfpplus7 link up (speed 10G, full duplex) 
09:22:30 interface,info sfp-sfpplus7 link down 
09:22:31 interface,info sfp-sfpplus7 link up (speed 10G, full duplex) 
09:23:14 interface,info sfp-sfpplus6 link down 
09:23:15 interface,info sfp-sfpplus6 link up (speed 10G, full duplex) 
09:26:08 interface,info sfp-sfpplus7 link down 
09:26:09 interface,info sfp-sfpplus7 link up (speed 10G, full duplex) 
09:35:19 interface,info sfp-sfpplus6 link down 
09:35:20 interface,info sfp-sfpplus6 link up (speed 10G, full duplex) 
09:36:34 interface,info sfp-sfpplus7 link down 
09:36:35 interface,info sfp-sfpplus7 link up (speed 10G, full duplex) 
09:38:11 interface,info sfp-sfpplus7 link down 
09:38:12 interface,info sfp-sfpplus7 link up (speed 10G, full duplex)

Any ideas?
photo_2020-09-01_10-03-51.jpg
photo_2020-09-01_10-03-51 (3).jpg
photo_2020-09-01_10-03-51 (2).jpg

2

What do the logs on the other side of sfp-sfpplus6 and 7 says? (on the POE-switches…)

Since it is only flapping against two of the three POE switches you have, I would first search the logs on the other side - then try to cross-switch the DAC-cables with the one working.

3

Same message on POE - link up / link down
No other errors

4

Raise a case with support. I (and lots of other users) had the same problem. I had a DAC cable between the two, but the problem occured for others with other SFP connections if I remember correctly). The problem occurs ONLY between the CRS317 and the CRS328, other connections on the CRS317 worked always flawless. I had to send in my CRS317 for repair. A second CRS317 that I borrowed from the shop had the same problem. Only the repair worked. The link has been absolutely stable since.
One warning: It may take Mikrotik a LONG time to do the repair. My repair took over 3 months
You may want to look at the forum for the old port flapping thread.

5

Thanks for reply
Can you provide link to thread?

Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+
6

http://forum.mikrotik.com/t/severe-port-flapping-on-crs328-24p-4s-and-crs317-1g-16s/125157/166
and have a look at post #94