Page 1 of 1

Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Thu Nov 15, 2018 6:47 pm
by andersa
Hi

I am running 17 CRS328-24P-4S+ connected to 2 CRS317-1G-16S+ using this SFP S+32LC10D. The 328 boxes are running RouterOS 6.43.2 and the CRS317 boxes are running 6.43.4. I am experiencing that all the ports on the CRS317 are going up and down with anything between minutes to hours apart. It is almost no traffic on the network. I am totally in the dark here, do any of you have any idea what this could be?

Example from log:
17:02:12 interface,info sfp-sfpplus2 link down
17:04:07 interface,info sfp-sfpplus2 link up (speed 10G, full duplex)
17:09:04 interface,info sfp-sfpplus8 link down
17:09:05 interface,info sfp-sfpplus8 link up (speed 10G, full duplex)
17:18:00 interface,info sfp-sfpplus3 link down
17:18:01 interface,info sfp-sfpplus3 link up (speed 10G, full duplex)
17:24:13 interface,info sfp-sfpplus4 link down
17:24:15 interface,info sfp-sfpplus4 link up (speed 10G, full duplex)
17:25:04 interface,info sfp-sfpplus16 link down
17:25:05 interface,info sfp-sfpplus16 link up (speed 10G, full duplex)
17:34:00 interface,info sfp-sfpplus4 link down
17:34:01 interface,info sfp-sfpplus4 link up (speed 10G, full duplex)

Interface stats from one of the CRS328:
/interface ethernet print stats from=sfp-sfpplus2
name: sfp-sfpplus2
driver-rx-byte: 4 096 343 318
driver-rx-packet: 44 864 695
driver-tx-byte: 1 460 265 180
driver-tx-packet: 9 040 331
rx-bytes: 4 277 853 088
rx-too-short: 0
rx-too-long: 0
rx-unicast: 9 695 367
rx-broadcast: 31 908 257
rx-pause: 0
rx-multicast: 3 273 516
rx-error-events: 239
rx-fcs-error: 0
rx-fragment: 0
rx-overflow: 0
rx-jabber: 0
tx-bytes: 1 498 564 543
tx-unicast: 9 049 910
tx-broadcast: 231
tx-pause: 0
tx-multicast: 4 463
tx-underrun: 0
tx-collision: 0
tx-late-collision: 0
tx-rx-64: 31 287 843
tx-rx-65-127: 4 828 178
tx-rx-128-255: 17 182 394
tx-rx-256-511: 179 280
tx-rx-512-1023: 436 533
tx-rx-1024-1518: 17 622
tx-queue0-packet: 9 037 205
tx-queue1-packet: 0
tx-queue2-packet: 0
tx-queue3-packet: 0
tx-queue4-packet: 0
tx-queue5-packet: 0
tx-queue6-packet: 0
tx-queue7-packet: 0

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Dec 04, 2018 9:19 pm
by keefe007
Try 6.43.7 and make sure routerboot is also updated.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Dec 04, 2018 10:38 pm
by andersa
Sure, i dont see anything about the 317s in the changelog, but i'll try. Will give a new status in about 24h, will see if anything has changed.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Dec 05, 2018 9:30 am
by andersa
Try 6.43.7 and make sure routerboot is also updated.
Sad to say, but this made no difference.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Dec 05, 2018 4:30 pm
by keefe007
I am also seeing some link downs on CRS328 switches but the firmware update made them significantly less for me.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Dec 10, 2018 10:01 pm
by keefe007
Did you find a fix yet?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Dec 11, 2018 9:47 am
by andersa
Unfortunately not. I sendt them the support files at 22. Nov and did a follow up on 4. Des. Still no reply. Would be great if MT would have a look at this.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Dec 11, 2018 10:59 am
by mducharme
From the MikroTik site it looks to me like the S+32LC10D is specifically meant to have the S+23LC10D module on the other end. Are you using the same module on both ends?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Dec 11, 2018 11:21 am
by Elliot
I have the SAME EXACT problem. Mikrotik needs to fix this. No problem with 1Gbps SFP but with OWM SFP+ it's hell. I think this could be combination of software and hardware problem with SFP+ modules.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Dec 11, 2018 11:54 am
by andersa
I have the SAME EXACT problem. Mikrotik needs to fix this. No problem with 1Gbps SFP but with OWM SFP+ it's hell. I think this could be combination of software and hardware problem with SFP+ modules.
I agree with you assumtion that this is an edge case between the SFP units and the switch. But strangely it only occurs on one of our two CRS317 in the installation.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Dec 11, 2018 11:55 am
by andersa
From the MikroTik site it looks to me like the S+32LC10D is specifically meant to have the S+23LC10D module on the other end. Are you using the same module on both ends?
I am using the paired SFP+ from Mikrotik S+2332LC10D. So yes.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Dec 11, 2018 6:51 pm
by keefe007
What's the fix Mikrotik?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Sat Dec 22, 2018 6:42 pm
by cowgirl
Push - she problem on CRS328 here.... waiting for a fix

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Sat Dec 22, 2018 6:49 pm
by sff
We had this same problem appear on one of our 317s. I am not 100% sure when it started, but it was observed on 6.43.4. The upgrade to 6.43.7 fixed it, but ONLY after I did a full factory reset and reloaded the configuration by hand (it wasn't complex).

We also observed it for a brief time on several of the 328s. Switching from using SFP+1 to SFP+2 for the uplink fixed it.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Fri Dec 28, 2018 6:37 pm
by wotoni
We are experiencing the same thing. Running 6.42.7 with DAC Cable from CRS317 to CRS328-4C-20S-4S+RM.

I had to go to a SFP 1G on both units for the flapping to stop. I had tried SFP+ DAC and SFP+ single mode fiber with the same issue.

We have 8 CRS212 plugged into the CRS317 via DAC Cable with no issue. Only the 328 SFP+ ports appear to have the issue.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Feb 06, 2019 11:21 am
by AVA
Exactly the same problem here. Links between CRS317 and CRS328-4C-20S-4S+ are flapping. I have 3 pairs of 317 <-> 328 and all have the same problem. I tried different SFP/DAC. The latest firmware (6.43.8) does not help. No problem with 1GbE links....... :(

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Feb 06, 2019 1:05 pm
by andersa
I sendt MT a service dump in november - still no answer. Could i bother you guys that are experiencing the same problem with sending MT an email urging them to have a look at the issue - or at least give some feedback. I am experiencing total radio silence.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Feb 06, 2019 2:52 pm
by jprietove
Are any of you using port ether1 or just sfp-plus ports? If you are, remember that "The new Cloud Router Switch 317-1G-16S+RM is a rack-mountable manageable switch with Layer3 features, it has 16 SFP+ ports for high performance 10GbE connectivity and a 1GbE copper port for management. "

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Feb 06, 2019 3:04 pm
by andersa
Are any of you using port ether1 or just sfp-plus ports? If you are, remember that "The new Cloud Router Switch 317-1G-16S+RM is a rack-mountable manageable switch with Layer3 features, it has 16 SFP+ ports for high performance 10GbE connectivity and a 1GbE copper port for management. "
I can only speak for myself - and i am only using the SFP+ ports.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Feb 06, 2019 4:35 pm
by AVA
I think the problem is the CRS328. We use the CRS317 as Core with downlinks to different CA devices, some with 1GbE and some with 10GbE. The only affected links are those between CRS317 and CRS328 with 10GbE.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Thu Feb 07, 2019 4:29 pm
by sebg
exact same issue here, same symptoms, same workaround (force 1G link), it would be great if Mikrotik took this issue seriously...

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Thu Feb 07, 2019 8:20 pm
by Redmor
I ever had bad experiences with CRSs port flapping and I don't use them anymore, both ethernets and SFPs.
Sometimes on CCRs ports go totally down, disabling and enabling them solves the problem.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Fri Feb 08, 2019 9:57 am
by sebg
Sometimes on CCRs ports go totally down, disabling and enabling them solves the problem.
In production context ? :)

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Feb 18, 2019 5:08 pm
by AVA
I've wrote to Mikrotik Support ~2 weeks ago and also our Distributor said he would contact Mikrotik but till now no reaction. :cry:

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Feb 25, 2019 11:50 am
by AVA
Hi,

I got an answer from support. They could reproduce the flapping issue and are working on a solution.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Fri Mar 01, 2019 8:01 pm
by RFG
Same problem over here with CRS328-4C-20S-4S+RM on Sfp+ ports. I'm connected to a Dell X1052 with S+DA0003.
Let's wait for the correction.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Mar 05, 2019 5:48 am
by K0IBM
I will add my agreement to this thread. I have a CRS317-1G-16S+ with 4 CRS328-24P-4S+ and 3 CSS326-24G-2S+RM All are using the Maxxwave MW-SX+MM-US Modules. I have the same port flaping on all the CRS328 ports and the CRS317 reboots on a regular basis. The links to the CSS326's seem to be stable. I too am interested in this fix.

Steve Cearley
Steve@cearley.us
Wellshire Presbyterian Church.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Mar 05, 2019 12:56 pm
by inatec
hi,

same for us. We fighting with the same problem on our CRS317-1G-16S+ (viewtopic.php?f=2&t=146044). We thougth, it might be a configuration issue. I replaced cable, SFP+ modules (Allnet / Ubiquitiy) but nothing helped. Now I replaced the both ports (core-03 and core-04) with the 10G copper SFP ports and its seems now stable.
We replaced in the first time also one switched, which helped (core-lan-01). But now we have the same issue between core-lan-03 and core-lan-04.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Thu Mar 07, 2019 6:26 pm
by Stick1970
i mean the same problem
port flapping on CRS328-24P-4S+ and CRS317-1G-16S+
software version 6.44
I tried different SFP modules.
Are there any solutions to this problem?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Fri Mar 08, 2019 8:21 am
by coelliale
Me too..
317 - Ccr 10Gb mk Dac ok
317 - 326 10Gb mk S+85DLC03D ok
317 - 328 10Gb mk Dac flapping
The strange thing is that for over one month all worked fine, after that, without any change, the flapping began.
Forcing the Dac to 1Gbps stop the flapping immediatly, now we are monitoring it, but we need 10Gb connections.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Fri Mar 08, 2019 8:34 am
by roggles
hello folks,

we have multiple CRS317-1G-16S+ and CRS328-24P+4S+ with the same problem.
All running RouterOS and firmware 6.44.
They all connected togehter with Mikrotik DAC S+DA0001.
After switching to RouterOS and firmware to 6.45beta6 (Testing) don't solve the flapping port problem.

Last try....
Finally we switched on the CRS317-1G-16S+ to SwitchOS 2.9.

So the CRS328-24P+4S+ with RouterOS works with the CRS317-1G-16S+ with SwitchOS for 24h without problems...


So i think it's a problem on the CRS317-1G-16S+ with RouterOS.

regards

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Fri Mar 08, 2019 9:06 am
by inatec
hi,
Me too..
317 - Ccr 10Gb mk Dac ok
317 - 326 10Gb mk S+85DLC03D ok
317 - 328 10Gb mk Dac flapping
The strange thing is that for over one month all worked fine, after that, without any change, the flapping began.
Forcing the Dac to 1Gbps stop the flapping immediatly, now we are monitoring it, but we need 10Gb connections.
since, not all ports flapping, just one or two, I assume that there is a hardware failure. But we have to wait, until MikroTik have something so say.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Fri Mar 08, 2019 10:33 pm
by hapi
hmm, same problem

CRS317-1G-16S+ vs CRS326-24G-2S+ - DAC

today CRS317 stop traffic and in same time CRS326 stop working port connecting to CRS317.
MKv6.44

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Mar 11, 2019 10:08 am
by roggles
hello folks,

update:

After we switched on the CRS317-1G-16S+ to SwitchOS 2.9
5 days without port flapping. :)

regards

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Mar 11, 2019 10:13 am
by andersa
hello folks,

update:

After we switched on the CRS317-1G-16S+ to SwitchOS 2.9
5 days without port flapping. :)

regards
This is good news and indicative of software issues. Raising the probability of a fix. I have however met nothing but radio silence from MT lately, have any of you guys gotten any info from them?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Mar 25, 2019 8:17 pm
by keefe007
What's the stats of the fix?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Thu Mar 28, 2019 8:43 am
by coelliale
After we switched on the CRS317-1G-16S+ to SwitchOS 2.9
5 days without port flapping. :)
Hi roggles, still stable with SwitchOs?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Thu Mar 28, 2019 11:01 am
by roggles
yes, since there are no log entries

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Thu Mar 28, 2019 11:26 am
by hapi
CRS317 replaced with UBNT EdgeSwitch 16XG. Problem solved. I will not wait for the device to stop 1Gbit traffic.

I won't wait for it to ever be fixed. Similar to flapping on RB3011 and the like.



MIkrotik -1000.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Apr 03, 2019 10:51 am
by maxpower
Have the same problem with UF-SM-10G & CRS317-1G-16S+ with 6.42.12.
Any RoOS version with fix?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Apr 03, 2019 11:35 am
by Guntis
Have the same problem with UF-SM-10G & CRS317-1G-16S+ with 6.42.12.
Any RoOS version with fix?
Could you please send us email, about your setup and problem, to support@mikrotik.com?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Apr 03, 2019 11:43 am
by andersa
Have the same problem with UF-SM-10G & CRS317-1G-16S+ with 6.42.12.
Any RoOS version with fix?
Could you please send us email, about your setup and problem, to support@mikrotik.com?
Hi! How Is the fix coming along?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Apr 03, 2019 12:45 pm
by maxpower
Could you please send us email, about your setup and problem, to support@mikrotik.com?
Ticket#2019040322003846

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Apr 03, 2019 12:57 pm
by andersa
Could you please send us email, about your setup and problem, to support@mikrotik.com?
Ticket#2019040322003846
Where do i get additional information about tickets? I see no links in the emails i have gotten.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Apr 09, 2019 5:29 pm
by asadsayyad123
Severe port flapping on 10G links, works with auto-negotiation turned off and set to 1g.

CRS317-1G-16S+ <<>> CCR1009-8G-1S-1S+, Flapping at 10G, Works stable at 1G

CRS317-1G-16S+ <<>> CRS328-4C-20S-4S+, Flapping at 10G, Works stable at 1G

CRS317-1G-16S+ <<>> CRS212-1G-10S-1S+, Works Stable at 10G.

CRS328-4C-20S-4S+<<>> CRS328-4C-20S-4S+, Works Stable at 10G.




So, it all boils down to ROS BUG in CRS317.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Sun Apr 14, 2019 6:09 pm
by maxpower
Does anyone have port flapping problem with 10G link between CRS317-1G-16S+ <<>> CRS317-1G-16S+ ?
In our case it starts to happen after about a week after reboot and it happens on 2 of 3 ports. We tried to interchange modules used on stable link - but on other link it also flaps.

We have flapping on 1 & 5 ports, but no flapping on 9 port. Coincidence?

In SwitchOS you dont have log entries and the link is stable also? May be in SwOS it just does not log the short term failures?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Sun Apr 14, 2019 6:21 pm
by coelliale
Mikrotik support tell us that the problem is well known and that they don't have ETA for the resolution.
Their official workarround is to downgrade links to 1Gb or use switchos on 317
I hope that someone take care about this terrible bug!!

Alessandro

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Sun Apr 14, 2019 7:26 pm
by neutronlaser
file bug

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Apr 15, 2019 6:18 am
by asadsayyad123
Hello,

SwitchOS is one big pile of shit with no real featues, as it was intended for the general audience.

When i add SFP media converters or non-mikrotik or unmanaged switches on the other end it does not pick the link up.

But ROS does, hence we cannot use SWOS at all (As a last resort even if it a very crap OS .).

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Apr 15, 2019 9:13 am
by kosyot
Hello,

SwitchOS is one big pile of shit with no real featues, as it was intended for the general audience.

When i add SFP media converters or non-mikrotik or unmanaged switches on the other end it does not pick the link up.

But ROS does, hence we cannot use SWOS at all (As a last resort even if it a very crap OS .).
You are absolutely right about SwOS, but what can do with many introduced MiktoTik devices with only SwOS ?
It's not normal to produce a lot of hardware with no software capable to handle it...
I still believe, that MiktoTik will get attention on this and will produce real working OS.
btw I keep all my with SwOS 2.7

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Apr 22, 2019 1:42 pm
by maxpower
BTW, how do u guys can confirm that there no port flapping under SwOS? I didnt find any log section in SwOS web interface...

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Apr 22, 2019 2:12 pm
by coelliale
You have to check it on 328 side, looking for disconnections or rstp recalculation

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Apr 30, 2019 11:30 am
by maxpower
So seems like the problem is with CRS317-1G-16S+
We have one CRS317-1G-16S+ in center and three CRS317-1G-16S+ at remote ends, after 6 days from reboot we got port flapping - but not on all ports, we tried to change SFP+ modules, cables but no sense. After we changed from RoOS to SwOS on central CRS317-1G-16S+ log files of the remote CRS317-1G-16S+ (that are still running RoOS) are showing no port flapping after more than 8 days!

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Apr 30, 2019 7:30 pm
by keefe007
I have issues on the CRS328 connecting to a Siklu.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Thu May 02, 2019 8:14 pm
by coelliale
I can confirm that the problem appear also with 2 CRS317 connected trought mk DAC.
PLEASE MIKROTIK, GIVE US SUPPORT!!!!!!!!!

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed May 15, 2019 3:51 pm
by Zito
I confirm the problem with the CRS328 and CRS317 connection.
The problem is partially solved by the change of the CRS317 system to SwOS. But still Rx FCS Errors occur on the 10G interfaces.
[Ticket#2019051422003403]

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed May 15, 2019 4:06 pm
by sebg
Mikrotik, this issue has almost 6 months !!

It seems that Mikrotik acknowledges the problem, my reseller confirmed this point, you should be able to prioritize this issue as it affects core features : we're unable to deliver 10G connectivity !!

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Fri May 17, 2019 5:47 pm
by maxpower
The problem is partially solved by the change of the CRS317 system to SwOS. But still Rx FCS Errors occur on the 10G interfaces.
We can confirm the same, but not on all interfaces, those that were stable on RoOS have only 2 Errors.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Fri May 17, 2019 6:03 pm
by coelliale
Mikrotik support, please WAKE UP!!
Customers are angry and whants different brand hardware!

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed May 22, 2019 11:02 am
by Zito
*) crs3xx - improved switch-chip resource allocation on CRS326, CRS328, CRS305;
..but still problem not fixed.
1.PNG
2.PNG

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Fri May 24, 2019 8:02 pm
by maxpower
Can anybody try to add attenuators on problematic links? Would it change anything?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Sat May 25, 2019 6:53 pm
by llag
Can anybody try to add attenuators on problematic links? Would it change anything?
That seems unlikely as I have the same link flapping on my DAC links btween the CRS328 and CRS317 on 10G links. Something seems to be wrong (software or hardware??), but Mikrotik does not seem to be interested...

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Sun May 26, 2019 7:43 pm
by maxpower
Does the DAC link have Rx FCS Errors also?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon May 27, 2019 11:42 am
by Zito
Does the DAC link have Rx FCS Errors also?
Yes
This is the setup on my desk. It does nothing, only connections between switches.
3.PNG

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon May 27, 2019 6:05 pm
by nicoh
We are having this issue as well between two CRS317-1G-16S+ in our datacenter. We wrote an E-Mail to MikroTik Support on the 18th of May.

There has not been any reaction since*.

EDIT: *except for the automated reply message.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue May 28, 2019 4:16 pm
by gentrice
Our client has 200+ units installed in different locations, now even in 1G is not stable.
We have been pushed for months, mikrotik nothing to say.
Very bad service, client try to use Ubq to replace all CRS328.
I am thinking about it too, maybe we should close the distribution with Mikrotik.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed May 29, 2019 8:10 am
by coelliale
Very bad service, client try to use Ubq to replace all CRS328.
Hi gentrice, we are angry too, but the problem we are talking here is on crs317.. what kind of problem do you have on crs328?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed May 29, 2019 12:33 pm
by nicoh
There has not been any reaction since*.
We received an answer that MikroTik is aware of this issue and is working on a solution.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed May 29, 2019 12:39 pm
by andersa
Have anyone gotten any time estimates on when flapgate is beeing fixed?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed May 29, 2019 2:05 pm
by coelliale
new reply by mk support:
We are working on this problem, it could take up to a couple of weeks, to find a proper solution and test it.
we hope it...

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Jun 05, 2019 11:21 am
by nicoh
Hi gentrice, we are angry too, but the problem we are talking here is on crs317.. what kind of problem do you have on crs328?
We were able to reproduce this issue on CRS328-24P-4S+ connected via SFP+ DAC to a CRS317-1G-16S+. We tested the following scenarios:

.) change SFP+ port on CRS328 -> port keeps flapping
.) change SFP+ port on CRS317 -> port keeps flapping
.) take a freshly delivered CRS317 connect it to the CRS328 -> port keeps flapping
.) take different DAC -> port keeps flapping

Turning off Auto Negotiation and pinning it on 10Gbps (tick the radiobutton under Interfaces->Ethernet) on both CRS results in a stable 1Gbps link.

The setup firmwares and Package Versions:

CRS328-24P-4S+: 6.43.12 (stable), Firmware: 6.42.5 (Factory)
CRS317-1G-16S+: 6.44.3 (stable), Firmware: 6.44.3, Factory Firmware: 6.42.11

We added this information to the ticket we have opened.

EDIT: We updated the CRS328-24P-4S+ to 6.44.3 (Firmware & Packages) -> port keeps flapping

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Jun 05, 2019 4:36 pm
by SX1
In our scenario we needed to connect CCR1036-8G-2S+ (ROS 6.44.1) to CRS328-4C-20S-4S+ (SwitchOS 2.9) using Mikrotik SFP+ DAC 1m. We got 10G FDX connection, but it was always unstable. We had the following troubles:
  • 10G link flapping, sometimes, when it wants)
  • After a few hours of good working tagged traffic between CCR1036-8G-2S+ and CRS328-4C-20S-4S+ begun to drop. Connections begun to close one by one, until we reboot CRS. Than it works good again, but for a few hours only. After that we seen packet loss again.
  • If we tried to change any SFP+ settings like auto negotiation or set manual 1G speed, link goes down after applying and not became up. Link became up only if we reboot CRS or physically replug SFP+ DAC. And many other port settings in SwitchOS has the same bug - not working without reboot. DHCP client starts to work only after reboot too.
I think that's enough. Now we definitely completely refuse to work with mikrotik switch. It has too many unsolvable problems.
Bcause if we tell our customers that they need to wait a few weeks/months/years without Internet connection, until Mikrotik fix it, they will simply connect to another provider, and our business will go bankrupt.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Jun 10, 2019 3:12 am
by keefe007
What's the status on this Mikrotik?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Jun 10, 2019 4:42 am
by K0IBM
Any progress on this Mikrotik? It has been 6 months with no fix. Could we have an update and a plan to a fix?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Jun 10, 2019 9:49 am
by hapi
do not use DAC.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Jun 11, 2019 11:58 am
by sebg
do not use DAC.
this issue also happens with mikrotik SFP+ and fibers

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Jun 11, 2019 2:34 pm
by kosyot
I can confirm that
there is the same problem on :
CSS326-24G-2S+ with SwOS 2.9
Any SFP+ device connected to port EXCEPT connection to CRS317-1G-16S+ with SwOS 2.7 (with any SFP+ optic module and DAC working as required)


there in not problem at all on:
CRS317-1G-16S+ with SwOS 2.7 (we are unable to upgrade to 2.8 or 2.9 because switch stops to work after 10-15 mins, but you can find this on forum as well)
many switches with many different modules and vendors SFP+s, DACs, S+RJ10

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Jun 11, 2019 11:55 pm
by nicoh
Today we started to request the replacement of flapping CRS328, where we were able to systematically reproduce it, at our distributor.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Jun 12, 2019 10:20 am
by ilkogd
Hello. I need to buy two 24 port switches and I have to choose between CRS326 and second hand Cisco 2960G, the price is the same. If I take CRS326 it will be used with RouterOS for access switch. From what I read in forum, SwOS is not ready for "in production" setups and this is the reason to choose CRS326 instead of CSS326.

My question is - if I use only 100Mbps and 1000Mbps (random mix of ports, PCs, VoIP phones etc.) connections and not 10G connections, the problem with port flapping will be present or not?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Jun 12, 2019 10:31 am
by GuJack20
Hello. I need to buy two 24 port switches and I have to choose between CRS326 and second hand Cisco 2960G, the price is the same. If I take CRS326 it will be used with RouterOS for access switch. From what I read in forum, SwOS is not ready for "in production" setups and this is the reason to choose CRS326 instead of CSS326.

My question is - if I use only 100Mbps and 1000Mbps (random mix of ports, PCs, VoIP phones etc.) connections and not 10G connections, the problem with port flapping will be present or not?
According to all posters here, the problem presents itself only in connection with CRS317 and 10G SFP+. So it will not happen to your setup.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Jun 12, 2019 11:02 am
by ilkogd
According to all posters here, the problem presents itself only in connection with CRS317 and 10G SFP+. So it will not happen to your setup.
I think the same GuJack20. Do you have CRS326 in production? If you have, what is your opinion for the switch? Somebody else do you have CRS326 in production running RouterOS?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Jun 12, 2019 11:59 am
by nicoh
I think the same GuJack20. Do you have CRS326 in production? If you have, what is your opinion for the switch? Somebody else do you have CRS326 in production running RouterOS?
Hi,
please open or write in a thread/section where your question is addressed. This thread is about a severe port flapping issue on CRS328 and CRS317 as the title reflects.
Thanks,
nh

EDIT: corrected a typo of product names.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Jun 12, 2019 12:56 pm
by ilkogd
Hi,
please open or write in a thread/section where your question is addressed. This thread is about a severe port flapping issue on CRS328 and CRS317 as the title reflects.
Thanks,
nh

EDIT: corrected a typo of product names.
I'm sorry if this is mistake. I posted here because CRS328 is the same as CRS326 but with PoE. If moderators want they can delete my posts.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Jun 17, 2019 1:44 am
by Dekado
CRS328 to HP DL380 10gbit direct connection, port flapping exists.
so this it not mikrotik-to-core or mikrotik-to-mikrotik connection issue, something wrong in hardware design, I guess.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Jun 26, 2019 2:50 pm
by nmt1900
Same problem appeared suddenly after months of uptime. Two links between CRS326 and CRS317 (two of both devices) - one works perfectly and other link is flooding logs and causing packet loss. All troubleshooting attempts lead to conclusion, that CRS317 is the source of the problem - when we cross-connected the uplinks (leading to CRS326s), then port flapping stayed on same CRS317. Changing patches and GBIC's did nothing at all. Changing the uplink to other unused port on CRS317 did not help either. Links to other devices were fine on that same switch.

Upgrades or downgrades from 6.43.16 (same version on all devices) had no effect either.

One other detail could not be unnoticed - link downs had made their way up to tomorrow already :)

Screenshot (602).png

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Fri Jun 28, 2019 6:42 pm
by nmt1900
Now we did one test with the problematic link - we converted it to 10GbE with CAT7 and pair of RJ-10 modules. For more than hour of heavy load there was absolutely no link downs or any packet loss - so this is not likely related to topology issues etc. When we connected ports with fiber as before, all problems were back.

P. S. RJ-10 modules behaved differently on different hardware - everything was OK on CRS317, but module on CRS326 got quickly very hot (almost 90°C)...

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Jul 02, 2019 10:50 am
by roggles
Good Morning,

has anyone check this behaivour with the new version 6.45.1?

Regards

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Jul 02, 2019 12:38 pm
by nmt1900
It is bad, when things get broken, but it is not better, when they suddenly get "cured" by itself.

My problem just suddenly disappeared by itself. Link has been up from friday afternoon and all packet loss has also disappeared. Nobody has done anything to this link or the devices, nobody has even been in the server room, but port flapping just ended and now everything is fine for fourth day in the row. I have not been able to find any outside factors that could have caused this change...

It is possible to understand, why this kind of bugs can be hard to chase down - these kind of problems can take a very long time to reproduce...

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Thu Jul 04, 2019 9:51 am
by rmaDK
On the latest version 6.45.1, the problem is still exist.
The bundle ( CRS328-24P-4S+ and CRS317-1G-16S+ ) constantly (~ 5-7 minutes) flapping occurs

Mikrotik, you can finally pay attention to the problem !!!
Several tickets were created in the support, and not one received an answer on this issue.

Many people suffer from this problem for many months.
And there is still no official response.

If necessary, I can provide access to the equipment.
But start doing something already!

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Thu Jul 04, 2019 4:48 pm
by llag
I have the same combination: CRS328-24P-4S+ and CRS317-1G-16S+. Yesterday I installed 6.45.1. Since that time the number of port flaps increased dramatically (from once every few days to 70 during the last 24 hours!)
The CRS328 reports:
Rx Error Events 2335

The CRS317 reports:
Rx FCS Error 2389
Rx Error Events 2650

What is happening here? Was something changed in 6.45.1 that affects this?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Fri Jul 05, 2019 1:01 pm
by michaels
First, I love MikroTik products and RouterOS.

It was a big mistake to buy the combination (1x CRS317-1G-16S+ and 2x CRS328-4C-20S-4S+).
The expectation was that it should work fine, because it goes from the same manufacturer to the same manufacturer :)


10G to 10G Fiber SFP+ Tests with RouterOS 6.45.1

from CRS317-1G-16S+ to CRS328-4C-20S-4S+ flapping all day, about every 5min flapping
from CRS317-1G-16S+ to CCR1072-1G-8S+, works fine - no flapping
from CRS317-1G-16S+ to C4500X-32, works fine - no flapping

from CRS328-4C-20S-4S+ to CRS328-4C-20S-4S+, works fine - no flapping
from CRS328-4C-20S-4S+ to CCR1072-1G-8S+, works fine - no flapping
from CRS328-4C-20S-4S+ to C4500X-32, works fine -no flapping

Has anyone already found a solution that works with RouterOS.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Fri Jul 05, 2019 9:09 pm
by llag
It is funny to see that in your case it is the combination CRS317-1G-16S+ to CRS328-4C-20S-4S+ that is flapping too. I have some links from the CRS317 to server that work great. The same applies to other links on the CRS328. One would guess that Mikrotik would be able to track the problem easily as it occurs between these specific Mikrotik products?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Jul 08, 2019 10:28 pm
by nmt1900
It's might not be always like that. My problem lasted for weeks, then it suddenly disappeared and now emerged again after 10 days. Now it has been flapping again for a second day in the row. This kind of problems can take a lot of time if there's no ways to "accelerate" the process...

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Jul 09, 2019 2:11 pm
by becs
Our investigation has revealed that there might be a SFP+ compatibility issue with specific CRS317-1G-16S+ units when connected to other CRS3xx devices resulting in random link downs. Please, contact MikroTik support if you experience similar issue, we will verify it and arrange a repair if necessary.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Jul 09, 2019 3:40 pm
by drbunsen
Could you be more precise? Is there a way to identify the cause and what would be done during repair?
Think about the folks which are running these switches now without flapping ports, but might run into problems as soon as CRS3xx are installed.
We would like to know about the issue before the installation if possible.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Jul 09, 2019 4:01 pm
by roggles
Hello,

i can't belive that anyone else have a working connection between the CRS317-1G-16S+ and the CRS3xx devices.
We have three CRS317-1G-16S+ and over 10 CRS328-24P-4S+RM and no combination works without flapping.
All of them are updated to 6.45.1. ( firmware also! ) and we used original Mikrotik DAC Cables.

For Mikrotik it would be so easy to test and confirm this behaviour. Especially because it runs better with Switch OS!
I hoped for so long that Mikrotik fix this, i won't send this devices back, but slowly i have no more option.

Many people contact already the support and became no help since months...

so your (becs) answer is no help at all

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Jul 09, 2019 4:10 pm
by nmt1900
Our investigation has revealed that there might be a SFP+ compatibility issue with specific CRS317-1G-16S+ units when connected to other CRS3xx devices resulting in random link downs. Please, contact MikroTik support if you experience similar issue, we will verify it and arrange a repair if necessary.
Is there something hardware specific related to that - in a way that some devices might not be affected at all? I have one 317 with problem while other has been without problems...

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Jul 09, 2019 7:15 pm
by kosyot
Hahaha
Not well done MikroTik - hardware related problem?
And why when you switch to SwOS there is no such problem, only CRS3017 with RouterOS?
Same hardware - different behavior = simply software bug!

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Jul 10, 2019 11:28 pm
by llag
Our investigation has revealed that there might be a SFP+ compatibility issue with specific CRS317-1G-16S+ units when connected to other CRS3xx devices resulting in random link downs. Please, contact MikroTik support if you experience similar issue, we will verify it and arrange a repair if necessary.
The worst problems with port flapping (more that 70 per day) after the upgrade to 6.45.1 have been resolved after another reboot of the CRS317, but I still see port flaps (1 per day??). I guess that the issues I noticed where my UNMS monitoring the network often reports nodes not responding and returning after 1 -3 minutes and frequent drops of the internet radio streams are related too (no proof of that though).
I reported my case to support (Ticket#2019071022007154). Let's see what comes out of this

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Thu Jul 11, 2019 10:00 am
by michaels
CRS317 and CRS328 would be a great hardware combination and ideal for FTTX deployments.
I hope that the error can be corrected with a software update.

Both devices work great (in my case), if they are not connected to each other.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Jul 16, 2019 9:16 am
by rmaDK
Yesterday connected the new one CRS317-1G-16S+
On it, not one flap happened.
On old one also connected to same CRS328-24P-4S+ for this time 88 times.

And also noticed a strange thing.
Interface status, last link up / down shows a strange time.
The system time is correct, and in the status is wrong/strange time.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Jul 16, 2019 1:02 pm
by michaels
Yesterday connected the new one CRS317-1G-16S+
On it, not one flap happened.
So it could really be a hardware problem :(
And also noticed a strange thing.
Interface status, last link up / down shows a strange time.
time is not right for me either. strange bug...

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Jul 16, 2019 2:23 pm
by angriukas
I think
CRS326-24G-2S+RM
and
CRS125-24G-1S
contains same problem.
SFP+ port in 1G mode - no problems.
SFP+ port in 10G mode - problematic case even with Mikrotik DAC's.

Update:
problems raised when connecting two identical CRS326-24G-2S+RM devices with 10G DAC cable on SFP+ port.
Not all of devices was affected in my case. Totally installed 4 devices, two of them are problematic (from more early batch).

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Jul 16, 2019 5:23 pm
by llag
Our investigation has revealed that there might be a SFP+ compatibility issue with specific CRS317-1G-16S+ units when connected to other CRS3xx devices resulting in random link downs. Please, contact MikroTik support if you experience similar issue, we will verify it and arrange a repair if necessary.
@BECS: is there a way to identify if a CRS is affected? Does the serial numer tell if the device is affected?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Jul 29, 2019 2:34 pm
by pacman88
Is there any update on this?

can we buy mt hardware again or does this issue still persist?

BR

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Jul 29, 2019 2:42 pm
by coelliale
as MK support said: "if possible, please, consider using CRS309-1G-8S+ as replacement in similar applications to decrease probability of encountering this issue."

:shock: :shock: :shock: :shock: :shock: :shock: :shock:

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Jul 29, 2019 4:42 pm
by andersa
as MK support said: "if possible, please, consider using CRS309-1G-8S+ as replacement in similar applications to decrease probability of encountering this issue."

:shock: :shock: :shock: :shock: :shock: :shock: :shock:
If this is the case - it is simply not good enough. I have 30 of these devices in an installation, and -all- of them are experiencing this issue. It has almost been a year since i reported this to mikrotik support, and all i have got - despite several attempts at getting updated answers - is "we have recreated the issue".

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Jul 29, 2019 9:07 pm
by Elliot
To all posters here,
i'm afraid that this is a combination of HW and SW problem when there's a SFP+ connection between CRS 317 and CRS 328 and simply put, don't expect that anything's gonna change anytime soon.

We've replaced CRS 328 with much more expensive Cisco devices and it works like a charm.

So my advice would be:
If you can replace one of the devices, DO IT!
If your network won't fully utilize 10G just go 1G and figure replacement later.
If you can't or don't want to replace a lot devices for an non-Mikrotik equivalent sell CRS 317 and buy CRS 309 instead.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Jul 29, 2019 11:56 pm
by nicoh
I have to say that it seems that there are either

a) just a few CRS317 affected
or/and
b) most buyers not aware of this issue
or/and
c) the amount of CRS317 <- 10G SFP+ -> CRS328 combinations are low

It seems that MikroTik accepts the risk to repair/replace a few CRS317 and continues selling a device which does not provide 10G connections to CRS328. I would go with a) or b) when comparing the stock levels over time of online shops to the amount of posts and active people in this thread.

Telling customers not to buy a product in order to reduce the probability of failure is another way to tell them to look for alternatives as Elliot did.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Jul 30, 2019 12:30 am
by andersa
To all posters here,
i'm afraid that this is a combination of HW and SW problem when there's a SFP+ connection between CRS 317 and CRS 328 and simply put, don't expect that anything's gonna change anytime soon.

We've replaced CRS 328 with much more expensive Cisco devices and it works like a charm.

So my advice would be:
If you can replace one of the devices, DO IT!
If your network won't fully utilize 10G just go 1G and figure replacement later.
If you can't or don't want to replace a lot devices for an non-Mikrotik equivalent sell CRS 317 and buy CRS 309 instead.
I can live with GE a while longer, and i'm sure there will be a replacement program at some point. I just wish MT would communicate a little more. What switches did you go for? Changing the poe switches would be a nightmare for us, but changing the crs328 would be less painful. You say you chose cisco, did you consider other options? If so, what were they?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Jul 30, 2019 9:13 am
by kosyot
Let's add some advice:
  • use CRS317 with SwOS (no other ver than 3.7) instead of RouterOS

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Jul 31, 2019 9:50 am
by cmavced
I think we have a similar problem.
I'm testing the two just arrived Microtik CRS 317-1G-16+ switches.

I have my computer connected on ETH 10/100/1000 management/data port and a 100mbs ethernet link connected throught a S-RJ01 SFP (listed in MicroTik compatibility list https://wiki.mikrotik.com/wiki/MikroTik ... lity_table ).
The connection is not stable and the ports seems to flap.

The problem occours with SwitchOS but not with RouterOS.
I've also tried the connection using two S-RJ01 SFP, one for my computer and one for the 100mbs link, but the problem is the same.
No problem if I use a 1000mbs ethernet link, either using S-RJ01 SFP or directly attached to ETH 10/100/1000 management/data port.

I have also done a third test:
I've connected the 100mbs ethernet link to the eth 10/100/1000 management/data port of the first switch, than I connected the two CRS317 with a Cisco 10Gbs DAC SFP+, and I connected my computer to the eth 10/100/1000 management/data port of the second switch (avoiding the use of S-RJ01 SFP). The problem is the same.
Also in this case, no problem if I use a 1000mbs ethernet link.

It seems a problem with MicroTik Switch OS software.

I'm using 2.9 SwitchOS version and 6.45.2 RouterOS.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Jul 31, 2019 12:42 pm
by kosyot
Just use (mention above) SwOS 2.7 - 2.9 has a lot of bugs (see topic viewtopic.php?f=21&t=144031)
p.s. never use management port for traffic on switch

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Jul 31, 2019 8:53 pm
by Elliot
To all posters here,
i'm afraid that this is a combination of HW and SW problem when there's a SFP+ connection between CRS 317 and CRS 328 and simply put, don't expect that anything's gonna change anytime soon.

We've replaced CRS 328 with much more expensive Cisco devices and it works like a charm.

So my advice would be:
If you can replace one of the devices, DO IT!
If your network won't fully utilize 10G just go 1G and figure replacement later.
If you can't or don't want to replace a lot devices for an non-Mikrotik equivalent sell CRS 317 and buy CRS 309 instead.
I can live with GE a while longer, and i'm sure there will be a replacement program at some point. I just wish MT would communicate a little more. What switches did you go for? Changing the poe switches would be a nightmare for us, but changing the crs328 would be less painful. You say you chose cisco, did you consider other options? If so, what were they?
We have our Mikrotik CRS 317 connected to couple of Cisco SF350-48P (48 port POE switch) via 10G SFP+. And another couple of NETGEAR GS752TXP connected via SPF+ too. In both cases no port flapping.
I might add that we haven't thrown out CRS328 but we run only 1G on uplink due to port flapping when running port in 10G. It's pretty sad because otherwise it would be great value solution when you have one or two core CRS 317 switches connected to multiple site with CRS328.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Thu Aug 01, 2019 12:44 am
by llag
I have to say that it seems that there are either

a) just a few CRS317 affected
or/and
b) most buyers not aware of this issue
or/and
c) the amount of CRS317 <- 10G SFP+ -> CRS328 combinations are low

It seems that MikroTik accepts the risk to repair/replace a few CRS317 and continues selling a device which does not provide 10G connections to CRS328. I would go with a) or b) when comparing the stock levels over time of online shops to the amount of posts and active people in this thread.

Telling customers not to buy a product in order to reduce the probability of failure is another way to tell them to look for alternatives as Elliot did.
Well I reported my issues to support and now will have to return the CRS317 for RMA. I can only hope that that will solve my issues. It remains a mystery that this apparently happens only between the CRS317 and CRS328. It still sounds as if the issue is partly caused by the CRS328 as well, otherwise it should occur between the CRS317 and other switches and/or 10G network cards....

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Aug 06, 2019 11:30 am
by slarner
We are having massive issues with this. We have replaced the switch reset the config and its still shocking. Its 100% the CRS328's not the CRS317's

We have also changed the optics i dont know how many times tried DAC's its still awful.

see the picture this is less than 24hours up-time!

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Aug 06, 2019 5:19 pm
by nmt1900
Its 100% the CRS328's not the CRS317's
My experience tells otherwise - links between 317 and 326 are flapping, but when link from one 317 is connected to other 326, then flapping travels along with "sick" 317.

I don't know about 328-24P-4S+, but 326 seems to have similar hardware (counting out PoE and amount of sfp+ ports) hence problems might be of similar nature.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Fri Aug 09, 2019 12:33 am
by llag
Its 100% the CRS328's not the CRS317's
My experience tells otherwise - links between 317 and 326 are flapping, but when link from one 317 is connected to other 326, then flapping travels along with "sick" 317.

I don't know about 328-24P-4S+, but 326 seems to have similar hardware (counting out PoE and amount of sfp+ ports) hence problems might be of similar nature.
I raised a case with support and they told me to RMA the CRS317. We will see what will happen when I get the CRS317 (or a replacement) back....

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Fri Aug 09, 2019 2:16 am
by machack
Some can see the post, no port flapping, but other problems,

viewtopic.php?f=2&t=150971

Thanks!

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Aug 14, 2019 3:19 pm
by llag
Its 100% the CRS328's not the CRS317's
My experience tells otherwise - links between 317 and 326 are flapping, but when link from one 317 is connected to other 326, then flapping travels along with "sick" 317.

I don't know about 328-24P-4S+, but 326 seems to have similar hardware (counting out PoE and amount of sfp+ ports) hence problems might be of similar nature.
I raised a case with support and they told me to RMA the CRS317. We will see what will happen when I get the CRS317 (or a replacement) back....
I RMAd the device and the problem got acknowledged. The switch now has to be sent to Mikrotik for repair, which will take 6-8 weeks. That is a loooong time.
I was told that replacing it will not solve the issue as even new models have the problem.

Does this mean that Mikrotik knowingly continues to sell a faulty product?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Aug 14, 2019 3:26 pm
by nicoh
Does this mean that Mikrotik knowingly continues to sell a faulty product?
According to the findings within this thread and the information I received from our distributor: Yes.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Aug 14, 2019 4:00 pm
by sebg
Does this mean that Mikrotik knowingly continues to sell a faulty product?
According to the findings within this thread and the information I received from our distributor: Yes.
So almost one year after, we're at the same place....

We've bought high-end devices from Mikrotik, hoping for full 10G support : Mikrotik Support ignores the sheer volume of testing done in this thread, reaching the conclusion that crs317 are the core issue :-(

They should at least acknowledge the issue, recall crs317 from customers wishing to upgrade to CRS326-24S+2Q+RM for instance, and offer competitive pricing to compensate for the inconvenience of months without basic core fonctionality (10G support)

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Thu Aug 15, 2019 12:41 pm
by wanthalf
So, we just bought a faulty device too, I see! We got a second piece of CRS317-1G-16S+ and the interconnection between the two of them has exactly the problems discussed here - port flapping and packet loss. It seems to be much better when connecting the two devices on higher ports rather than on the lower ports, but still there is some small packet loss at 10Gb. Fortunately, we do not really need to have the two devices interconnected at a high speed at the moment (just for management), but this is a disappointment and if there was an alternative at the same price level, I would immediately return the device and buy something else. I do not like to work with unreliable hardware at all. Good to know we should be more careful when buying MikroTik next time. :-(

Fortunately, everything seems to work well on connections to the servers and even on the connection to the CCR1036-8G-2S+, which is the important factor at the moment.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Sun Aug 18, 2019 3:47 pm
by shanecaznet
Exact same problem here, but it is between any CRS317-1G-16S+. Have confirmed its present on ALL links between CRS317's, but not any link to any other device or switch.

Initially I thought it was a cable fault, but we've changed cables, SFPs, etc. Same result. Once I found this thread we've come to realise its yet another example of a product flaw with no real information from the vendor and no solution in sight.

Unfortunately for us, this was the final straw. We've just placed a order for a whole batch of Cisco kit and all of our Mikrotik gear will be coming out once it arrives. Now officially a dissapointed ex-Mikrotik user but looking forward to dealing with reliable products and actual engagement from the vendor when things aren't right with their product.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Aug 20, 2019 8:41 pm
by emunt6
Hi!

I just found this topic.

Did you use the Mikrotik switch through an UPS(uninterruptible power supply - pure sinus-wave type) hardware ?
If not, then this maybe a main problem - the power supply might have receiving some OVP/UVP/NOISE effects - causing the port flapping.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Aug 20, 2019 8:44 pm
by andersa
Hi!

I just found this topic.

Did you use the Mikrotik switch through an UPS(uninterruptible power supply - sinus wave type) hardware ?
If not, then this maybe a main problem - the power supply might have receiving some OVP/OCP/SCP effects - causing the port flapping.
In my case - yes, they were on a UPS.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Aug 20, 2019 8:47 pm
by andersa
Hi people! Looks like MT is finally stepping up. I got this email from them today.

We got the Supout.rif from CRS317 that you just sent, looks like we didn't receive couple of emails that you sent previously (sic). But yeah, from Supout.rif we can see that CRS317 - 846C083E29E4 is indeed faulty and needs an hardware repair.

We have become aware of a small number of CRS317-1G-16S+RM devices that may exhibit sporadic port flapping issues in certain usage scenarios. Since we are unable to pinpoint exact devices which could potentially exhibit this issue, we do not recommend replacing the device with an identical model, instead, we recommend using CRS309-1G-8S+IN ( 8x SFP+ ports) and CRS312-4C+8XG-RM ( 4x SFP+/10G Ethernet combo ports and 8x 10G Ethernet ports) as temporary replacements, until you receive repaired units through our RMA procedure. Please contact the seller to submit the device for warranty repair.

You can submit Supout.rif's from any other affected CRS317's that you have and we will take a look at them and check if they need to be sent to RMA as well.


Any of you guys gotten the same?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Aug 20, 2019 8:55 pm
by emunt6
Hi!

I just found this topic.

Did you use the Mikrotik switch through an UPS(uninterruptible power supply - sinus wave type) hardware ?
If not, then this maybe a main problem - the power supply might have receiving some OVP/OCP/SCP effects - causing the port flapping.
In my case - yes, they were on a UPS.
Hi!

I read in other topics/people, the power supply comes with the mikrotik switch - has not the best "quality".
I dont have any other idea what causing the port flapping.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Aug 20, 2019 9:32 pm
by kosyot
If the problem is on power supply there will be port flapping always, no matter what OS is running.
I have always port flapping when winning RouterOS (every single version on it) and never when running SwOS 2.7. We have a lot of 317 some connected to 326, 328 - different versions, all with SFP+ 10G running without single flap - all 317 running SwOS 2.7, other switches RouterOS or SwOS 2.9.
This is SOFTWARE problem, but I'm sure that MikroTik don't know how to resolve it - just see how many they need to produce new version on SwOS, because 2.9 is hell of the bugs.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Aug 20, 2019 10:12 pm
by andersa
Hi!

I just found this topic.

Did you use the Mikrotik switch through an UPS(uninterruptible power supply - sinus wave type) hardware ?
If not, then this maybe a main problem - the power supply might have receiving some OVP/OCP/SCP effects - causing the port flapping.
In my case - yes, they were on a UPS.
Hi!

I read in other topics/people, the power supply comes with the mikrotik switch - has not the best "quality".
I dont have any other idea what causing the port flapping.
Can you elaberate on what quality parameters are not up to scratch? Also, the CRS317 has internal PSUs. Furthermore - as per my last post, they have admitted hardware faults in some CRS317.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Aug 20, 2019 10:28 pm
by emunt6
If the problem is on power supply there will be port flapping always, no matter what OS is running.
I have always port flapping when winning RouterOS (every single version on it) and never when running SwOS 2.7. We have a lot of 317 some connected to 326, 328 - different versions, all with SFP+ 10G running without single flap - all 317 running SwOS 2.7, other switches RouterOS or SwOS 2.9.
This is SOFTWARE problem, but I'm sure that MikroTik don't know how to resolve it - just see how many they need to produce new version on SwOS, because 2.9 is hell of the bugs.
I heard a story, RouterBoard copper RJ45 ports had the same strange port flapping issue - they could not solve/trace the problem: software upgrade/config changes.
They tried as a last resort: After replacing the PSU the port flapping is gone.

I am not sure about that.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Aug 20, 2019 10:43 pm
by kosyot
It's very normal to have issues on copper port, because of potential difference generated by two sides of the cable. On twisted pair technology this must be no problem at all, but very often it is indeed.
On the other hand optic ports have no such problem at all and power related problems are just fiction.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Aug 20, 2019 10:53 pm
by emunt6
Hi!

I just found this topic.

Did you use the Mikrotik switch through an UPS(uninterruptible power supply - sinus wave type) hardware ?
If not, then this maybe a main problem - the power supply might have receiving some OVP/OCP/SCP effects - causing the port flapping.
In my case - yes, they were on a UPS.
Hi!

I read in other topics/people, the power supply comes with the mikrotik switch - has not the best "quality".
I dont have any other idea what causing the port flapping.
Can you elaberate on what quality parameters are not up to scratch? Also, the CRS317 has internal PSUs. Furthermore - as per my last post, they have admitted hardware faults in some CRS317.
I'm not an expert about that.

But you can compare:
https://viva-telecom.org/13121/mikrotik ... rm/review/
https://viva-telecom.org/13024/mikrotik ... rm/review/
https://www.youtube.com/watch?v=KNDZZ5bD-wM

CSR317 have some "custom build" PSU: 2x "OF-WT2402500" in one cage.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Aug 26, 2019 9:57 pm
by Adundee89
Since it sounds like its not an isolated issue (lol), I wont bother posting all my topology. However, We just deployed close to 10 317s, and are experiencing flapping on ONE of them. If that's useful to anyone. All 317 to 317 connections.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Aug 26, 2019 10:34 pm
by K0IBM
Hi I have a CRS317-1G-16S+ and have had or trouble ticket open with Mikrotik support for months. I am wondering if anyone has had success getting their unit repaired or replaced and if so was the problem resolved with the new/repaired unit. In my case it works with 1Gb links. 10 GB links flap attached to any CRS switch. I have on 10GB link that is stable to a security video server.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Aug 26, 2019 10:43 pm
by ochlocracy
I am wondering if anyone has had success getting their unit repaired or replaced and if so was the problem resolved with the new/repaired unit.
I have one RMA'd with my supplier. On week 2 of the 4-6 week turn around...

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Aug 28, 2019 6:38 pm
by K0IBM
Has anyone tried the new CRS326-24S+2Q+RM Cloud Router Switch? I am wondering if it has the same port issues when connection at 10GB to other CRS3xx switches as the CRS317.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Thu Aug 29, 2019 12:39 am
by llag
I am wondering if anyone has had success getting their unit repaired or replaced and if so was the problem resolved with the new/repaired unit.
I have one RMA'd with my supplier. On week 2 of the 4-6 week turn around...
My RMA (6 - 8 weeks) is only in the third week. I was told that a replacement would not make sense as new switches willhave the same problem.... So I am waiting for the repair while Mikrotik continues to selll defective equipment.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Thu Aug 29, 2019 8:43 am
by roggles
Hello,

we have the new CRS326-24S-2Q+RM here and they worked 2 days without problems now.
I have connected this device to an CRS328-24P-4S through Mikrotik DAC Cables and multiple Mikrotik S+R10 r2.

We have also sent three of our CRS317-1G-16S+ back for RMA.

regards

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Thu Aug 29, 2019 8:49 am
by macgaiver
Has anyone tried the new CRS326-24S+2Q+RM Cloud Router Switch? I am wondering if it has the same port issues when connection at 10GB to other CRS3xx switches as the CRS317.
All the sources indicate that is only that specific combination that this topic is affected, and maybe even then only some part of CRS317 affected.
I have several CRS3xx connected to CRS317 no issues (with exception of some doggy DAC, that got replaced).

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Thu Sep 05, 2019 2:19 pm
by pacman88
we got 4 new crs317 as replacement (they have been tested at our supplier) but they show the same behaviour....

the recommended crs309 and 312 do not work as a replacement for us since we need the port density in that specific usecase. Has anyone given the new MikroTik CRS326-24S+2Q+RM a try?
in an other place I would consider the MikroTik CRS312-4C+8XG-RM but I got cautious with MT in general after those issues. Has anybody seen this issue occur on any other of the mentioned devices or only on 317?

BR
Alex

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Sat Sep 07, 2019 9:55 am
by michaels
Has anyone given the new MikroTik CRS326-24S+2Q+RM a try?

Yes, got a CRS326-24S+2Q+RM yesterday (great device, many switch chip features).
Works great in conjunction with CRS328-4C-20S-4S+RM.
No more SFP+ 10G Port flapping (we have one CRS317-1G-16S+RM in RMA).

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Fri Sep 13, 2019 2:12 pm
by aidanpryde
Hi.
I also see problems with port flapping when using CRS317-1G-16S+RM <-> S+DA0001 <-> CRS326-24G-2S+RM.
CRS317-1G-16S+RM <-> SNR-SFP+LR-2 <-> CRS326-24G-2S+RM the problem is present.
All ports are switched to 1G now. But, on one port after turning on the RX / TX Flow Control option to auto, we observed RX Errors at first time. Yesterday i reset the error counter and so far no errors on this port with 10G. Keep watching
With CRS317-1G-16S+RM <-> S+DA0001 <-> RB4011iGS+RM everything is ok on 10G: no link downs, no rx errors.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Fri Oct 11, 2019 11:53 am
by nicoh
Hi,

we can confirm that CRS328-24P-4S+ in combination with CRS312-4C+8XG show the exact same problematic port flapping behavior (RouterOS & FW v6.45.6) as the CRS317-1G-16S+. The exact testing scenario is:

- 2x CRS312-4C+8XG (both new)
- 2x CRS328-24P-4S+ (1 new, 1 used but "working as expected" from our supplier)
- The CRS328s are connected with SFP+ DACs to the CRS312 SFP+ combi ports
- The test scenario ran 9 days in an air-conditioned environment

The outcome is attached to this post. Some observations:
- Both CRS328 show just a few link downs (<20 Link Downs), there is no behavioral difference between the new and the used CRS328
- The CRS312s show different results 50 vs. 1129 Link Downs to the same CRS328
- The screenshot was taken on the Oct. 10th, the Last Link Down Time information do contain wrong information.

We also had two new CRS326 connected (via SFP+) to the CRS328 and CRS312 in question which did not show any port flappings during the testing period. This test is a consequence of the recommendations done by MikroTik to use the CRS312 and CRS326 as replacement for the flapping CRS317. In our tests we see that at least the CRS312 have the exact same issue as the CRS317s. There is still no evidence that the CRS328 is not part of the problem.

Conclusion:

- CRS312-4C+8XG combined with CRS328-24P-4S+ do have flapping SFP+ connections
- We did not receive any feedback from Mikrotik regarding this issue. Therefore, we will stop our tests of Mikrotiks problematic network devices.

br,
nicoh

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Oct 14, 2019 6:22 pm
by cbrown
I can confirm this as well.
Hi,

we can confirm that CRS328-24P-4S+ in combination with CRS312-4C+8XG show the exact same problematic port flapping behavior (RouterOS & FW v6.45.6) as the CRS317-1G-16S+. The exact testing scenario is:

- 2x CRS312-4C+8XG (both new)
- 2x CRS328-24P-4S+ (1 new, 1 used but "working as expected" from our supplier)
- The CRS328s are connected with SFP+ DACs to the CRS312 SFP+ combi ports
- The test scenario ran 9 days in an air-conditioned environment

The outcome is attached to this post. Some observations:
- Both CRS328 show just a few link downs (<20 Link Downs), there is no behavioral difference between the new and the used CRS328
- The CRS312s show different results 50 vs. 1129 Link Downs to the same CRS328
- The screenshot was taken on the Oct. 10th, the Last Link Down Time information do contain wrong information.

We also had two new CRS326 connected (via SFP+) to the CRS328 and CRS312 in question which did not show any port flappings during the testing period. This test is a consequence of the recommendations done by MikroTik to use the CRS312 and CRS326 as replacement for the flapping CRS317. In our tests we see that at least the CRS312 have the exact same issue as the CRS317s. There is still no evidence that the CRS328 is not part of the problem.

Conclusion:

- CRS312-4C+8XG combined with CRS328-24P-4S+ do have flapping SFP+ connections
- We did not receive any feedback from Mikrotik regarding this issue. Therefore, we will stop our tests of Mikrotiks problematic network devices.

br,
nicoh

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Oct 15, 2019 5:32 pm
by andersa
Anyone tested the CRS326-24S+2Q+RM yet?

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Oct 21, 2019 11:10 pm
by coelliale
I changed 2 317 with 2 326-24+
The link with all 328 and between 326-24+ are now fine @10Gbps
One link with 326-24G still flap and needed to be set @1Gpbs :-(
Better then before, but seems to be not fully resolved.
Alessandro

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Mon Oct 21, 2019 11:20 pm
by ochlocracy
Received my CRS-317 back from RMA. The problem was NOT fixed, still produces FCS Errors immediately when configuring the link at 10G with a CRS-328.

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Tue Oct 22, 2019 2:12 am
by sotirone
I am having the same problem with the CSS326-24G-2S+ on the SFP+ ports.

I am connecting a workstation with an Intel X520 10Gb card and Fiberstore Cisco SFP-10G-SR SFP+ modules.

My Linux dmesg is full of this:

ixgbe 0000:42:00.1 enp66s0f1: NIC Link is Down
ixgbe 0000:42:00.1 enp66s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX

Re: Severe port flapping on CRS328-24P-4S+ and CRS317-1G-16S+

Posted: Wed Oct 23, 2019 12:17 pm
by roggles
Hello,

we have received our CRS317-1G-16S+ back from rma on Oct, 10th since then no port flapping.
We have also running the new CRS326-24S+2Q+ since Aug, 23 and there are no port flapping.

regards