CCR226 2S+ v6.27 all ports flap

maznu · May 27, 2015, 3:37pm

Hello!

My first post on the MT forums is a problem…

We have this switch, running 6.27 (downgraded from 6.28 which was buggy). For about one week now, randomly (every 8-24 hours), the switch takes all the ports down:

14:01:11 certificate,info CRL updated for cert_3
16:15:22 interface,info ether01-mmr link down
16:15:22 interface,info ether02-mmr link down
16:15:22 interface,info ether06-mmr-metronet link down
16:15:22 interface,info ether21-mmr-exa link down
16:15:22 interface,info ether23-wap1 link down

…and then up…

16:15:23 interface,info ether01-mmr link up (speed 1G, full duplex)
16:15:23 interface,info ether02-mmr link up (speed 1G, full duplex)
16:15:23 interface,info ether21-mmr-exa link up (speed 100M, full duplex)
16:15:23 interface,info ether23-wap1 link up (speed 100M, full duplex)

…and down again…

16:15:38 interface,info ether01-mmr link down
16:15:39 interface,info ether02-mmr link down
16:15:39 interface,info ether06-mmr-metronet link down
16:15:39 interface,info ether21-mmr-exa link down
16:15:39 interface,info ether23-wap1 link down

…and up again…

16:15:40 interface,info ether01-mmr link up (speed 1G, full duplex)
16:15:40 interface,info ether02-mmr link up (speed 1G, full duplex)
16:15:40 interface,info ether21-mmr-exa link up (speed 100M, full duplex)
16:15:40 interface,info ether23-wap1 link up (speed 100M, full duplex)

…and down again!

16:16:25 interface,info ether01-mmr link down
16:16:25 interface,info ether02-mmr link down
16:16:25 interface,info ether06-mmr-metronet link down
16:16:25 interface,info ether21-mmr-exa link down
16:16:25 interface,info ether23-wap1 link down

…and up again!!

16:16:26 interface,info ether01-mmr link up (speed 1G, full duplex)
16:16:26 interface,info ether02-mmr link up (speed 1G, full duplex)
16:16:26 interface,info ether21-mmr-exa link up (speed 100M, full duplex)
16:16:26 interface,info ether23-wap1 link up (speed 100M, full duplex)

As you can imagine, this causes problems for OSPF, BGP, and our customer connectivity in general.

Has anybody seen these problems? Is there a fault with the hardware? Should we downgrade to 6.26 or earlier?

Kind regards,

Marek

maznu · May 29, 2015, 9:47am

MikroTik support suggested we try upgrading to v6.29. 10 hours, no flaps. But it’s still too early to be sure - the port down/up/down/up we saw was random between 8-36 hours…!

maznu · June 2, 2015, 7:29am

No joy. Yesterday…

jun/01 03:30:50 interface,info ether02-mmr link down
jun/01 03:30:50 interface,info ether23-wap1 link down
jun/01 03:30:50 interface,info ether01-mmr link down
jun/01 03:30:50 interface,info ether06-mmr-metronet link down
jun/01 03:30:50 interface,info ether21-mmr-exa link down

jun/01 03:30:51 interface,info ether02-mmr link up (speed 1G, full duplex)
jun/01 03:30:51 interface,info ether23-wap1 link up (speed 100M, full duplex)
jun/01 03:30:51 interface,info ether01-mmr link up (speed 1G, full duplex)
jun/01 03:30:51 interface,info ether21-mmr-exa link up (speed 100M, full duplex)

jun/01 03:31:04 interface,info ether02-mmr link down
jun/01 03:31:04 interface,info ether23-wap1 link down
jun/01 03:31:04 interface,info ether01-mmr link down
jun/01 03:31:04 interface,info ether06-mmr-metronet link down
jun/01 03:31:04 interface,info ether21-mmr-exa link down

jun/01 03:31:05 interface,info ether02-mmr link up (speed 1G, full duplex)
jun/01 03:31:05 interface,info ether23-wap1 link up (speed 100M, full duplex)
jun/01 03:31:05 interface,info ether01-mmr link up (speed 1G, full duplex)
jun/01 03:31:05 interface,info ether21-mmr-exa link up (speed 100M, full duplex)
jun/01 03:31:06 interface,info ether06-mmr-metronet link up (speed 1G, full duplex)

jun/01 03:31:20 interface,info ether02-mmr link down
jun/01 03:31:20 interface,info ether23-wap1 link down
jun/01 03:31:20 interface,info ether01-mmr link down
jun/01 03:31:20 interface,info ether06-mmr-metronet link down
jun/01 03:31:20 interface,info ether21-mmr-exa link down

jun/01 03:31:21 interface,info ether02-mmr link up (speed 1G, full duplex)
jun/01 03:31:21 interface,info ether23-wap1 link up (speed 100M, full duplex)
jun/01 03:31:21 interface,info ether01-mmr link up (speed 1G, full duplex)
jun/01 03:31:21 interface,info ether21-mmr-exa link up (speed 100M, full duplex)
jun/01 03:31:22 interface,info ether06-mmr-metronet link up (speed 1G, full duplex)
jun/01 03:31:23 interface,info eoip-fs100-mgmt link down

And the same again at: jun/01 08:27:01 - jun/01 08:27:16

And the same again at: jun/01 10:23:39 - jun/01 10:24:10

And the same again at: jun/01 17:01:11 - jun/01 17:01:44

And the same again at: jun/01 23:13:24 - jun/01 23:14:30

And the same again at: jun/02 03:33:31 - jun/02 03:36:32

But not again at jun/02 08:27 - all fine just now.

So no, 6.29 has not fixed it.

chechito · June 2, 2015, 4:51pm

have you tryed 6.29.1??

maznu · June 2, 2015, 5:06pm

Not yet, no… it took three days since installing 6.29 for the first down/up flaps… guess we’ll have to give it a go, and hope to hear something back from MikroTik on ticket #2015052766000629

rni · June 5, 2015, 3:52pm

Having the same problem with 6.5 on a CRS125-24G-1S-2HnD

This installation is several months old and it started about two weeks ago, happens very infrequently and I’m still not able to determine the trigger. Every ethernet port seems to go down for a few seconds then back up.

Nothing unusual precedes in the logs.

maznu · June 8, 2015, 3:27pm

After 5 days, 17 hours, v6.29.1 has had the same problem: down up down up.

15:39:15 interface,info ether01-mmr link down
15:39:15 interface,info ether02-mmr link down
15:39:15 interface,info ether06-mmr-metronet link down
15:39:15 interface,info ether21-mmr-exa link down
15:39:15 interface,info ether23-wap1 link down

15:39:16 interface,info ether01-mmr link up (speed 1G, full duplex)
15:39:16 interface,info ether02-mmr link up (speed 1G, full duplex)
15:39:16 interface,info ether21-mmr-exa link up (speed 100M, full duplex)
15:39:16 interface,info ether23-wap1 link up (speed 100M, full duplex)

15:39:22 interface,info ether01-mmr link down
15:39:22 interface,info ether02-mmr link down
15:39:22 interface,info ether06-mmr-metronet link down
15:39:22 interface,info ether21-mmr-exa link down
15:39:23 interface,info ether23-wap1 link down

15:39:24 interface,info ether01-mmr link up (speed 1G, full duplex)
15:39:24 interface,info ether02-mmr link up (speed 1G, full duplex)
15:39:24 interface,info ether21-mmr-exa link up (speed 100M, full duplex)
15:39:24 interface,info ether23-wap1 link up (speed 100M, full duplex)

First MikroTik support said to try 6.29, then 6.29.1… not sure what’s next!

maznu · June 10, 2015, 4:03pm

…and today, after about six days of uptime, a kernel failure.

16:50:22 system,error,critical System rebooted because of kernel failure
16:50:22 system,error,critical router was rebooted without proper shutdown

maznu · June 22, 2015, 8:36pm

We replaced the CRS226. After five days: the same problems.

We have been told by MikroTik support to try 6.30rc22. We will install this tomorrow, and test for another six days.

105547111 · June 23, 2015, 12:15am

I missed this thread as Ive got a similar and I got to say its looking great so far 8 hours up on RC22

antoninn · June 26, 2015, 7:23am

What about current status of port flapping after few days? Is this problem solved by 6.30rc22?

105547111 · July 3, 2015, 2:02pm

Im running 6.30rc26 and no my CRS226 dropped all the ethers and SFP after 9 hours up time. I’ve even given support access and sent supouts.

The other thread I started is http://forum.mikrotik.com/t/crs226-24g-2s-rm-ether-drops-intermittent/88993/1

I bought the CRS226 through Amazon and I’ve got until 17th to return it, so I hope this get fixed.

I’ve tried every single RC since 6.29 as the bug drops all our downstream links out and we need it fixed.