Community discussions

MikroTik App
 
devicespl
just joined
Topic Author
Posts: 1
Joined: Sat Dec 10, 2022 11:39 am

CCR2116-12G-4S+ - Crashing on high load

Sun Dec 11, 2022 1:54 am

Hi!
I have bought recently two identical CCR2116. Installed as replacement for CCR2004 due its cpu power limits.
Both of them has been working for simple task as simple routing OSPF (no NAT),VRRP, DHCP servers over few VLANs and Simple Queueing by Radius.
They worked fine for a few days load balancing traffic for ~4k customers easily having ~30% CPU usage at rush hours.
Suddenly when I was changing settings of DHCP servers one of them have started to massively dropping packets around ~30-40%. Strangely dedicated managment port (as ether13) has the same problem with dropped packets so even reboot the router remotely became not possible. So I have pulled on of them down and .... the same issue started on the second router.

For few hours I have tried to figured out what's going on and I assume that one of the cause of the hang up seems to be correlated to average load ~ 8Gbit/s. When such traffic become larger CCR starting to be unstable, dropping the frames and after few seconds hangs. I've thought that CPU usage is the issue and when it started to dropping packets I have checked the usage but CPU is less than 40%. To my surprise in the moment when its not possible to log in - i can easily without any hiccup use console.

In both of CCR the issue was identical. I have tried to upgrade firmware to the latest betas to fix recent issues, i have disabled L3HW offload on both CCRs, I have disabled and removed interface bondings that I have used to load balance the traffic across switches. Nothing helped.
console_crash_ccr.txt
One of them have hanged out for good and crashed with kernel panic that I have captured from console. (added in attachment)

I decided to switch back to CCR2004 and those CCR2116 I have put on the workbench to test them on the desk.
I have tried for hours to reproduce the issue on my home lab by torturing both CCRs with a lot of BTESTs across the same configuration for another few days. And nothing happen - no hangs and other issues.
You do not have the required permissions to view the files attached to this post.
 
User avatar
pcunite
Forum Guru
Forum Guru
Posts: 1345
Joined: Sat May 25, 2013 5:13 am
Location: USA

Re: CCR2116-12G-4S+ - Crashing on high load

Thu Dec 29, 2022 4:24 pm

Disable Connection Tracking.
 
doush
Long time Member
Long time Member
Posts: 665
Joined: Thu Jun 04, 2009 3:11 pm

Re: CCR2116-12G-4S+ - Crashing on high load

Mon Jan 02, 2023 4:06 pm

Disable Connection Tracking.
Does the same CCR1072 lock up issue persists in the new CCR 2x series ?

Who is online

Users browsing this forum: No registered users and 13 guests