CCR2216 Lockups

Hi All,

We’ve got 2x CCR2216 that are exhibiting similar symptoms of locking up under load. The setup is terminating approximately 350 VPLS sessions per CCR2216 carrying PPPoE traffic within these VPLS tunnels. The PPPoE traffic is handed off from the VPLS to our separate BNG. The BNG are connected to a different Edge CCR2216 so no BGP is being performed on the 2x that are locking up. The configuration is fairly simple with only the OSPF routing protocol in use.

It appears that only 4 of the 16 CPU cores are in use and spikes at peak periods could be causing a single core to reach 100% that monitoring is not capturing.

Does anyone have any advice? Does anyone know how to get Mikrotik’s attention to assist? We’ve raised a ticket and waited for a while already. I’d love any help we can get.

Thanks
Screenshot 2024-03-14 170411.png
Screenshot 2024-03-14 170459.png

I am not sure why it is only operating on 4 CPUs as I was under the understanding that most of the applications that were single threaded were made multithread in v7 of the ROS. I had lockup issues with some CCR1072 routers where they could be swamped before my DDoS protocols kicked in. In order for me to fix that, I had to turn off connection tracking and I moved some of my rules to the RAW filter. That cut my normal CPU load in half and I have not had any issues since with loss of access due to DDoS. I don’t know if that is possible to do when terminating VPLS tunnels.

Thanks for your reply.

We don’t have firewall rules on the routers since they are only handling private OSPF networks with no public IP. The Connections Table is also empty so it’s not tracking any connections. Might be possible there’s a bug in the switch chip or something that is occurring hence why nothing is coming through to the logs.

Show process, who load cpu. tool - profile - CPU: ALL and filter by Usage

The inability to distribute MPLS traffic between different cores is a known problem for both ARM and ARM64 architectures.

You could try using a vlan interface on the interface facing the MPLS cloud. I have verified that this configuration makes the CCR2004 use 2 CPU cores instead of 1.