Community discussions

MikroTik App
 
LunaticRv
newbie
Topic Author
Posts: 40
Joined: Mon Dec 31, 2018 8:50 am

CCR2216 High CPU Usage even with 20 Gbps traffic (Fast Path, L3 HW Offloading Enabled)

Wed May 31, 2023 5:47 pm

Hello,

Today I upgraded my shiny 1072 ( v7.8 ) to 2216 ( v7.9.1 ) as my network starts to grow.

In terms of speed, nothing has been changed for now; 30 Gbps at peak times.

After migrating to 2216, I noticed high CPU even with 20 Gbps of traffic which is around 50-55% all the time while 1072 only had 5-7% CPU usage overall within the same configuration and traffic.

I have no Firewall/NAT rules at all and just have some VLANS through ports and static routes, thats all.

In terms of configuration everything is exact same with my CCR1072 but the CPU is really high I dont know why. Any advices?

Thanks in advance!
Last edited by LunaticRv on Thu Jun 01, 2023 6:07 pm, edited 1 time in total.
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 2989
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: CCR2216 High CPU Usage even with 20 Gbps traffic (Fast Path, L3 HW Offloading Enabled)

Wed May 31, 2023 6:32 pm

same configuration

My config;

In terms of configuration everything is exact same with my CCR1072 but the CPU is really high I dont know why. Any advices?

Thanks in advance!
I think you cannot use the same config

you must reconfigure using bridge VLAN filtering to be able to enjoy L2 and L3 Hardware Off-load, which is the way to take advantage of CCR2216

remember to build a single bridge

also dont include management ethernet1 interface in that bridge or any high speed task, must be used strictly for management purposes
 
LunaticRv
newbie
Topic Author
Posts: 40
Joined: Mon Dec 31, 2018 8:50 am

Re: CCR2216 High CPU Usage even with 20 Gbps traffic (Fast Path, L3 HW Offloading Enabled)

Wed May 31, 2023 7:33 pm

same configuration

My config;

In terms of configuration everything is exact same with my CCR1072 but the CPU is really high I dont know why. Any advices?

Thanks in advance!
I think you cannot use the same config

you must reconfigure using bridge VLAN filtering to be able to enjoy L2 and L3 Hardware Off-load, which is the way to take advantage of CCR2216

remember to build a single bridge

also dont include management ethernet1 interface in that bridge or any high speed task, must be used strictly for management purposes
Thank you very much for quick response.

But even without L3 Hardware Offloading, isn't the CPU still very high compared to 1072? 20 Gbps traffic only consumes 5-7% CPU of 1072 while it consumes 55-57% traffic of 2216 while Fast Path is active.
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 2989
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: CCR2216 High CPU Usage even with 20 Gbps traffic (Fast Path, L3 HW Offloading Enabled)

Wed May 31, 2023 8:19 pm

in ccr1072 average cpu usage It is the result of averaging the use of the 72 cores, which tends to deliver extremely low values even when you may have some cores with a lot of load.

in ccr2216 this average calculation is done only by 16 cores which leads us in most cases to a higher more realistic value

so that numbers have a lot of room for interpretation

i think on the paper on pure CPU power ccr2216 is inferior to ccr1072
but
is easier to take advantage of ccr2216 cpu resources because they are not spread across 72 cores of only 1ghz with in-order-execution architecture

for single core tasks one ccr2216 core can do almost the same work as 3 cores of the ccr1072 because have twice the clock rate and is out-of-order execution architecture
 
LunaticRv
newbie
Topic Author
Posts: 40
Joined: Mon Dec 31, 2018 8:50 am

Re: CCR2216 High CPU Usage even with 20 Gbps traffic (Fast Path, L3 HW Offloading Enabled)

Wed May 31, 2023 9:44 pm

in ccr1072 average cpu usage It is the result of averaging the use of the 72 cores, which tends to deliver extremely low values even when you may have some cores with a lot of load.

in ccr2216 this average calculation is done only by 16 cores which leads us in most cases to a higher more realistic value

so that numbers have a lot of room for interpretation

i think on the paper on pure CPU power ccr2216 is inferior to ccr1072
but
is easier to take advantage of ccr2216 cpu resources because they are not spread across 72 cores of only 1ghz with in-order-execution architecture

for single core tasks one ccr2216 core can do almost the same work as 3 cores of the ccr1072 because have twice the clock rate and is out-of-order execution architecture
I see, but still the CPU usage looked quite high to me compared to 1072, I checked 1072 CPUs one by one, no bottleneck even though some where at 50-60% but overall it was 12% when 2216 was at 60% on console and 71% on winbox cpu (i dont know why its also inconsistent:))

Also no idea what is "unclassified" but it also consumes reasonable amount of CPU

Image
 
User avatar
raimondsp
MikroTik Support
MikroTik Support
Posts: 267
Joined: Mon Apr 27, 2020 10:14 am

Re: CCR2216 High CPU Usage even with 20 Gbps traffic (Fast Path, L3 HW Offloading Enabled)  [SOLVED]

Thu Jun 01, 2023 8:16 am

For inter-VLAN routing, L3HW requires a hardware bridge for VLAN tagging. Please check out the following documentation topics:
 
LunaticRv
newbie
Topic Author
Posts: 40
Joined: Mon Dec 31, 2018 8:50 am

Re: CCR2216 High CPU Usage even with 20 Gbps traffic (Fast Path, L3 HW Offloading Enabled)

Thu Jun 01, 2023 6:06 pm

For inter-VLAN routing, L3HW requires a hardware bridge for VLAN tagging. Please check out the following documentation topics:

Thank you very much, I configured accordingly and voila CPU dropped to 0% at 17 Gbps traffic. I will be monitoring this evening when it pushes up to 50 Gbps.
 
User avatar
Maggiore81
Trainer
Trainer
Posts: 558
Joined: Sun Apr 15, 2012 12:10 pm
Location: Italy
Contact:

Re: CCR2216 High CPU Usage even with 20 Gbps traffic (Fast Path, L3 HW Offloading Enabled)

Sun Jun 04, 2023 11:50 am

can you explain wich part you revised?
can you post the relevant part of config, before and after, so can be usefult for the users?
 
LunaticRv
newbie
Topic Author
Posts: 40
Joined: Mon Dec 31, 2018 8:50 am

Re: CCR2216 High CPU Usage even with 20 Gbps traffic (Fast Path, L3 HW Offloading Enabled)

Mon Jun 05, 2023 1:47 pm

can you explain wich part you revised?
can you post the relevant part of config, before and after, so can be usefult for the users?
This was the part I was having issue;
https://help.mikrotik.com/docs/display/ ... portorbond

Looks like VLAN interface must be set on the bridge due to Layer 2 Dependency. I was setting them directly on a port or bonding interface. After moving them to bridge and enable filtering, it all started to work. Performance is amazing now, but many drawbacks such as not able to use traffic flow etc.
 
User avatar
PapsGrey
just joined
Posts: 22
Joined: Wed Dec 01, 2021 10:58 am
Location: Accra-Ghana
Contact:

Re: CCR2216 High CPU Usage even with 20 Gbps traffic (Fast Path, L3 HW Offloading Enabled)

Sat Jul 29, 2023 6:18 pm

For inter-VLAN routing, L3HW requires a hardware bridge for VLAN tagging. Please check out the following documentation topics:

Thank you very much, I configured accordingly and voila CPU dropped to 0% at 17 Gbps traffic. I will be monitoring this evening when it pushes up to 50 Gbps.




I have tried this and it works ok but the challenge I am facing is related to the VLAN interfaces on the single BRIDGE. I have set the MTU of the bridge and the vlan interfaces to 9216 . However, after a device reboot, the actual MTU of the vlan interface reverts to 1500, causing my OSPF to go into an "exstart" state. To resolve this, I have to mac-telnet into the router and disable/enable the interface, which then allows the actual MTU to return to 9216.

I would like to know if there is a way to make the MTU settings on the interface VLAN persist even after a reboot, to avoid this manual intervention each time.
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11381
Joined: Thu Mar 03, 2016 10:23 pm

Re: CCR2216 High CPU Usage even with 20 Gbps traffic (Fast Path, L3 HW Offloading Enabled)

Sat Jul 29, 2023 10:09 pm

MTU size is inherited ... bridge gets lowest MTU size of any of bridge port interfaces. Vlan interfaces get MTUs from their "trunk" interfaces (in this case from bridge). So verify what are MTU settings of all bridge ports. If they are all 9216 or higher, then you've encountered a bug and should contact support (preferably add supout file taken when MTU of bridge/vlan interfaces is not correct).
 
User avatar
PapsGrey
just joined
Posts: 22
Joined: Wed Dec 01, 2021 10:58 am
Location: Accra-Ghana
Contact:

Re: CCR2216 High CPU Usage even with 20 Gbps traffic (Fast Path, L3 HW Offloading Enabled)

Sun Jul 30, 2023 2:21 pm

MTU size is inherited ... bridge gets lowest MTU size of any of bridge port interfaces. Vlan interfaces get MTUs from their "trunk" interfaces (in this case from bridge). So verify what are MTU settings of all bridge ports. If they are all 9216 or higher, then you've encountered a bug and should contact support (preferably add supout file taken when MTU of bridge/vlan interfaces is not correct).

Thanks a lot @mkx. It will be a bug then and I have contacted support. However, I have used script/scheduler to automate bridge-restart anytime the device reboots and it works Ok for me now.
 
southAce
just joined
Posts: 1
Joined: Sat May 27, 2023 11:43 pm

Re: CCR2216 High CPU Usage even with 20 Gbps traffic (Fast Path, L3 HW Offloading Enabled)

Mon Sep 25, 2023 2:20 am

can you explain wich part you revised?
can you post the relevant part of config, before and after, so can be usefult for the users?
This was the part I was having issue;
https://help.mikrotik.com/docs/display/ ... portorbond

Looks like VLAN interface must be set on the bridge due to Layer 2 Dependency. I was setting them directly on a port or bonding interface. After moving them to bridge and enable filtering, it all started to work. Performance is amazing now, but many drawbacks such as not able to use traffic flow etc.

Hello All

I have tried all you mentioned, and I have seen a significant drop in CPU utilization. Before the single bridge, I was pushing 2gig and CPU was around 27%, but I am seeing 12% utilization on 14gig utilization. and about 2 individual CPU cores are about 80-90%. You mentioned you are pushing 17gig and CPU is 0%. I would like that a lot. Please assist.
I do mpls/ospf/bgp/vrfs. Thanks.
 
ernieball17
just joined
Posts: 16
Joined: Thu Jan 28, 2021 9:55 pm

Re: CCR2216 High CPU Usage even with 20 Gbps traffic (Fast Path, L3 HW Offloading Enabled)

Tue Oct 31, 2023 10:23 pm

For inter-VLAN routing, L3HW requires a hardware bridge for VLAN tagging. Please check out the following documentation topics:

Thank you very much, I configured accordingly and voila CPU dropped to 0% at 17 Gbps traffic. I will be monitoring this evening when it pushes up to 50 Gbps.
Hi! Can you please post your config before and after the changes. It's impossible to drop the cpu usage. I'm facing 27-30% with only 15 Gbps

Thanks!

Who is online

Users browsing this forum: Bing [Bot], GoogleOther [Bot] and 76 guests