Recently deployed ccr2216 for bgp, but i found cpu usage is high ~80% at peak hour with with traffic around 20Gbps.
Im only running bgp and some vlans, and I have 40+ bgp peers. Is there any master can give me some advises please.
Check for BGP affinity setting and I hope you have a minimal firewalling rules that might eat your CPU, paste your config here and sanitized it so that other people might be able to help you
Hi loloski, below is my bgp router config:
/interface ethernet
set [ find default-name=qsfp28-1-1 ] l2mtu=9570 mtu=9570
set [ find default-name=qsfp28-1-2 ] l2mtu=9570 mtu=9570
set [ find default-name=qsfp28-1-3 ] l2mtu=9570 mtu=9570
set [ find default-name=qsfp28-1-4 ] l2mtu=9570 mtu=9570
set [ find default-name=qsfp28-2-1 ] l2mtu=9570 mtu=9570
set [ find default-name=qsfp28-2-2 ] l2mtu=9570 mtu=9570
set [ find default-name=qsfp28-2-3 ] l2mtu=9570 mtu=9570
set [ find default-name=qsfp28-2-4 ] l2mtu=9570 mtu=9570
set [ find default-name=sfp28-1 ] l2mtu=9570 mtu=9570
set [ find default-name=sfp28-2 ] l2mtu=9570 mtu=9570
set [ find default-name=sfp28-3 ] l2mtu=9570 mtu=9570
set [ find default-name=sfp28-4 ] l2mtu=9570 mtu=9570
set [ find default-name=sfp28-5 ] l2mtu=9570 mtu=9570
set [ find default-name=sfp28-6 ] l2mtu=9570 mtu=9570
set [ find default-name=sfp28-7 ] l2mtu=9570 mtu=9570
set [ find default-name=sfp28-8 ] l2mtu=9570 mtu=9570
set [ find default-name=sfp28-9 ] l2mtu=9570 mtu=9570
set [ find default-name=sfp28-10 ] l2mtu=9570 mtu=9570
set [ find default-name=sfp28-11 ] l2mtu=9570 mtu=9570
set [ find default-name=sfp28-12 ] l2mtu=9570 mtu=9570
/interface vlan
add interface=qsfp28-1-1 vlan-id=176
add interface=qsfp28-1-1 vlan-id=175
add interface=qsfp28-1-1 vlan-id=251
add interface=qsfp28-1-1 vlan-id=290
add interface=qsfp28-1-1 vlan-id=280
add interface=qsfp28-1-1 vlan-id=172
add interface=qsfp28-1-1 vlan-id=250
add interface=qsfp28-1-1 vlan-id=11
add interface=qsfp28-1-1 vlan-id=177
add interface=qsfp28-1-1 vlan-id=200
add interface=qsfp28-1-1 vlan-id=174
add interface=qsfp28-1-1 vlan-id=173
add interface=qsfp28-1-1 vlan-id=12
/ip firewall filter
add action=drop chain=input dst-port=8999 protocol=tcp src-address-list=!WhiteList
/ip firewall nat
add action=dst-nat chain=dstnat dst-port=19811 protocol=udp to-addresses=192.168.11.2 to-ports=161
/ip firewall service-port
set ftp disabled=yes
/ip service
set ftp disabled=yes
set ssh disabled=yes
set telnet disabled=yes
set www disabled=yes
set api disabled=yes
set api-ssl disabled=yes
set winbox port=8999
/ipv6 address
/routing bgp connection
add afi=ip as=123456 disabled=no input.filter=HEv4-in local.address=xx.xx.xx.xx .role=ebgp name=HurricaneElectric_v4 output.filter-chain=HEv4-out .network=MY_IPv4 remote.address=xx.xx.xx.xxx/32 .as=6939 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=BigBandv4-in local.address=xx.xx.xx.xx .role=ebgp name=BigBand_v4 output.filter-chain=BigBandv4-out .network=MY_IPv4 remote.address=xx.xx.xx.xx/32 .as=132111 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=MyIXv4-in local.address=218.100.44.xx .role=ebgp name=HuaweiCloudGlobal1_v4 output.filter-chain=MyIXv4-out .network=MY_IPv4 remote.address=218.100.44.82/32 .as=136907 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=MyIXv4-in local.address=218.100.44.xx .role=ebgp name=HuaweiCloudGlobal2_v4 output.filter-chain=MyIXv4-out .network=MY_IPv4 remote.address=218.100.44.83/32 .as=136907 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=MyIXv4-in local.address=218.100.44.xx .role=ebgp name=Google1_v4 output.filter-chain=MyIXv4-out .network=MY_IPv4 remote.address=218.100.44.92/32 .as=15169 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=MyIXv4-in local.address=218.100.44.xx .role=ebgp name=TIME1_v4 output.filter-chain=MyIXv4-out .network=MY_IPv4 remote.address=218.100.44.112/32 .as=9930 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=MyIXv4-in local.address=218.100.44.xx .role=ebgp name=MyIX1_v4 output.filter-chain=MyIXv4-out .network=MY_IPv4 remote.address=218.100.44.138/32 .as=55822 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=MyIXv4-in local.address=218.100.44.xx .role=ebgp name=MyIX2_v4 output.filter-chain=MyIXv4-out .network=MY_IPv4 remote.address=218.100.44.139/32 .as=55822 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=MyIXv4-in local.address=218.100.44.xx .role=ebgp name=Netflix_v4 output.filter-chain=MyIXv4-out .network=MY_IPv4 remote.address=218.100.44.143/32 .as=2906 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=MyIXv4-in local.address=218.100.44.xx .role=ebgp name=Meta1_v4 output.filter-chain=MyIXv4-out .network=MY_IPv4 remote.address=218.100.44.146/32 .as=32934 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=MyIXv4-in local.address=218.100.44.xx .role=ebgp name=Microsoft1_v4 output.filter-chain=MyIXv4-out .network=MY_IPv4 remote.address=218.100.44.154/32 .as=8075 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=MyIXv4-in local.address=218.100.44.xx .role=ebgp name=Amazon1_v4 output.filter-chain=MyIXv4-out .network=MY_IPv4 remote.address=218.100.44.155/32 .as=16509 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=MyIXv4-in local.address=218.100.44.xx .role=ebgp name=Google2_v4 output.filter-chain=MyIXv4-out .network=MY_IPv4 remote.address=218.100.44.158/32 .as=15169 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=MyIXv4-in local.address=218.100.44.xx .role=ebgp name=TM1_v4 output.filter-chain=MyIXv4-out .network=MY_IPv4 remote.address=218.100.44.182/32 .as=4788 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=MyIXv4-in local.address=218.100.44.xx .role=ebgp name=Cloudflare1_v4 output.filter-chain=MyIXv4-out .network=MY_IPv4 remote.address=218.100.44.185/32 .as=13335 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=MyIXv4-in local.address=218.100.44.xx .role=ebgp name=TIME2_v4 output.filter-chain=MyIXv4-out .network=MY_IPv4 remote.address=218.100.44.195/32 .as=9930 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=MyIXv4-in local.address=218.100.44.xx .role=ebgp name=TencentGlobal1_v4 output.filter-chain=MyIXv4-out .network=MY_IPv4 remote.address=218.100.44.201/32 .as=132203 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=MyIXv4-in local.address=218.100.44.xx .role=ebgp name=TencentGlobal2_v4 output.filter-chain=MyIXv4-out .network=MY_IPv4 remote.address=218.100.44.202/32 .as=132203 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=MyIXv4-in local.address=218.100.44.xx .role=ebgp name=TencentCDN1_v4 output.filter-chain=MyIXv4-out .network=MY_IPv4 remote.address=218.100.44.203/32 .as=139341 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=MyIXv4-in local.address=218.100.44.xx .role=ebgp name=Meta2_v4 output.filter-chain=MyIXv4-out .network=MY_IPv4 remote.address=218.100.44.207/32 .as=32934 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=MyIXv4-in local.address=218.100.44.xx .role=ebgp name=TencentCDN2_v4 output.filter-chain=MyIXv4-out .network=MY_IPv4 remote.address=218.100.44.208/32 .as=139341 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=MyIXv4-in local.address=218.100.44.xx .role=ebgp name=Microsoft2_v4 output.filter-chain=MyIXv4-out .network=MY_IPv4 remote.address=218.100.44.214/32 .as=8075 router-id=xx.xx.xx.xx routing-table=main
add afi=ip as=123456 disabled=no input.filter=MyIXv4-in local.address=218.100.44.xx .role=ebgp name=Amazon2_v4 output.filter-chain=MyIXv4-out .network=MY_IPv4 remote.address=218.100.44.219/32 .as=16509 router-id=xx.xx.xx.xx routing-table=main
add afi=ipv6 as=123456 disabled=no input.filter=HEv6-in local.address=2001:470:1:e24::2 .role=ebgp name=HurricaneElectric_v6 output.filter-chain=HEv6-out .network=MY_IPv6 remote.address=2001:470:1:e24::1/128 .as=6939 router-id=xx.xx.xx.xx
add afi=ipv6 as=123456 disabled=no input.filter=HEv6-in local.address=2001:df0:272::1:6:2 .role=ebgp name=BigBand_v6 output.filter-chain=HEv6-out .network=MY_IPv6 remote.address=2001:df0:272::1:6:1/128 .as=132111 router-id=xx.xx.xx.xx routing-table=main
add afi=ipv6 as=123456 disabled=no input.filter=MyIXv6-in local.address=2001:de8:10::xx .role=ebgp name=TIME1_v6 output.filter-chain=MyIXv6-out .network=MY_IPv6 remote.address=2001:de8:10::2/128 .as=9930 router-id=xx.xx.xx.xx
add afi=ipv6 as=123456 disabled=no input.filter=MyIXv6-in local.address=2001:de8:10::xx .role=ebgp name=Google1_v6 output.filter-chain=MyIXv6-out .network=MY_IPv6 remote.address=2001:de8:10::1a/128 .as=15169 router-id=xx.xx.xx.xx routing-table=main
add afi=ipv6 as=123456 disabled=no input.filter=MyIXv6-in local.address=2001:de8:10::xx .role=ebgp name=MyIX1_v6 output.filter-chain=MyIXv6-out .network=MY_IPv6 remote.address=2001:de8:10::3d/128 .as=55822 router-id=xx.xx.xx.xx routing-table=main
add afi=ipv6 as=123456 disabled=no input.filter=MyIXv6-in local.address=2001:de8:10::xx .role=ebgp name=Meta1_v6 output.filter-chain=MyIXv6-out .network=MY_IPv6 remote.address=2001:de8:10::46/128 .as=32934 router-id=xx.xx.xx.xx routing-table=main
add afi=ipv6 as=123456 disabled=no input.filter=MyIXv6-in local.address=2001:de8:10::xx .role=ebgp name=Microsoft1_v6 output.filter-chain=MyIXv6-out .network=MY_IPv6 remote.address=2001:de8:10::54/128 .as=8075 router-id=xx.xx.xx.xx routing-table=main
add afi=ipv6 as=123456 disabled=no input.filter=MyIXv6-in local.address=2001:de8:10::xx .role=ebgp name=Amazon2_v6 output.filter-chain=MyIXv6-out .network=MY_IPv6 remote.address=2001:de8:10::55/128 .as=16509 router-id=xx.xx.xx.xx routing-table=main
add afi=ipv6 as=123456 disabled=no input.filter=MyIXv6-in local.address=2001:de8:10::xx .role=ebgp name=TM1_v6 output.filter-chain=MyIXv6-out .network=MY_IPv6 remote.address=2001:de8:10::83/128 .as=4788 router-id=xx.xx.xx.xx routing-table=main
add afi=ipv6 as=123456 disabled=no input.filter=MyIXv6-in local.address=2001:de8:10::xx .role=ebgp name=Cloudflare1_v6 output.filter-chain=MyIXv6-out .network=MY_IPv6 remote.address=2001:de8:10::86/128 .as=13335 router-id=xx.xx.xx.xx routing-table=main
add afi=ipv6 as=123456 disabled=no input.filter=MyIXv6-in local.address=2001:de8:10::xx .role=ebgp name=TIME2_v6 output.filter-chain=MyIXv6-out .network=MY_IPv6 remote.address=2001:de8:10::97/128 .as=9930 router-id=xx.xx.xx.xx routing-table=main
add afi=ipv6 as=123456 disabled=no input.filter=MyIXv6-in local.address=2001:de8:10::xx .role=ebgp name=Meta2_v6 output.filter-chain=MyIXv6-out .network=MY_IPv6 remote.address=2001:de8:10::9f/128 .as=32934 router-id=xx.xx.xx.xx routing-table=main
add afi=ipv6 as=123456 disabled=no input.filter=MyIXv6-in local.address=2001:de8:10::xx .role=ebgp name=Microsoft2_v6 output.filter-chain=MyIXv6-out .network=MY_IPv6 remote.address=2001:de8:10::a9/128 .as=8075 router-id=xx.xx.xx.xx routing-table=main
add afi=ipv6 as=123456 disabled=no input.filter=MyIXv6-in local.address=2001:de8:10::xx .role=ebgp name=Amazon1_v6 output.filter-chain=MyIXv6-out .network=MY_IPv6 remote.address=2001:de8:10::b4/128 .as=16509 router-id=xx.xx.xx.xx routing-table=main
add afi=ipv6 as=123456 disabled=no input.filter=MyIXv6-in local.address=2001:de8:10::xx .role=ebgp name=Google2_v6 output.filter-chain=MyIXv6-out .network=MY_IPv6 remote.address=2001:de8:10::df/128 .as=15169 router-id=xx.xx.xx.xx routing-table=main
add afi=ipv6 as=123456 disabled=no input.filter=MyIXv6-in local.address=2001:de8:10::xx .role=ebgp name=HuaweiCloudGlobal1_v6 output.filter-chain=MyIXv6-out .network=MY_IPv6 remote.address=2001:de8:10::ea/128 .as=136907 router-id=xx.xx.xx.xx routing-table=main
add afi=ipv6 as=123456 disabled=no input.filter=MyIXv6-in local.address=2001:de8:10::xx .role=ebgp name=HuaweiCloudGlobal2_v6 output.filter-chain=MyIXv6-out .network=MY_IPv6 remote.address=2001:de8:10::eb/128 .as=136907 router-id=xx.xx.xx.xx routing-table=main
add afi=ipv6 as=123456 disabled=no input.filter=MyIXv6-in local.address=2001:de8:10::xx .role=ebgp name=TencentCDN1_v6 output.filter-chain=MyIXv6-out .network=MY_IPv6 remote.address=2001:de8:10::ec/128 .as=139341 router-id=xx.xx.xx.xx routing-table=main
add afi=ipv6 as=123456 disabled=no input.filter=MyIXv6-in local.address=2001:de8:10::xx .role=ebgp name=TencentGlobal1_v6 output.filter-chain=MyIXv6-out .network=MY_IPv6 remote.address=2001:de8:10::ee/128 .as=132203 router-id=xx.xx.xx.xx routing-table=main
add afi=ipv6 as=123456 disabled=no input.filter=MyIXv6-in local.address=2001:de8:10::xx .role=ebgp name=TencentGlobal2_v6 output.filter-chain=MyIXv6-out .network=MY_IPv6 remote.address=2001:de8:10::ef/128 .as=132203 router-id=xx.xx.xx.xx routing-table=main
add afi=ipv6 as=123456 disabled=no input.filter=MyIXv6-in local.address=2001:de8:10::xx .role=ebgp name=Netflix_v6 output.filter-chain=MyIXv6-out .network=MY_IPv6 remote.address=2001:de8:10::f5/128 .as=2906 router-id=xx.xx.xx.xx routing-table=main
add afi=ipv6 as=123456 disabled=no input.filter=MyIXv6-in local.address=2001:de8:10::xx .role=ebgp name=TencentCDN2_v6 output.filter-chain=MyIXv6-out .network=MY_IPv6 remote.address=2001:de8:10::10c/128 .as=139341 router-id=xx.xx.xx.xx routing-table=main
Please surround block of data with the < / > preformatted text tag when you post code.
Hi, done edit my code.
Appreciate if you could give me some idea, thank you!
Have you watched this? https://www.youtube.com/watch?v=py4up-lO8zY
Mayby tweaking affinities could help?
I only had a quick skim through the config, and the pic with my iPhone so take this with a grain of salt, but this looks more like forwarding/dataplane CPU than BGP itself.
The main hint is that networking is the biggest CPU user. At ~20 Gbps, if L3HW offload is not actually active for that traffic path, the CCR2216 will push a lot of packets through CPU.
I’d first check whether L3HW is really enabled/active and if firewall/NAT or some unsupported feature is kicking the traffic back to CPU. BGP can burn CPU during updates, but steady high networking CPU usually smells like software forwarding.
Just wanted to say that your cpu usage stats don't seem off at all, and the performance you're seeing is basically "what it says on the tin", so other than things like bgp affinity, which might speed up initial loading, your config is probably not bad.
Where the 2216 is really strong is L3HW. That requires setting it up.
If you're doing bgp, you'll probably want "pure/proper" L3HW where the actual routes are pushed to hardware. (The other one, mentioned as L3HW Fasttrack pushes only individual connections/flows. Don't confuse the two.)
Offload works fine either on bare interfaces or on vlans of a vlan filtered bridge, or a combination thereof. I'd suggest s single vlan filtered bridge with all the interfaced you intend to use for routing, but that's just me.