Community discussions

MikroTik App
 
txfz
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 57
Joined: Tue Mar 10, 2020 9:02 am

CRS354 high CPU usage

Tue Mar 01, 2022 11:20 am

Hi,

I have a CRS354 on 6.47.9 that's constantly running at near 100% CPU usage. It doesn't do any routing except for management access, and as far as I can tell, only WinBox traffic is going on. I've not noticed any issues with switching performance, and it rarely gets more than a few GB/s. WinBox is slow as a result, and just exporting the configuration took like a minute or two.

Image
# mar/01/2022 09:52:57 by RouterOS 6.47.9
# software id = QCBG-JR1Q
#
# model = CRS354-48G-4S+2Q+
# serial number = B8450B7DC3C1
/interface ethernet set [ find default-name=ether19 ] disabled=yes
/interface ethernet set [ find default-name=ether20 ] disabled=yes
/interface ethernet set [ find default-name=ether21 ] disabled=yes
/interface ethernet set [ find default-name=ether22 ] disabled=yes
/interface ethernet set [ find default-name=ether23 ] disabled=yes
/interface ethernet set [ find default-name=ether24 ] disabled=yes
/interface ethernet set [ find default-name=ether30 ] disabled=yes
/interface ethernet set [ find default-name=ether31 ] disabled=yes
/interface ethernet set [ find default-name=ether35 ] comment=bonding1
/interface ethernet set [ find default-name=ether36 ] comment=bonding1
/interface ethernet set [ find default-name=ether46 ] disabled=yes
/interface ethernet set [ find default-name=ether47 ] comment=bonding2
/interface ethernet set [ find default-name=ether48 ] comment=bonding2
/interface ethernet set [ find default-name=ether49 ] comment=Management
/interface ethernet set [ find default-name=qsfpplus1-1 ] auto-negotiation=no comment="bond qsfp"
/interface ethernet set [ find default-name=qsfpplus1-2 ] comment="bond qsfp"
/interface ethernet set [ find default-name=qsfpplus1-3 ] comment="bond qsfp"
/interface ethernet set [ find default-name=qsfpplus1-4 ] comment="bond qsfp"
/interface ethernet set [ find default-name=qsfpplus2-1 ] auto-negotiation=no comment="bond qsfp"
/interface ethernet set [ find default-name=qsfpplus2-2 ] comment="bond qsfp"
/interface ethernet set [ find default-name=qsfpplus2-3 ] comment="bond qsfp"
/interface ethernet set [ find default-name=qsfpplus2-4 ] comment="bond qsfp"
/interface ethernet set [ find default-name=sfp-sfpplus4 ] auto-negotiation=no speed=10Gbps
/interface bridge add admin-mac=C4:AD:34:B6:31:1F auto-mac=no name=bridge vlan-filtering=yes
/interface vlan add interface=bridge name=management vlan-id=5
/interface bonding add mode=802.3ad name=bonding1 slaves=ether35,ether36
/interface bonding add mode=802.3ad name=bonding2 slaves=ether47,ether48
/interface bonding add mode=802.3ad name=bonding3 slaves=ether43,ether44
/interface bonding add mode=802.3ad name=qsfp slaves=qsfpplus1-1,qsfpplus1-2,qsfpplus1-3,qsfpplus1-4,qsfpplus2-1,qsfpplus2-2,qsfpplus2-3,qsfpplus2-4
/interface list add name=management-list
/user group add name=local policy=local,reboot,read,write,policy,test,winbox,password,sniff,sensitive,!telnet,!ssh,!ftp,!web,!api,!romon,!dude,!tikapp
/interface bridge msti add bridge=bridge disabled=yes identifier=1 vlan-mapping=5,12,19
/interface bridge msti add bridge=bridge disabled=yes identifier=2 vlan-mapping=11,13
/interface bridge msti add bridge=bridge disabled=yes identifier=3 vlan-mapping=20,30
/interface bridge port add bridge=bridge interface=ether1
/interface bridge port add bridge=bridge interface=ether2
/interface bridge port add bridge=bridge interface=ether3 pvid=5
/interface bridge port add bridge=bridge interface=ether4
/interface bridge port add bridge=bridge interface=ether5
/interface bridge port add bridge=bridge interface=ether6 pvid=5
/interface bridge port add bridge=bridge interface=ether7
/interface bridge port add bridge=bridge interface=ether8
/interface bridge port add bridge=bridge interface=ether9 pvid=5
/interface bridge port add bridge=bridge interface=ether10
/interface bridge port add bridge=bridge interface=ether11
/interface bridge port add bridge=bridge interface=ether12 pvid=5
/interface bridge port add bridge=bridge interface=ether13
/interface bridge port add bridge=bridge interface=ether14
/interface bridge port add bridge=bridge interface=ether15 pvid=5
/interface bridge port add bridge=bridge interface=ether16
/interface bridge port add bridge=bridge interface=ether17
/interface bridge port add bridge=bridge interface=ether18 pvid=5
/interface bridge port add bridge=bridge interface=ether19
/interface bridge port add bridge=bridge interface=ether20
/interface bridge port add bridge=bridge interface=ether21
/interface bridge port add bridge=bridge interface=ether22
/interface bridge port add bridge=bridge interface=ether23
/interface bridge port add bridge=bridge interface=ether24
/interface bridge port add bridge=bridge interface=ether25
/interface bridge port add bridge=bridge interface=ether26
/interface bridge port add bridge=bridge interface=ether27 pvid=30
/interface bridge port add bridge=bridge interface=ether28 pvid=30
/interface bridge port add bridge=bridge interface=ether29 pvid=5
/interface bridge port add bridge=bridge interface=ether30
/interface bridge port add bridge=bridge interface=ether31
/interface bridge port add bridge=bridge interface=ether32 pvid=11
/interface bridge port add bridge=bridge interface=ether33
/interface bridge port add bridge=bridge interface=ether34
/interface bridge port add bridge=bridge disabled=yes interface=ether35
/interface bridge port add bridge=bridge disabled=yes interface=ether36
/interface bridge port add bridge=bridge interface=ether37
/interface bridge port add bridge=bridge interface=ether38 pvid=13
/interface bridge port add bridge=bridge interface=ether39 pvid=11
/interface bridge port add bridge=bridge interface=ether40 pvid=11
/interface bridge port add bridge=bridge interface=ether41 pvid=5
/interface bridge port add bridge=bridge interface=ether42 pvid=912
/interface bridge port add bridge=bridge disabled=yes interface=ether43
/interface bridge port add bridge=bridge disabled=yes interface=ether44
/interface bridge port add bridge=bridge interface=ether45 pvid=921
/interface bridge port add bridge=bridge interface=ether46
/interface bridge port add bridge=bridge disabled=yes interface=ether47
/interface bridge port add bridge=bridge disabled=yes interface=ether48
/interface bridge port add bridge=bridge interface=sfp-sfpplus1
/interface bridge port add bridge=bridge interface=sfp-sfpplus2
/interface bridge port add bridge=bridge interface=sfp-sfpplus3
/interface bridge port add bridge=bridge interface=sfp-sfpplus4
/interface bridge port add bridge=bridge interface=qsfp
/interface bridge port add bridge=bridge interface=bonding1 pvid=5
/interface bridge port add bridge=bridge interface=bonding2
/interface bridge port add bridge=bridge interface=bonding3 pvid=11
/interface bridge port add bridge=bridge interface=ether49 pvid=5
/ip neighbor discovery-settings set discover-interface-list=management-list
/interface bridge vlan add bridge=bridge comment=Management tagged=bridge,bonding2,sfp-sfpplus3,sfp-sfpplus4,qsfp,ether1,ether2,ether4,ether5,ether7,ether8,ether10,ether11,ether13,ether14,ether16,ether17,ether25,ether26,ether31 vlan-ids=5
/interface bridge vlan add bridge=bridge comment=VM tagged=qsfp,bonding2,sfp-sfpplus3,sfp-sfpplus4,ether1,ether2,ether4,ether5,ether7,ether8,ether10,ether11,ether13,ether14,ether16,ether17 vlan-ids=13
/interface bridge vlan add bridge=bridge comment=VMotion tagged=qsfp,sfp-sfpplus3 vlan-ids=30
/interface bridge vlan add bridge=bridge comment=VM tagged=qsfp,bonding2,sfp-sfpplus3,sfp-sfpplus4,ether1,ether2,ether4,ether5,ether7,ether8,ether10,ether11,ether13,ether14,ether16,ether17,ether33,ether34 vlan-ids=11
/interface bridge vlan add bridge=bridge tagged=qsfp vlan-ids=912
/interface bridge vlan add bridge=bridge comment=vSAN tagged=qsfp vlan-ids=20
/interface bridge vlan add bridge=bridge comment=VM tagged=qsfp,bonding2,sfp-sfpplus3,sfp-sfpplus4,ether1,ether2,ether4,ether5,ether7,ether8,ether10,ether11,ether13,ether14,ether16,ether17 vlan-ids=12,19
/interface bridge vlan add bridge=bridge tagged=qsfp vlan-ids=921
/interface list member add interface=management list=management-list
/interface list member add interface=ether49 list=management-list
/ip address add address=10.7.5.5/24 interface=management network=10.7.5.0
/ip dns set servers=10.7.5.1
/ip firewall service-port set ftp disabled=yes
/ip firewall service-port set tftp disabled=yes
/ip firewall service-port set irc disabled=yes
/ip firewall service-port set h323 disabled=yes
/ip firewall service-port set sip disabled=yes
/ip firewall service-port set pptp disabled=yes
/ip firewall service-port set udplite disabled=yes
/ip firewall service-port set dccp disabled=yes
/ip firewall service-port set sctp disabled=yes
/ip route add distance=1 gateway=10.7.5.1
/ip service set telnet disabled=yes
/ip service set ftp disabled=yes
/ip service set www disabled=yes
/ip service set api disabled=yes
/ip service set api-ssl disabled=yes
/ip ssh set strong-crypto=yes
/system clock set time-zone-name=Europe/Stockholm
/system identity set name=THN-DC-SW1
/system routerboard settings set auto-upgrade=yes boot-os=router-os
#error exporting /system swos
/tool bandwidth-server set enabled=no
/tool mac-server set allowed-interface-list=none
/tool mac-server mac-winbox set allowed-interface-list=management-list
/tool mac-server ping set enabled=no
/tool netwatch add disabled=yes down-script="/system reboot" host=10.7.5.1 interval=3m
/tool romon set enabled=yes
You do not have the required permissions to view the files attached to this post.
 
txfz
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 57
Joined: Tue Mar 10, 2020 9:02 am

Re: CRS354 high CPU usage

Mon May 02, 2022 10:35 am

Issue remains. I've not been able to verify, but now I'm starting to suspect it is in fact affecting switching throughput. It is very much a production device and cannot be taken down for maintenance on a whim.

Anyone got any ideas?
 
Kindis
Member
Member
Posts: 434
Joined: Tue Nov 01, 2011 6:54 pm
Location: Sweden

Re: CRS354 high CPU usage

Mon May 02, 2022 11:03 am

Well my guess is there that somehow hardware offload is not active on a port somehow.

If you run the following:
/interface bridge port print
Do you then get an H marked before all ports? My guess is that some port is not and for this reason traffic hits the CPU.
Bonding and LACP should provide hardware offload but take a look and see if any port is not active in regards to hardware offload.
 
txfz
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 57
Joined: Tue Mar 10, 2020 9:02 am

Re: CRS354 high CPU usage

Mon May 02, 2022 12:41 pm

Thanks. All enabled, active bridge ports seem to be hardware offloaded.

Who is online

Users browsing this forum: ameliask, mikronoob89, normis, Vojta and 100 guests