Page 1 of 1

CPU Load

Posted: Sat Feb 12, 2011 6:15 pm
by Bishop
Hardware: Core i7-920 (4 core) Hypertrading off
Try RouterOS 5.0rc7
About 800 PPPoE clients with shapeing

Cpu load (very strange balancing across multiple cores)
cpuload.JPG
rapidly increasing number of drop packets for transmission
drops.JPG
how to correct the situation with the loading cores?

Re: CPU Load

Posted: Sat Feb 12, 2011 10:56 pm
by martini
/tool profile

Re: CPU Load

Posted: Sun Feb 13, 2011 7:49 pm
by Bishop
/tool profile
after manual alignment interrupt per cores the situation has stabilized a bit
server.JPG
there is a suspicion that the case in network card on the Intel 82576 chip

what has caused this behavior to me is completely incomprehensible.

My IRQ table
/system resource pci> print 
 # DEVICE   VENDOR                                         NAME                                        IRQ       
 0 02:00.1  nVidia Corporation                             High Definition Audio Controller (rev: 161) 5         
 1 02:00.0  nVidia Corporation                             unknown device (rev: 162)                   10        
 2 01:00.1  Intel Corporation                              82576 Gigabit Network Connection (rev: 1)   5         
 3 01:00.0  Intel Corporation                              82576 Gigabit Network Connection (rev: 1)   10        
 4 00:1f.3  Intel Corporation                              82801JI (ICH10 Family) SMBus Controller ... 11        
 5 00:1f.0  Intel Corporation                              82801JIR (ICH10R) LPC Interface Controll... 0         
 6 00:1e.0  Intel Corporation                              82801 PCI Bridge (rev: 144)                 0         
 7 00:1d.7  Intel Corporation                              82801JI (ICH10 Family) USB2 EHCI Control... 3         
 8 00:1d.2  Intel Corporation                              82801JI (ICH10 Family) USB UHCI Controll... 11        
 9 00:1d.1  Intel Corporation                              82801JI (ICH10 Family) USB UHCI Controll... 15        
10 00:1d.0  Intel Corporation                              82801JI (ICH10 Family) USB UHCI Controll... 3         
11 00:1a.7  Intel Corporation                              82801JI (ICH10 Family) USB2 EHCI Control... 11        
12 00:1a.2  Intel Corporation                              82801JI (ICH10 Family) USB UHCI Controll... 15        
13 00:1a.1  Intel Corporation                              82801JI (ICH10 Family) USB UHCI Controll... 14        
14 00:1a.0  Intel Corporation                              82801JI (ICH10 Family) USB UHCI Controll... 10        
15 00:14.3  Intel Corporation                              5520/5500/X58 I/O Hub Throttle Registers... 0         
16 00:14.2  Intel Corporation                              5520/5500/X58 I/O Hub Control Status and... 0         
17 00:14.1  Intel Corporation                              5520/5500/X58 I/O Hub GPIO and Scratch P... 0         
18 00:14.0  Intel Corporation                              5520/5500/X58 I/O Hub System Management ... 0         
19 00:07.0  Intel Corporation                              5520/5500/X58 I/O Hub PCI Express Root P... 0         
20 00:03.0  Intel Corporation                              5520/5500/X58 I/O Hub PCI Express Root P... 0         
21 00:01.0  Intel Corporation                              5520/5500/X58 I/O Hub PCI Express Root P... 0         
22 00:00.0  Intel Corporation                              5520/5500/X58 I/O Hub to ESI Port (rev: 19) 0

Re: CPU Load

Posted: Sun Feb 13, 2011 8:23 pm
by martini
and now show your firewall.
do you have nat enabled on this router ?

Re: CPU Load

Posted: Sun Feb 13, 2011 8:40 pm
by exe
P.S. Question for those who have intel motherboards with xeon processors: Do you have different irq for every PCI-E (x1, x4, x8),PCI slot and integrated intel nics?

Bishop,

Disable Audio and USB in BIOS.

Add core for ether interfaces, not auto, different ones.

Every ether should have different irq, then balance irqs better.

Check mine, HT on, I7-950

pppoe4.jpg

Re: CPU Load

Posted: Mon Feb 14, 2011 12:10 am
by mars
we found the more irq we disable in the bios the less there is for ros to use

Re: CPU Load

Posted: Mon Feb 14, 2011 3:20 am
by exe
we found the more irq we disable in the bios the less there is for ros to use
How mean you disable irqs? Do you use older motherboard? It is important that irq be different for every nic.

Re: CPU Load

Posted: Mon Feb 14, 2011 1:58 pm
by Bishop
After switching hypertrading ON and manual purpose interrupts have here is a picture.
Membase4.jpeg
What's interesting is that, when appointing interrupt 2 on eth1-TxRx-0 load on it is always greater than the core 3, although it appointed more: eth1-TxRx-1, eth1-TxRx-2, eth1-TxRx-3, eth1-TxRx-4, eth1-TxRx-5, eth1-TxRx-6, eth1-TxRx-7. Driver problem?

To: exe
Audio chip is disabled in BIOS, but in mikrotik still present.
I need a USB to connect the USB drive with MikrotikOS.

To: martini
Nat not used
my firewall:
/ip firewall connection tracking
set enabled=yes generic-timeout=10m icmp-timeout=10s tcp-close-timeout=10s \
    tcp-close-wait-timeout=10s tcp-established-timeout=1d tcp-fin-wait-timeout=\
    10s tcp-last-ack-timeout=10s tcp-syn-received-timeout=5s \
    tcp-syn-sent-timeout=5s tcp-syncookie=no tcp-time-wait-timeout=10s \
    udp-stream-timeout=3m udp-timeout=10s
/ip firewall filter
add action=accept chain=input connection-state=new disabled=no protocol=icmp
add action=accept chain=input connection-state=established disabled=no
add action=accept chain=input connection-state=related disabled=no
add action=accept chain=input connection-state=new disabled=no src-address-list=input_allowed_list
add action=drop chain=input disabled=no
add action=accept chain=forward connection-state=established disabled=no
add action=accept chain=forward connection-state=related disabled=no
add action=accept chain=forward disabled=no dst-address-list=allow_vip_list
add action=accept chain=forward connection-state=new disabled=no dst-address=165.12.16.128/25
add action=reject chain=forward disabled=no dst-address-list=restricted_list reject-with=icmp-admin-prohibited src-address-list=restricted_list
add action=reject chain=forward disabled=no dst-address=192.168.0.0/16 reject-with=icmp-admin-prohibited src-address-list=restricted_list
add action=reject chain=forward disabled=no dst-address=10.8.8.0/8 reject-with=icmp-admin-prohibited src-address-list=restricted_list
add action=accept chain=forward connection-state=new disabled=no dst-address=172.30.1.0/24 src-address-list=restricted_list
add action=accept chain=forward connection-state=new disabled=no dst-address=172.32.1.0/24 src-address-list=restricted_list
add action=reject chain=forward disabled=no dst-address=172.16.0.0/24  reject-with=icmp-admin-prohibited src-address-list=restricted_list
add action=accept chain=forward connection-state=new disabled=no  src-address-list=allow_to_out_list
add action=reject chain=forward disabled=no reject-with=icmp-admin-prohibited

Re: CPU Load

Posted: Mon Feb 14, 2011 7:13 pm
by martini
try this command /ip firewall connection tracking set enabled=no
You can disable tracking because you not use nat, after that cpu load decrease.

I have big RX and TX drops on intel nics and xeon/core CPU on early v5 beta, after installing RC - all works without any drops. Most stable for me rc5 and rc7.
I have the same nic on 12 routers, 2 of this routers work as PPPoE server (200-400 active clients and 200-500MB traffic) - dont have any drops.

Re: CPU Load

Posted: Tue Feb 15, 2011 5:12 pm
by Bishop
Read on forum http://forum.nag.ru/forum/index.php?sho ... 54586&st=0
that the technology distribution network card interrupts across multiple cores, it works only for incoming TCP/IP traffic and does not work for outbound.

That is what I observe. Distribute the load does not get it on an interface with PPPoE and that does not TCP/IP

Re: CPU Load

Posted: Wed Feb 16, 2011 1:01 pm
by normis
there are some processes that don't benefit from multi threading, so those can run only on single core, for example one core will be 100%, rest will be zero. if you will also run other processes, each of them will be using another core.