Community discussions

MikroTik App
 
sbeauchamp
newbie
Topic Author
Posts: 29
Joined: Fri Sep 16, 2016 3:27 pm

Does routerOS and CHR handle multicore systems differently?

Thu Mar 09, 2017 8:32 pm

On a CHR I have installed, i notice a huge difference in throughput when RPS is turned off. When its on, im lucky to get 35mbps. When its off, I can get at least 300mbps. My understanding is that RPS helps to spread the load of packets across multiple cores. Does it make any attempt to keep single flows/connections on the same core so packet ordering stays the same? Does routerOS behave any differently from CHR? The CHR I had was running with 8 cores.
 
pbeverage
just joined
Posts: 1
Joined: Fri Mar 10, 2017 4:35 pm

Re: Does routerOS and CHR handle multicore systems differently?

Fri Mar 10, 2017 4:41 pm

My question is along the same lines. I'm playing with my first CHR in ESXI and it does not appear that the additional cores are being utilized during a traffic-generator test. From my CCRs the traffic-generator does lean on cpu0 more heavily, but the other cores do show usage. But the CHR shows almost no usage at all from the other cores.

Image
 
karwos
Frequent Visitor
Frequent Visitor
Posts: 96
Joined: Thu Apr 02, 2015 7:28 pm
Location: Poland

Re: Does routerOS and CHR handle multicore systems differently?

Sun Mar 12, 2017 5:30 pm

The RPS is software IRQ load balancer.
It's helpful when your ethernet card have 1 Rx/Tx queue and IRQ, and you wanna spread incoming packets through 4-8-16 cores.
If you use ESXi and vmxnet3, which have 4 TX innterrupts, and 4 RX interrupts, just disable RPS, and use multi-queue-ethernet default for best load balance/performance.
If you don't see enough IRQs for your card, then you may use RPS to spread load accros cores.

I also found disabling RPS in esxi+vmxnet3 gets less cpu load, because drivers, hypervisor and interface queues are already SMP optimized.
RPS is good for older eth drivers, or PCI passthrough and using eth cards which don't use msi-x and multiqueuing.
You can get more CPU usage and less performance, if you use RPS for interfaces which already have CPU load balancing by hypervisor, so loadbalancing tasks have to be done multiple times by SW.
 
bbs2web
Member Candidate
Member Candidate
Posts: 232
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: Does routerOS and CHR handle multicore systems differently?

Sun Mar 26, 2017 9:02 pm

Running CHR under KVM using VirtIO and have configured multiple queues for the network card, essentially matching the number of queues to the number of vCPUs. The processes, on the KVM host, are pinned to cores on the same physical CPU.

I unfortunately only see activity on a single CPU, when I set ether1 to use mq-pfifo and then disable RPS.

I know VirtIO drivers support multiple queues but assume the lack of support may relate to the old kernel version in Router OS and should get resolved when Mikrotik release their v7 unicorn?

Who is online

Users browsing this forum: No registered users and 11 guests