Community discussions

MikroTik App
 
frednks
newbie
Topic Author
Posts: 26
Joined: Sun Jul 30, 2017 3:12 am
Location: Brasil

CHR on VMware 9Gbs on Btest low CPU, but in practice it gets high CPU

Wed Aug 19, 2020 3:34 am

Hello, I come to share my experience with CHR on VMware ESXi6.5
My VMware server: Dell T420
CPU (s): 12 x Intel (R) Xeon (R) E5-2420 CPU 0 @ 1.90GHz (1 Socket)
32Gbs memory
480Gb SSD HD
NIC Intel X520-DA2

I created 2 VMs, each with CHR, called CHR-1 with two cores and CHR-2 with three core. Both with 4Gb of memory.

In this test, they are not communicating to the vmware bridge.
CHR-1 and CHR-2 have the SFP + port (dedicated PCI PASSTHROUG) going to switch raisecom SFP +
They communicate through the switch everything with 10Gbs gbic

In test on Btest, it is working fine, passing 9.2Gbs, TCP, from CHR-1 to CHR-2, with 65% of CPU.

PROBLEM: but when I put it to do BGP in practice, without full routes, CPU is 50% that passing 700Mb at the moment.
what could be causing this!

I'll put some prints here:

1) Btest CHR-1 to CHR-2 - CPU it is low due to the traffic that is going through!
CHR-1.png
2) Btest CHR-2 to CHR-1
CHR-2.png
3) VMs - CPU of VMs at the time of this test in Btest.
vmware-Virtual_Machines.png
4) HOST - General VMware CPU at the time of testing
vmware-host-cpu.png
5) For more traffic, I enabled the PCI PASSTHROUGH option, my card is Intel X520-DA2
vmware-pci-passthrough.png
6) VM CHR-1 - CPU Info
vmware-info-cpu-vmchr.png
7) VM CHR-1 - Memory Info
vmware-info-memoria-vmchr.png
8) VM CHR-1 - Hard disk Info
vmware-info-hd-vmchr.png
9) VM CHR-1 - SCSI info
vmware-info-scsi-vmchr.png
That's the settings!

Just complementing about the settings:
I respected the CPU core number, as my intel processor has 6 cores, I left 3 cores for CHR-1 and 2 cores for CHR-2
I disable hyperthreading on the hypervisor's CPU

Am I wrong somewhere?
I ask the CHR specialists to help me with this issue of CPU getting high in BGP without full routes.

Sorry my english, i used a bit of google translator!
I'm from Brasil!
You do not have the required permissions to view the files attached to this post.
 
frednks
newbie
Topic Author
Posts: 26
Joined: Sun Jul 30, 2017 3:12 am
Location: Brasil

Re: CHR on VMware 9Gbs on Btest low CPU, but in practice it gets high CPU

Wed Aug 19, 2020 8:15 pm

I took a print screen of him running BGP in practice.
look in the print that, it is currently only 621Mbps and the CPU is at 42%
I'm only using CHR-1, with three cores as BGP, and the two SFP + interface is on this VM, link in and link out.

I noticed that when the volume increased to 800Mbps the CPU remained at 42%
in fact it ranges from 37% to 45%
the fast-path enabled
without full route
the two BGP sessions are with the same operator, one is backup, so I don't use full routes.
CHR-RUNNING-BGP.png
look that sfp link in, and, sfp1 link out

Note:
I heard that running BGP with CHR, it only uses a hypervision CPU core.
that is, it is useless to put 4 cores, because it will only use one!
is this true?
You do not have the required permissions to view the files attached to this post.
 
frednks
newbie
Topic Author
Posts: 26
Joined: Sun Jul 30, 2017 3:12 am
Location: Brasil

Re: CHR on VMware 9Gbs on Btest low CPU, but in practice it gets high CPU

Thu Sep 03, 2020 3:15 am

@TomjNorthIdaho
I'm quoting you here, I need to take a doubt with you

looking at a topic of yours: viewtopic.php?t=124191
you quote as follows: "On the physical BOX, when ever possible, try not to have any two devices use the same CPU interrupt. Example, two network cards using the same interrupt forces the host operating system to spend time trying to figure out what device needs attention - this takes away from CPU processing speed to do other things."

so my doubt is:
I have an Intel X520-DA2 NIC with two SFP + ports
I left the PCI PASSTHROUGH function active on this NIC card
I left one SFP + port on VM1-CHR1, and the other SFP + port on VM2-CHR2
in that case, I will have problems with CPU interrupt???
 
frednks
newbie
Topic Author
Posts: 26
Joined: Sun Jul 30, 2017 3:12 am
Location: Brasil

Re: CHR on VMware 9Gbs on Btest low CPU, but in practice it gets high CPU

Thu Sep 03, 2020 6:19 pm

Well I read all the topic and following the guidelines of TomjNorthIdaho, I got the following good practices for VMware with CHR.
Using VMware because of the PCI Express card driver, in no case did I use an Intel X520-DA2.
I liked vmware for the ease of updating the network card driver.

IN THE PHYSICAL SERVER BIOS: Delete / remove all devices you don't need
1) Disable all power management:
- In the option POWER MANAGEMENT choose CUSTOM and requires in the options below:
CPU power and performance management ...... => MAXIMUM PERFORMANCE
FAN performance and power management ....... => MAXIMUM PERFORMANCE
Memory energy and performance management ... => MAXIMUM PERFORMANCE
2) Disable Hyper-Threading

SETTINGS on VMware:
- Enable PCI PASSTHROUGH on Network cards that will be dedicated to VM

SETTINGS on the VM:
- Place the VM to enter the BIOS at the next boot, and check out:
1) disable as functions that you do not use such as: CD-ROM; USB; Floppy, Serial Ports
2) disable all power management:
- In the option POWER MANAGEMENT choose CUSTOM and requires in the options below:
CPU power and performance management ...... => MAXIMUM PERFORMANCE
FAN performance and power management ....... => MAXIMUM PERFORMANCE
Memory energy and performance management ... => MAXIMUM PERFORMANCE
- In the CPU properties in Limit, leave unlimited
- In the CPU and MEMORY properties, set the shareds to HIGH
- In the CPU properties in CPU / MMU Virtualization, enter: Hardware CPU and MMU
- In the CPU properties, activate the Hardware Virtualization and Performance counters option
- In the properties of HARD DISK in Type, leave provisioned; in controller location, leave the IDE controller
- In the NETWORK priorities, in Adapter type, leave VMXNET3
- Disable all devices that are not used as CD-ROM, serial port and others
- In the SCSI Controller 0 properties, leave VMWare Paravirtual; in SCSI Bus Sharing let Virutal.

Now I only have the doubt if I leave "delayed ACK" deactivated, as TomjNorthIdaho said the robust 10Gb NIC cards, it may not make any difference.

Who is online

Users browsing this forum: No registered users and 10 guests