Community discussions

MikroTik App
 
AiroSam
newbie
Topic Author
Posts: 31
Joined: Mon Jan 30, 2012 7:48 pm
Location: Oklahoma, USA

Maximizing Proxmox CHR performance

Thu May 06, 2021 5:27 pm

I'm running CHR on Proxmox Virtual Environment as a core/edge router for our network. It's having some performance issues and I'm looking for advice on how to resolve them.
Here is the setup:
-Dell PowerEdge R630 with Intel® Xeon® Processor E5-2680 v3
-Proxmox Virtual Environment 6.3-3
-RouterOS CHR 6.46.8 (P unlimited license)

It had been working decently well throughout the day, with its CPU-to-bandwidth ratio being about 1% CPU for every 100Mbps of traffic. But once it got to 2,800 Mbps of traffic, the CPU spiked to 45% and I started seeing 5% packet loss across multiple interfaces.

In Proxmox, I have the VM's Processors configured as such:
Sockets: 2
Cores: 22
Total Cores: 44
Type: host
CPU units: 1024
Enable NUMA: Yes
I also set the SFP+ Network Devices to use the VirtIO model and 8 Multiqueues. And the Dell BIOS has Performance mode enabled

One odd thing I'm seeing in Winbox is, CPUs 0 and 8 are always being used more than any of the other ones. 8 got up to 55% during peak time last night.

Any tips on improving performance would be much appreciated!
 
User avatar
StubArea51
Trainer
Trainer
Posts: 1739
Joined: Fri Aug 10, 2012 6:46 am
Location: stubarea51.net
Contact:

Re: Maximizing Proxmox CHR performance

Thu May 06, 2021 6:32 pm

Do you have Hyperthreading disabled?
 
AiroSam
newbie
Topic Author
Posts: 31
Joined: Mon Jan 30, 2012 7:48 pm
Location: Oklahoma, USA

Re: Maximizing Proxmox CHR performance

Fri May 07, 2021 12:09 am

Do you have Hyperthreading disabled?
Thanks for the reply! When I was building/testing this router, I tried disabling Hyperthreading in the Dell BIOS, but it seemed to just cut the CPU potential in half so I reenabled it. Perhaps it would be beneficial to reenable it and see how it performs with live traffic?

It's possible to disable Hyperthreading in Proxmox, but it seems to require a script that runs each time it boots up. I figure disabling it at the Dell BIOS is the better approach.
 
expo
Frequent Visitor
Frequent Visitor
Posts: 52
Joined: Tue Jan 27, 2009 7:57 am

Re: Maximizing Proxmox CHR performance

Tue May 11, 2021 4:50 am

chr with proxmox is a disaster, I've been attempting for a year now lol.

Few things to try;

PCI Passthrough
Try ovs instead of linux bridges
Try vmxnet3 instead of virtio
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 2095
Joined: Mon Jan 14, 2008 1:53 pm
Location: Over the Rainbow
Contact:

Re: Maximizing Proxmox CHR performance

Tue May 11, 2021 5:54 am

chr with proxmox is a disaster, I've been attempting for a year now lol.

Few things to try;

PCI Passthrough
Try ovs instead of linux bridges
Try vmxnet3 instead of virtio
Have you tried RouterOS v7 ? It has newer drivers with multi-queue support.
 
AiroSam
newbie
Topic Author
Posts: 31
Joined: Mon Jan 30, 2012 7:48 pm
Location: Oklahoma, USA

Re: Maximizing Proxmox CHR performance

Tue May 11, 2021 11:27 pm

Do you have Hyperthreading disabled?

I've disabled hyperthreading on one of our other VMs that handles less traffic. Here are the results.


RouterOS CPU graph:
Image


Proxmox Node CPU graph:
Image


Proxmox Node Server Load graph:
Image



So basically, the CPU doubled, and the Server Load went down slightly.

One other odd thing is that CPU0 always has a higher load than the other ones in RouterOS:
Image



Has anyone seen any difference in performance using VirtIO vs VMXNET3 vs PCI-passthrough?
 
expo
Frequent Visitor
Frequent Visitor
Posts: 52
Joined: Tue Jan 27, 2009 7:57 am

Re: Maximizing Proxmox CHR performance

Thu May 13, 2021 6:43 am

chr with proxmox is a disaster, I've been attempting for a year now lol.

Few things to try;

PCI Passthrough
Try ovs instead of linux bridges
Try vmxnet3 instead of virtio
Have you tried RouterOS v7 ? It has newer drivers with multi-queue support.
I've been considering this, but given all of the features missing and that it's beta I can't put it into a production environment... Maybe I'll test and try if I'm brave.

Who is online

Users browsing this forum: No registered users and 11 guests