Maximizing Proxmox CHR performance

RouterOS Virtualization
1

I’m running CHR on Proxmox Virtual Environment as a core/edge router for our network. It’s having some performance issues and I’m looking for advice on how to resolve them.
Here is the setup:
-Dell PowerEdge R630 with Intel® Xeon® Processor E5-2680 v3
-Proxmox Virtual Environment 6.3-3
-RouterOS CHR 6.46.8 (P unlimited license)

It had been working decently well throughout the day, with its CPU-to-bandwidth ratio being about 1% CPU for every 100Mbps of traffic. But once it got to 2,800 Mbps of traffic, the CPU spiked to 45% and I started seeing 5% packet loss across multiple interfaces.

In Proxmox, I have the VM’s Processors configured as such:
Sockets: 2
Cores: 22
Total Cores: 44
Type: host
CPU units: 1024
Enable NUMA: Yes
I also set the SFP+ Network Devices to use the VirtIO model and 8 Multiqueues. And the Dell BIOS has Performance mode enabled

One odd thing I’m seeing in Winbox is, CPUs 0 and 8 are always being used more than any of the other ones. 8 got up to 55% during peak time last night.

Any tips on improving performance would be much appreciated!

2

Do you have Hyperthreading disabled?

3

Thanks for the reply! When I was building/testing this router, I tried disabling Hyperthreading in the Dell BIOS, but it seemed to just cut the CPU potential in half so I reenabled it. Perhaps it would be beneficial to reenable it and see how it performs with live traffic?

It’s possible to disable Hyperthreading in Proxmox, but it seems to require a script that runs each time it boots up. I figure disabling it at the Dell BIOS is the better approach.

4

chr with proxmox is a disaster, I’ve been attempting for a year now lol.

Few things to try;

PCI Passthrough
Try ovs instead of linux bridges
Try vmxnet3 instead of virtio

5

Have you tried RouterOS v7 ? It has newer drivers with multi-queue support.

6

I’ve disabled hyperthreading on one of our other VMs that handles less traffic. Here are the results.


RouterOS CPU graph:




Proxmox Node CPU graph:




Proxmox Node Server Load graph:





So basically, the CPU doubled, and the Server Load went down slightly.

One other odd thing is that CPU0 always has a higher load than the other ones in RouterOS:





Has anyone seen any difference in performance using VirtIO vs VMXNET3 vs PCI-passthrough?

7

I’ve been considering this, but given all of the features missing and that it’s beta I can’t put it into a production environment… Maybe I’ll test and try if I’m brave.

VM performance - x86 vs CHR