Community discussions

MikroTik App
 
sebas1221
just joined
Topic Author
Posts: 1
Joined: Mon Jan 13, 2025 11:36 pm

10 Gbit CHR performance on Proxmox not what it should be?

Tue Jan 14, 2025 12:23 am

Hi everyone,

I'm testing 10 Gbit networking in CHR on proxmox between two CHR VMs with bandwidth test and I'm only able to hit speeds of about 3-3.5 Gbit/s on both UDP and TCP (udp a little bit higher, but encountering network stalls - transfer randomly drops to zero for a second on udp, before recovering). Each CHR has 4x 2.6 GHz cores and 1 GB ddr4 ram. CPU utilization is at around 20-25% on both ends during transfer. CHR instances are connected with linux bridge using VirtIO. Firewall is disabled. I tried multiqueue, changing CPU to "host" - no difference. Vmxnet3 and intel e1000/e1000e all perform worse.
To double check I tried transferring a big file from one virtual windows machine to another over a network share, and hit similar speed (~300 MB/s reported). Hard Drive (ssd) performance is not a factor, capable of 4 GB/s +.

This is a server with an intel xeon 6142 cpu.
VM specs:
image_2025-01-13_225156851.png
Bandwidth test
image_2025-01-13_225719920.png

I also tested with exact same setup on a different server with a Xeon Silver 4210R and on that one I can hit double the speed on TCP at 6+ Gb/s and nearly triple on UDP at 10 Gb/s on UDP, while also not encountering any connection stalls. It is seems strange how this lower performing CPU (albeit 2 years newer) is able to do this, while the other one cant:
Bandwidth test
image_2025-01-13_230742683.png

Anyone got any clue or tips how I can improve performance or what am I doing wrong?
You do not have the required permissions to view the files attached to this post.
 
elbob2002
Member Candidate
Member Candidate
Posts: 285
Joined: Tue May 15, 2018 8:15 pm
Location: Ireland

Re: 10 Gbit CHR performance on Proxmox not what it should be?

Tue Jan 14, 2025 10:29 am

Don't use the built in bandwidth test as it can hammer the CPU affecting performance and not giving real performance results.

Use something like iperf on each side of your CHR to measure traffic through your CHR.

Also your choice of NIC driver for the VM can affect performance. For best performance you should use passthrough to the physical NIC.