Community discussions

MikroTik App
 
hapoo
newbie
Topic Author
Posts: 45
Joined: Wed Apr 24, 2019 1:35 am

Hitting throughput limits when using encryption on CHR

Wed Dec 08, 2021 1:54 am

I'm testing out CHR as a VPN Concentrator for up to 500 concurrent connections with maybe 50 actively using it at any one time. Up until now i've only used actual mikrotik hardware, so its possible I don't know how to optimize the setup but I'm hitting hard caps and want to know if anyone has any experience with this.

I have two machines for the test:
Machine 1 (M1):
AMD Ryzen 5900x with HT off
x570 Board
32gb Ram
Mellanox Connectx-3 sfp+ card
ESXi
VMXNET 3
1 CHR P-10 Instance with 11 cores & 4gb ram
"Public" IP: 192.168.100.10
"Private" IP: 192.168.110.1
no nat/firewall rules

Machine 2 (M2):
AMD Ryzen 2950x with HT off
x399 Board
32gb Ram
Mellanox Connectx-3 sfp+ card
ESXi
VMXNET 3
3 CHR P-10 Instances
"Public" IP: 192.168.100.20
"Private" IP: 192.168.120.1
no nat/firewall rules


Both are connected via DAC to a CRS309.

Running a bandwidth test from the public address of M2 (1 instance) to M1 (1 instance) I get a stable 9.5+Gbps in both tcp tests and udp tests.

Running a bandwidth test from the private address of M2 (1 instance) to M1 (1 instance) through an ike2 ipsec tunnel with sha1 AES-GCM 128 I get a stable ~2.5gbps tcp.

Running a bandwidth test from the private address of M2 (3 instance) to M1 (1 instance) through an ike2 ipsec tunnel with sha1 AES-GCM 128 I get wildly fluctuating ~6gbps tcp, with about 50% CPU utilization.

Running a bandwidth test from the private address of M2 (1 instance) to M1 (1 instance) through a wireguard tunnel I get a stable ~2.5gbps tcp.

Running a bandwidth test from the private address of M2 (3 instance) to M1 (1 instance) through a wireguard tunnel I get wildly fluctuating ~5gbps tcp with about 25% CPU utilization.


I can't seem to get full line speed, and I don't know what the bottleneck is.

Who is online

Users browsing this forum: vic3apex and 19 guests