Hi, we are currently testing WireGuard to see what possibilities arise from this new implementation.
We noticed speed losses that we cannot explain.
The following simple scenario:
MikroTik RB5009 connects to a MikroTik CHR with a 10GBE uplink in a data center via a 1000Mbit/500Mbit company fiber internet connection.
The latency is about 16ms.
A bandwidth test between the two routers without WireGuard achieves 940Mbit/510Mbit with 20 TCP connections, i.e. very good values.
However, the test only achieved approx. 460 Mbit in both directions if we run it trough WireGuard.
Of course we are aware of the general problem of decreasing bandwidths of TCP connections with increasing latency, but that doesn’t seem to be the problem here, since the test outside of WireGuard reaches the full bandwidth. However, WireGuard itself works via UDP, so this shouldn’t actually lead to such a large loss of performance, should it?
Have you had similar experiences or any idea how this came about? Are we missing something?
How did you test ?
I trust not running bandwidth tool on the devices themselves ?
If you did, did you also monitor processor usage on both devices ? Since WG runs purely on CPU, if you add BW-tool, it also consumes the same CPU. If you reach 100%, bye bye performance.
From what I have learned it’s best to run on devices connected to both routers (can be Mikrotik as well, can be Windows/Linux/whatever using iperf)
I am a bit confused about getting 940/510 without WG if the fiber is 1000/500 ? How can the test be larger then what’s available ?
We measured with the internal bandwidth test. But a test with ipferf between two hosts behind the routers shows similar results. The 1000/500 Internet line seems to offer more than it promises, the provider doesn’t seem to limit it so harshly.
I don’t think CPU usage is a problem. As you said, the 5009 should do more and runs at a maximum of 39%. The CHR runs at 4x 4.0GHz with less than 10%.
…running a similar setup (RB4011 with 1G/55M I-Net and 1x3GHz/2M CHR,) I can confirm, that 450Mbps is max for traffic via the wg-link.
As RB4011 and RB5009 have 4C-ARM CPUs…maybe CPU Usage in general is not the right measure, but maybe single core load (maybe wg is single threaded on MT?)
+1 The encryption/decryption of the payload, together with the asymmetric bandwidth (1000/500), plus overhead, imho would have a lot to do with this. To co-incidental that the OP has 500Mbps upload, and get’s limited to 460Mbps.
It’s not because of the MikroTik specific implementation. Even an identical WireGuard connection between two Windows computers does not achieve more than the 450 Mbits mentioned.
On the other hand, a second MikroTik CHR connecting in the same data center with 1Gbit uplink to the 10Gbit CHR at 2ms latency achieves 960 Mbits.
I also think that the reason is either the latency or the limited upload speed of 500 Mbits. But since WireGuard is UDP-based, I can’t explain it in terms of network technology. Can you?