Hi, we are currently testing WireGuard to see what possibilities arise from this new implementation.
We noticed speed losses that we cannot explain.

The following simple scenario:
MikroTik RB5009 connects to a MikroTik CHR with a 10GBE uplink in a data center via a 1000Mbit/500Mbit company fiber internet connection.
The latency is about 16ms.

A bandwidth test between the two routers without WireGuard achieves 940Mbit/510Mbit with 20 TCP connections, i.e. very good values.
However, the test only achieved approx. 460 Mbit in both directions if we run it trough WireGuard.

Of course we are aware of the general problem of decreasing bandwidths of TCP connections with increasing latency, but that doesn't seem to be the problem here, since the test outside of WireGuard reaches the full bandwidth. However, WireGuard itself works via UDP, so this shouldn’t actually lead to such a large loss of performance, should it?

Have you had similar experiences or any idea how this came about? Are we missing something?

Thanks,
Joshua

Post CPU usage too from both devices as written above.
As for RB5009 it is capable of more, as I've posted on another topic: viewtopic.php?t=182335#p906457

We measured with the internal bandwidth test. But a test with ipferf between two hosts behind the routers shows similar results. The 1000/500 Internet line seems to offer more than it promises, the provider doesn't seem to limit it so harshly.

I don't think CPU usage is a problem. As you said, the 5009 should do more and runs at a maximum of 39%. The CHR runs at 4x 4.0GHz with less than 10%.

How about UDP between those sites? same speed? you only mentioned 20 tcp streams.

I don't think CPU usage is a problem. As you said, the 5009 should do more and runs at a maximum of 39%. The CHR runs at 4x 4.0GHz with less than 10%.

...running a similar setup (RB4011 with 1G/55M I-Net and 1x3GHz/2M CHR,) I can confirm, that 450Mbps is max for traffic via the wg-link.
As RB4011 and RB5009 have 4C-ARM CPUs....maybe CPU Usage in general is not the right measure, but maybe single core load (maybe wg is single threaded on MT?)

I don't think CPU usage is a problem. As you said, the 5009 should do more and runs at a maximum of 39%. The CHR runs at 4x 4.0GHz with less than 10%.

...running a similar setup (RB4011 with 1G/55M I-Net and 1x3GHz/2M CHR,) I can confirm, that 450Mbps is max for traffic via the wg-link.
As RB4011 and RB5009 have 4C-ARM CPUs....maybe CPU Usage in general is not the right measure, but maybe single core load (maybe wg is single threaded on MT?)

+1 The encryption/decryption of the payload, together with the asymmetric bandwidth (1000/500), plus overhead, imho would have a lot to do with this. To co-incidental that the OP has 500Mbps upload, and get's limited to 460Mbps.

We tested further today:

It's not because of the MikroTik specific implementation. Even an identical WireGuard connection between two Windows computers does not achieve more than the 450 Mbits mentioned.

On the other hand, a second MikroTik CHR connecting in the same data center with 1Gbit uplink to the 10Gbit CHR at 2ms latency achieves 960 Mbits.

I also think that the reason is either the latency or the limited upload speed of 500 Mbits. But since WireGuard is UDP-based, I can't explain it in terms of network technology. Can you?

How about UDP between those sites? same speed? you only mentioned 20 tcp streams.

A test of the bandwidth via an single UDP connection between the routers without WireGuard maxes out at989/524 Mbits.