Community discussions

MikroTik App
 
ksoze
just joined
Topic Author
Posts: 8
Joined: Fri Nov 26, 2021 11:28 pm

RB5009 Wireguard only 150 Mbps

Mon Jan 17, 2022 9:55 pm

It appears my Wireguard setup is much slower on the RB5009 than on the RB4011, achieving only 150 Mbps to the same endpoint. Has anyone else experienced this issue?
 
erlinden
Forum Guru
Forum Guru
Posts: 1900
Joined: Wed Jun 12, 2013 1:59 pm
Location: Netherlands

Re: RB5009 Wireguard only 150 Mbps

Mon Jan 17, 2022 10:23 pm

What speed did you achieve with the RB4011?
Did it have the same configuration and did it use the same ISP connection?
Can you share the Wireguard part of the config from both routers?

/interface/wireguard export

I'm interested with the findings (as I have the same behavior between an RB4011 and a hEX S).
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18959
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: RB5009 Wireguard only 150 Mbps

Mon Jan 17, 2022 10:30 pm

Without seeing your config, its just noise...........
 
User avatar
kiler129
Member
Member
Posts: 352
Joined: Tue Mar 31, 2015 4:32 pm
Location: IL, USA
Contact:

Re: RB5009 Wireguard only 150 Mbps

Tue Jan 18, 2022 12:06 am

Also, how are you testing the speed? iperf between endpoints or btest?
 
tty1
just joined
Posts: 21
Joined: Thu Dec 09, 2021 10:34 pm

Re: RB5009 Wireguard only 150 Mbps

Tue Jan 18, 2022 12:09 am

Have you tried to evaluate the performance of the WireGuard on RB5009 in the local network? You need to control variables to narrow down the potential factors that cause the performance bottleneck.
 
ksoze
just joined
Topic Author
Posts: 8
Joined: Fri Nov 26, 2021 11:28 pm

Re: RB5009 Wireguard only 150 Mbps

Tue Jan 18, 2022 4:10 am

This configuration is the same on both routers—meticulously ported from export.

At this point, I suspect the difference is due to some interaction with the gigabit Ethernet adapter attached to the Thunderbolt doc. of my M1 Mac; other machines on the same network show higher Wireguard throughput. Another difference is that my test machine is directly attached to the bridge, while most other machines use an intervening switch.

Perhaps an interrupt coalescing issue? The profile shows the CPU at around 40%, with a large percentage dedicated to "ethernet."
 
psannz
Member Candidate
Member Candidate
Posts: 127
Joined: Mon Nov 09, 2015 3:52 pm
Location: Renningen, Germany

Re: RB5009 Wireguard only 150 Mbps

Tue Jan 18, 2022 3:05 pm

It appears my Wireguard setup is much slower on the RB5009 than on the RB4011, achieving only 150 Mbps to the same endpoint. Has anyone else experienced this issue?
Afaik, there is no IPSEC HW acceleration yet on the RB5009. Thus, Wireguard is done in software.
 
User avatar
jbl42
Member Candidate
Member Candidate
Posts: 214
Joined: Sun Jun 21, 2020 12:58 pm

Re: RB5009 Wireguard only 150 Mbps

Tue Jan 18, 2022 3:33 pm

Afaik, there is no IPSEC HW acceleration yet on the RB5009. Thus, Wireguard is done in software.
Currently not. But the RB5009 SoC supports crypto HW offload for IPSEC, Wireguard etc. MT support told me making it available in future ROS releases is to be expected.

Until this happens, the RB4011 is the better choice over RB5009 for high-bandwidth applications of IPSEC and VPNs like wireguard.
IMO, this should clearly be mentioned in the specs. For such use cases, this is a deal breaker.
 
gabacho4
Member
Member
Posts: 329
Joined: Mon Dec 28, 2020 12:30 pm
Location: Earth

Re: RB5009 Wireguard only 150 Mbps

Tue Jan 18, 2022 4:42 pm

Clearly not reading the release notes.

What's new in 7.1rc3 (2021-Sep-08 13:29):
*) added IPSec hardware acceleration support for RB5009;
 
User avatar
eworm
Forum Guru
Forum Guru
Posts: 1070
Joined: Wed Oct 22, 2014 9:23 am
Location: Oberhausen, Germany
Contact:

Re: RB5009 Wireguard only 150 Mbps

Tue Jan 18, 2022 4:59 pm

There is no hardware acceleration for Wireguard. That is always done in software.
 
User avatar
Znevna
Forum Guru
Forum Guru
Posts: 1347
Joined: Mon Sep 23, 2019 1:04 pm

Re: RB5009 Wireguard only 150 Mbps

Tue Jan 18, 2022 5:14 pm

IPsec has nothing to do with WireGuard.
Regarding RB5009 and WireGuard, RB5009 can do a little more than 150Mbps:
RB5009UG+S+IN WireGuard 001.PNG
RB5009UG+S+IN WireGuard 002.PNG
RB5009UG+S+IN WireGuard 003.PNG
You do not have the required permissions to view the files attached to this post.
 
User avatar
jbl42
Member Candidate
Member Candidate
Posts: 214
Joined: Sun Jun 21, 2020 12:58 pm

Re: RB5009 Wireguard only 150 Mbps

Tue Jan 18, 2022 9:24 pm

Clearly not reading the release notes.

What's new in 7.1rc3 (2021-Sep-08 13:29):
*) added IPSec hardware acceleration support for RB5009;
Good to hear I stand corrected for IPSEC and 7.1. Can't wait to give it a new try.
My last tests happend on 7.0.5, and I missed the 7.1rc3 release notes just checking the ones for official 7.1 and 7.1.1 releases...
Having release notes for official releases listing all changes since the last official release would be appreciated.

Znevna's Wireguard througput measurements is what one would expect regarding the RB5009 CPU number crunching power.
Wireguard is using ChaCha20 cypher for whitch no HW acceleration exists yet.
 
ksoze
just joined
Topic Author
Posts: 8
Joined: Fri Nov 26, 2021 11:28 pm

Re: RB5009 Wireguard only 150 Mbps

Sun Jan 23, 2022 2:26 am

This does indeed look like an interaction with the cheap gigabit transceiver of a Thunderbolt hub. Using a better USB-C to GigE adapter seems to have solved the issue. Not sure why the cheap adapter is better on other Mikrotiks, though.
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1025
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: RB5009 Wireguard only 150 Mbps

Sun Mar 13, 2022 3:45 pm

Currently not. But the RB5009 SoC supports crypto HW offload for IPSEC, Wireguard etc. MT support told me making it available in future ROS releases is to be expected.

Did they by any chance reveal any info regarding hw offloading, ie if any of the architectures are able to support ARX vector operations (or similar like the AVX2 instruction set) that can assist ChaCha20 to offload the cpu in the same way as for AES?
 
t4thfavor
just joined
Posts: 18
Joined: Tue Apr 13, 2021 4:40 pm

Re: RB5009 Wireguard only 150 Mbps

Mon Mar 14, 2022 6:20 am

Currently not. But the RB5009 SoC supports crypto HW offload for IPSEC, Wireguard etc. MT support told me making it available in future ROS releases is to be expected.

Did they by any chance reveal any info regarding hw offloading, ie if any of the architectures are able to support ARX vector operations (or similar like the AVX2 instruction set) that can assist ChaCha20 to offload the cpu in the same way as for AES?

For Wireguard, the implementation in software is so fast that nobody has bothered to build any hardware accelerators. There's some threads on it on the Netgate forums and a few other places. The CPU software implementation just gets faster as CPU's do, so I doubt anyone will ever make an accelerator for Wireguard except for maybe if the offshore vpn's start getting too slow (and adding more endpoints gets more expensive)
 
User avatar
Buckeye
Forum Veteran
Forum Veteran
Posts: 883
Joined: Tue Sep 11, 2018 2:03 am
Location: Ohio, USA

Re: RB5009 Wireguard only 150 Mbps

Mon Mar 14, 2022 7:16 am

This does indeed look like an interaction with the cheap gigabit transceiver of a Thunderbolt hub. Using a better USB-C to GigE adapter seems to have solved the issue. Not sure why the cheap adapter is better on other Mikrotiks, though.
Maybe it is because with slower MikroTiks it doesn't exceed the capabilities of the Thunderbolt hub, so fewer frames are dropped.

Did you compare iperf retransmissions on the slower MikroTiks vs. the RB5009?

The numbers posted by @Znevna are quite impressive. Especially when compared to OpenVPN.
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1025
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: RB5009 Wireguard only 150 Mbps

Mon Mar 14, 2022 9:21 am

For Wireguard, the implementation in software is so fast that nobody has bothered to build any hardware accelerators. There's some threads on it on the Netgate forums and a few other places. The CPU software implementation just gets faster as CPU's do, so I doubt anyone will ever make an accelerator for Wireguard except for maybe if the offshore vpn's start getting too slow (and adding more endpoints gets more expensive)

What is that observation based on? A single connection of your own or something you've read about?

What is important to me to get verified is if a possible WG solution might be sufficient enough to hold multiple (>50) sessions for remote users who need secure access to the companies infrastructure (ie "road warrior" vpn). The objective is to make an informed decision if it's possible to replace an existing VPN solution that is both expensive and administratively demanding.
 
holvoetn
Forum Guru
Forum Guru
Posts: 5317
Joined: Tue Apr 13, 2021 2:14 am
Location: Belgium

Re: RB5009 Wireguard only 150 Mbps

Mon Mar 14, 2022 9:38 am

What is important to me to get verified is if a possible WG solution might be sufficient enough to hold multiple (>50) sessions for remote users who need secure access to the companies infrastructure (ie "road warrior" vpn). The objective is to make an informed decision if it's possible to replace an existing VPN solution that is both expensive and administratively demanding.
What will it cost you more then some time ?
Maybe switch over some remote users case by case and monitor how it goes as more get added (esp. CPU usage).
It's dead simple.
The only thing lacking from Mikrotik's WG implementation is exactly this what you're looking at: mass-setup and -deployment of users. It's all manual for now.

I assume your current bandwidth is sufficient to serve all those users ?
 
huntermic
Member Candidate
Member Candidate
Posts: 111
Joined: Wed Oct 26, 2016 3:42 pm

Re: RB5009 Wireguard only 150 Mbps

Mon Mar 14, 2022 9:48 am

It appears my Wireguard setup is much slower on the RB5009 than on the RB4011, achieving only 150 Mbps to the same endpoint. Has anyone else experienced this issue?
Using a RB5009 i reach over 700Mbit over wireguard.
 
holvoetn
Forum Guru
Forum Guru
Posts: 5317
Joined: Tue Apr 13, 2021 2:14 am
Location: Belgium

Re: RB5009 Wireguard only 150 Mbps

Mon Mar 14, 2022 9:52 am

Also, how are you testing the speed? iperf between endpoints or btest?
VERY important remark ... unless I missed it I did not see a response to this question.

EDIT: got it. Switch was changed. And what is the new result ?
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1025
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: RB5009 Wireguard only 150 Mbps

Mon Mar 14, 2022 1:28 pm

What will it cost you more then some time ? Maybe switch over some remote users case by case and monitor how it goes as more get added (esp. CPU usage).
It's dead simple.The only thing lacking from Mikrotik's WG implementation is exactly this what you're looking at: mass-setup and -deployment of users. It's all manual for now.

We've performed a numerous of tests like that and it's somewhat naive to claim it's just "dead easy" in terms of hassle and time needed. Also, as a part of the initial assessment you also need to find out if prerequisites are met before you even consider to plan for any tests.
 
holvoetn
Forum Guru
Forum Guru
Posts: 5317
Joined: Tue Apr 13, 2021 2:14 am
Location: Belgium

Re: RB5009 Wireguard only 150 Mbps

Mon Mar 14, 2022 2:15 pm

We've performed a numerous of tests like that and it's somewhat naive to claim it's just "dead easy" in terms of hassle and time needed.
And what were the results of your tests, if I may ask ?
Was a lot of post-setup support needed ?
Lot's of interventions for users having problems ? More or less then your current solution ?
For those able to use it reliably, what did they experience ?

The only way you can really know if it works is to simulate all those connections using some traffic generators and whatnot. Not an easy task, that I am well aware of.
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1025
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: RB5009 Wireguard only 150 Mbps

Mon Mar 14, 2022 2:41 pm

My bad, I should've written tests in general but my point is that real performance tests performed for business purposes are never ever "dead easy". For obvious reasons this is normally know facts for people involved in this line of work.

So the question remains, if/when cc20 be implemented using arm vector support (or similar) if it even exists in the currents soc's.
 
holvoetn
Forum Guru
Forum Guru
Posts: 5317
Joined: Tue Apr 13, 2021 2:14 am
Location: Belgium

Re: RB5009 Wireguard only 150 Mbps

Mon Mar 14, 2022 3:33 pm

My bad, I should've written tests in general but my point is that real performance tests performed for business purposes are never ever "dead easy". For obvious reasons this is normally know facts for people involved in this line of work.
Depends on your population :lol:
I have some sales guys claiming (REALLY !) not knowing how to work with a fairly easy reporting program (pretty basic to work the numbers for sales, but who am I to question that) but each year they can book their travel online. And their bank stuff they also do online.
Don't get me started on their account password. PIN code for their bank card I never get but their account password, some forget it on a bi-weekly basis.
Willing and wanting are two different things, so it seems.

Using something like Forticlient or Wireguard client on Windows should be (conceptually) the same, no ? Activate the program if you want the tunnel to be made. Don't activate it, and you get no access to company resources.
What's more complex from a user point of view ? Just wondering (it's an open question from my side).

However, you did not respond fully.
If not with end-users, surely you have done some tests with a selected population using wireguard as VPN service (key- and/or powerusers). Persons you know who will not ask you were to find a certain button to click on ?
How did that go ? How many did run in parallel ? What was the load on your target system ?
You really need to do those tests yourself, you can wait for a long time before you'll get that answer from anyone. User adaption for personal use is (as always) way upfront compared to enterprise adaption.
My view.
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1025
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: RB5009 Wireguard only 150 Mbps

Mon Mar 14, 2022 6:55 pm

A lot of statements and questions thus I'm not sure what you are going with all this.

Anyhow, most every assessment and related tests differs from time to time depending of different criteria's and business requirements thus there is no general point of view of how these tests should be conducted and the expected end result.

Our way of working with the technical stuff is usually just one small part of the whole assignment. There are usually plenty of other challenges that need to be considered like budget, project planning, implementation, managing customer expectations, new requirements, training, etc, etc, etc. The list often feels endless when you are situated in the middle of the mess...
 
ffries
Member Candidate
Member Candidate
Posts: 177
Joined: Wed Aug 25, 2021 6:07 pm

Re: RB5009 Wireguard only 150 Mbps

Tue Jun 28, 2022 11:48 am

790Mbit/s is relatively slow compared to IPSEC accelerated with AES-NI.
You should use iperf3 for proper testing, furthermore did you use the 10Gb/s SFP+ port or the 2,5 Gb/s Ethernet port?
 
Whitehawk29FR
just joined
Posts: 18
Joined: Thu Oct 06, 2022 12:14 pm

Re: RB5009 Wireguard only 150 Mbps

Wed Jun 28, 2023 7:59 pm

Hello all,

So finally how much bandwith we can get through wireguard VPN on RB5009 ?

I have actually ~300Mbps over 1G connexion.

Regards
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18959
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: RB5009 Wireguard only 150 Mbps

Wed Jun 28, 2023 8:16 pm

That is probably a reasonable expectation actually.
 
ToTheCLI
Frequent Visitor
Frequent Visitor
Posts: 83
Joined: Mon Jan 04, 2016 3:54 am

Re: RB5009 Wireguard only 150 Mbps

Tue Jul 04, 2023 2:22 am

I have a symmetrical 1Gbps connection when using the RB5009 as a peer for a Wireguard VPN server(Node close by) I get about ~800Mbps down and when I connect to RB5009 as a Wireguard server with a PC connected by ethernet I get ~800Mbps but I get ~170Mbps to the RB5009 even when connected on same Network when using android (Galaxy S23) even though the WiFi can handle much more...
TL;DR
Seems Wireguard on android is limited to an extent can not get it to do more then 200Mbps (Average ~170Mbps)
 
User avatar
inteq
Member
Member
Posts: 402
Joined: Wed Feb 25, 2015 8:15 pm
Location: Romania

Re: RB5009 Wireguard only 150 Mbps

Sun Aug 27, 2023 7:31 pm

@ToTheCLI it seems in my case it is even worse on Android.

Local speed test through a RB1100AHx4 to a local speedtest server via WiFi ( Cap AX with only AC enabled)
Client: Android 13 (Moto Edge 40 Pro)

Wireguard enabled
wg.jpg
Wireguard disabled
no-wg.jpg
You do not have the required permissions to view the files attached to this post.
Last edited by inteq on Sun Aug 27, 2023 11:02 pm, edited 1 time in total.
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1025
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: RB5009 Wireguard only 150 Mbps

Sun Aug 27, 2023 10:50 pm

WireGuard encryption (ChaCha20) lacks support for hardware acceleration which makes it entirely dependent on CPU speed at both ends.

Who is online

Users browsing this forum: No registered users and 13 guests