Community discussions

MikroTik App
 
romrider
just joined
Topic Author
Posts: 10
Joined: Fri Oct 19, 2018 9:44 pm

Wireguard Slow upload

Fri Oct 28, 2022 1:06 pm

Hi there,

I'm running a roadwarrior wireguard setup on my CCR1009-7G-1C-1S+ (running version 7.6) on a fiber connection 1Gbps Down/500Mbps Up.
MTU set at 1420 on client and server.

Everything works well appart from upload speed...

Uploading to a server behind the VPN from a client (all running the latest version of the client):
  • Android phone client: 4.5Mbps
  • MacOS client: 4.5Mbps
  • Windows client: 4.5Mbps
Download speed is fine on any client and is around 250Mbps. The Mikrotik processor is never saturated.

Anything I could check to understand what is happening?
 
User avatar
BartoszP
Forum Guru
Forum Guru
Posts: 2855
Joined: Mon Jun 16, 2014 1:13 pm
Location: Poland

Re: Wireguard Slow upload

Sat Oct 29, 2022 11:41 pm

 
romrider
just joined
Topic Author
Posts: 10
Joined: Fri Oct 19, 2018 9:44 pm

Re: Wireguard Slow upload

Wed Nov 09, 2022 12:21 am

Thanks, but I know how to search and already read all those posts, they don't contain anything useful.
Since my initial post, I've switched to a CCR2004-16G-2S+

I've done some more tests: Setup is this one : Computer <-> Internet <-> Mikrotik <-> LAN <-> iperf Server
  • Computer connected to Wireguard VPN running on CCR (all the iperf are run from the computer):
    • Download (TCP)/Server Sending:
      iperf3 -c XXXX -p 5001 -4 -b 400M -R
      
      Connecting to host XXXX, port 5001
      Reverse mode, remote host XXXX is sending
      [  5] local YYYY port 37768 connected to XXXX port 5001
      [ ID] Interval           Transfer     Bitrate
      [  5]   0.00-1.00   sec  47.5 MBytes   398 Mbits/sec
      [  5]   1.00-2.00   sec  45.3 MBytes   381 Mbits/sec
      [  5]   2.00-3.00   sec  39.7 MBytes   333 Mbits/sec
      [  5]   3.00-4.00   sec  45.4 MBytes   380 Mbits/sec
      [  5]   4.00-5.00   sec  51.3 MBytes   430 Mbits/sec
      [  5]   5.00-6.00   sec  51.8 MBytes   434 Mbits/sec
      [  5]   6.00-7.00   sec  52.8 MBytes   443 Mbits/sec
      [  5]   7.00-8.00   sec  47.6 MBytes   400 Mbits/sec
      [  5]   8.00-9.00   sec  47.7 MBytes   400 Mbits/sec
      [  5]   9.00-10.00  sec  47.7 MBytes   400 Mbits/sec
      - - - - - - - - - - - - - - - - - - - - - - - - -
      [ ID] Interval           Transfer     Bitrate         Retr
      [  5]   0.00-10.02  sec   478 MBytes   400 Mbits/sec  314             sender
      [  5]   0.00-10.00  sec   477 MBytes   400 Mbits/sec                  receiver
      
    • Upload (TCP)/Server Receiving:
      iperf3 -c XXXX -p 5001 -4 -b 400M
       
      Connecting to host XXXX, port 5001
      [  5] local YYYY port 37772 connected to XXXX port 5001
      [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
      [  5]   0.00-1.02   sec   655 KBytes  5.26 Mbits/sec    9   36.6 KBytes
      [  5]   1.02-2.05   sec   618 KBytes  4.91 Mbits/sec    0   47.0 KBytes
      [  5]   2.05-3.05   sec   564 KBytes  4.63 Mbits/sec    0   54.9 KBytes
      [  5]   3.05-4.04   sec   527 KBytes  4.35 Mbits/sec    0   61.4 KBytes
      [  5]   4.04-5.06   sec   600 KBytes  4.84 Mbits/sec    0   66.6 KBytes
      [  5]   5.06-6.05   sec   627 KBytes  5.17 Mbits/sec    0   73.2 KBytes
      [  5]   6.05-7.04   sec   489 KBytes  4.04 Mbits/sec    0   78.4 KBytes
      [  5]   7.04-8.01   sec   604 KBytes  5.10 Mbits/sec    0   99.3 KBytes
      [  5]   8.01-9.00   sec   632 KBytes  5.21 Mbits/sec    0    140 KBytes
      [  5]   9.00-10.03  sec   640 KBytes  5.10 Mbits/sec    0    197 KBytes
      - - - - - - - - - - - - - - - - - - - - - - - - -
      [ ID] Interval           Transfer     Bitrate         Retr
      [  5]   0.00-10.03  sec  5.82 MBytes  4.86 Mbits/sec    9             sender
      [  5]   0.00-10.40  sec  5.65 MBytes  4.56 Mbits/sec                  receiver
      
  • Computer targeting directly the iperf server (forwarded the port through dnat)
    • Download (TCP)/Server Sending:
       iperf3 -c XXXX -p 5001 -4 -b 400M -R
      Connecting to host XXXX, port 5001
      Reverse mode, remote host vpn.wnetworks.org is sending
      [  5] local YYYY port 43072 connected to XXXX port 5001
      [ ID] Interval           Transfer     Bitrate
      [  5]   0.00-1.00   sec  46.4 MBytes   389 Mbits/sec
      [  5]   1.00-2.00   sec  48.9 MBytes   411 Mbits/sec
      [  5]   2.00-3.00   sec  47.6 MBytes   400 Mbits/sec
      [  5]   3.00-4.00   sec  47.7 MBytes   400 Mbits/sec
      [  5]   4.00-5.00   sec  47.6 MBytes   400 Mbits/sec
      [  5]   5.00-6.00   sec  47.8 MBytes   400 Mbits/sec
      [  5]   6.00-7.00   sec  47.5 MBytes   399 Mbits/sec
      [  5]   7.00-8.00   sec  47.8 MBytes   401 Mbits/sec
      [  5]   8.00-9.00   sec  47.7 MBytes   400 Mbits/sec
      [  5]   9.00-10.00  sec  47.5 MBytes   399 Mbits/sec
      - - - - - - - - - - - - - - - - - - - - - - - - -
      [ ID] Interval           Transfer     Bitrate         Retr
      [  5]   0.00-10.02  sec   478 MBytes   400 Mbits/sec   90             sender
      [  5]   0.00-10.00  sec   477 MBytes   400 Mbits/sec                  receiver
      
    • Upload (TCP)/Server receiving:
      iperf3 -c XXXX -p 5001 -4 -b 400M
      Connecting to host XXXX, port 5001
      [  5] local YYYY port 43076 connected to XXXX port 5001
      [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
      [  5]   0.00-1.01   sec  11.0 MBytes  92.1 Mbits/sec    0    488 KBytes
      [  5]   1.01-2.00   sec  46.1 MBytes   389 Mbits/sec    0   2.56 MBytes
      [  5]   2.00-3.00   sec  54.1 MBytes   452 Mbits/sec    0   2.78 MBytes
      [  5]   3.00-4.01   sec  53.1 MBytes   445 Mbits/sec    0   2.78 MBytes
      [  5]   4.01-5.00   sec  51.1 MBytes   430 Mbits/sec    1   2.78 MBytes
      [  5]   5.00-6.00   sec  48.1 MBytes   405 Mbits/sec    1   2.78 MBytes
      [  5]   6.00-7.00   sec  52.6 MBytes   441 Mbits/sec    0   2.78 MBytes
      [  5]   7.00-8.01   sec  38.0 MBytes   314 Mbits/sec    1   2.78 MBytes
      [  5]   8.01-9.02   sec  48.5 MBytes   405 Mbits/sec    0   2.78 MBytes
      [  5]   9.02-10.00  sec  49.0 MBytes   417 Mbits/sec    0   2.78 MBytes
      - - - - - - - - - - - - - - - - - - - - - - - - -
      [ ID] Interval           Transfer     Bitrate         Retr
      [  5]   0.00-10.00  sec   452 MBytes   379 Mbits/sec    3             sender
      [  5]   0.00-10.03  sec   452 MBytes   378 Mbits/sec                  receiver
      

In the end we have:
WG used: ~400Mbps Down/~5Mbps up (!!)
WG not used: ~400Mbps Down/~400Mbps up

==> Can't be a CPU bottleneck
==> MTU is 1390 (also tester 1420 with same results) on both WG server and client, which shouldn't be an issue (it's a plain fiber connection with 1500 MTU)
 
biomesh
Long time Member
Long time Member
Posts: 561
Joined: Fri Feb 10, 2012 8:25 pm

Re: Wireguard Slow upload

Wed Nov 09, 2022 12:35 am

On a CCR2004-16G-2S+ I get ~4.6 Gbps single threaded iperf with no wireguard both directions. With wireguard, I get 1.17Gbps (without -R) and 833Mbps (with -R). All tests single threaded. You might want to check your wireguard config on the computer or firewall/antivirus, etc.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Wireguard Slow upload

Wed Nov 09, 2022 12:38 am

Without evidence its all conjecture!
I would need to see the full config on the MT wireguard router/device (server for handshake).

/export file=anynameyouwish (minus device serial # and any public WANIP information ).
 
MikroUser
newbie
Posts: 47
Joined: Sat Sep 07, 2013 1:56 pm

Re: Wireguard Slow upload

Wed Nov 09, 2022 8:54 am

Try turn off fasttrack if enabled.
 
romrider
just joined
Topic Author
Posts: 10
Joined: Fri Oct 19, 2018 9:44 pm

Re: Wireguard Slow upload

Wed Nov 09, 2022 9:16 am

Edit, did the test again and actually it gives me ~150Mbits so it's not related to UDP...


==== Ignore below message =====
Thanks, I'll share my config soon, however I've figured that using iperf with UDP without wireguard gives me the same poor upload speed of around 5Mbits. So it seems unrelated to wireguard but related to UDP... Given wireguard uses UDP, I wanted to be in the same conditions.

(all the tests above where using TCP, so TCP in UDP for when the tunnel was established)
Last edited by romrider on Wed Nov 09, 2022 12:21 pm, edited 1 time in total.
 
romrider
just joined
Topic Author
Posts: 10
Joined: Fri Oct 19, 2018 9:44 pm

Re: Wireguard Slow upload

Wed Nov 09, 2022 9:54 am

So this is super strange: I've tested this from my android phone and can reproduce it 100% of the time:
1. Wireguard disconnected
2. Switch to 4G
3. Connect wireguard
4. Run iperf: around 5Mbits
5. While iperf is running, switch to wifi
6. Iperf gives now 150Mbits (I can run as many as I want, it will stay around 150M)
7. Disconnect wireguard
8. reconnect wireguard (we're still on wifi)
9. Iperf gives now around 5Mbits again
Last edited by romrider on Wed Nov 09, 2022 9:07 pm, edited 2 times in total.
 
romrider
just joined
Topic Author
Posts: 10
Joined: Fri Oct 19, 2018 9:44 pm

Re: Wireguard Slow upload

Wed Nov 09, 2022 10:23 am

Try turn off fasttrack if enabled.
It's not enabled as I'm using /ip bridge filter rules
 
romrider
just joined
Topic Author
Posts: 10
Joined: Fri Oct 19, 2018 9:44 pm

Re: Wireguard Slow upload

Wed Nov 09, 2022 12:56 pm

Without evidence its all conjecture!
I would need to see the full config on the MT wireguard router/device (server for handshake).

/export file=anynameyouwish (minus device serial # and any public WANIP information ).
Here you go, attached. I removed any reference to any public and private IP/domain names/passwords/login/...

You can ignore the second wireguard tunnel, it's for fallback ingress access through a LTE router (which doesn't have a public IP, it's through CGN on the operator side). Also this fallback LTE is on vlan255, that's why you see 2 default routes (but second one is with distance 2)

Let me know if you need more details.
You do not have the required permissions to view the files attached to this post.
 
cwtien
just joined
Posts: 4
Joined: Wed Dec 23, 2020 5:39 am

Re: Wireguard Slow upload

Thu Nov 10, 2022 3:39 am

I thought the issue was with my setup only...haha.

I have an Azure VM that I connect to home sometimes. I use the Azure VM to download stuff, then transfer to my NAS. I use 2 methods:
  • Wireguard.
  • Open up NAS HTTPS port to the internet while I transfer file.
On Wireguard, I get ~ 5 MB/sec upload speed. On HTTPS I get full speed (30 MB/sec).
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Wireguard Slow upload

Thu Nov 10, 2022 2:48 pm

config is unreadable, what is STRIPPED,,,,,,,,, I said public WANIP info, off to help next person that can follow instructions......
 
romrider
just joined
Topic Author
Posts: 10
Joined: Fri Oct 19, 2018 9:44 pm

Re: Wireguard Slow upload

Thu Nov 10, 2022 4:12 pm

config is unreadable, what is STRIPPED,,,,,,,,, I said public WANIP info, off to help next person that can follow instructions......
I won't share my config with more information on a public forum.
 
romrider
just joined
Topic Author
Posts: 10
Joined: Fri Oct 19, 2018 9:44 pm

Re: Wireguard Slow upload

Thu Nov 10, 2022 4:25 pm

I thought the issue was with my setup only...haha.

I have an Azure VM that I connect to home sometimes. I use the Azure VM to download stuff, then transfer to my NAS. I use 2 methods:
  • Wireguard.
  • Open up NAS HTTPS port to the internet while I transfer file.
On Wireguard, I get ~ 5 MB/sec upload speed. On HTTPS I get full speed (30 MB/sec).
My ISP is Orange in France. I start to suspect some rate limiting somewhere on the ISP side... I've tried to run a wireguard server on a server behind the router by forwarding the port directly, and I have the same results (i.e. full speed download and 5Mbits upload from the client's perspective). It is very strange...

I'm going to try with ipsec and see what the result is... but I don't like the idea of having to use IPSec...

Also I was able to reproduce the weird behavior from viewtopic.php?p=966917#p966647 (switching from 4G to some wifi) with a laptop and exactly the same behavior as with my phone (>100Mbits until I restat the tunnel...)
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Wireguard Slow upload

Thu Nov 10, 2022 5:38 pm

config is unreadable, what is STRIPPED,,,,,,,,, I said public WANIP info, off to help next person that can follow instructions......
I won't share my config with more information on a public forum.
Of course, I fully understand now that private addresses are a security risk. I didnt know.

So let me repost

/export file=anynameyouwish ( minus serial number and any public WANIP information AND for private IPs, replace with fake IPs but that are consistent accross the config ).

If you want help then you will have to make the effort.
 
romrider
just joined
Topic Author
Posts: 10
Joined: Fri Oct 19, 2018 9:44 pm

Re: Wireguard Slow upload

Thu Nov 10, 2022 7:46 pm



I won't share my config with more information on a public forum.
Of course, I fully understand now that private addresses are a security risk. I didnt know.

So let me repost

/export file=anynameyouwish ( minus serial number and any public WANIP information AND for private IPs, replace with fake IPs but that are consistent accross the config ).

If you want help then you will have to make the effort.
Here you go, it should be more readable.
You do not have the required permissions to view the files attached to this post.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Wireguard Slow upload

Thu Nov 10, 2022 10:46 pm

(1) Recommend move Vlanid=1 to vlan id=101 (dont use vlan1 for data as its the default vlan MT uses in the background
(2) Stemming from that, dont use the bridge for DHCP or anything else, just bridging.

(3) So adding vlan101 into the mix there are six in total.

a. why only 4 pools
b. why only 4 dhcp servers
c. why only 4 dhcp server networks

(4) Cannot wrap my head around these errors.........
/interface bridge port
add bridge=br-wan interface=vlan832
add bridge=bridge1 interface="SW2 Ports"

First, a vlan is not an interface to be identified in interface bridge ports, its one interface per line AKA an ethernet interface or a WLAN interface.
The second implies multiple interfaces on a single line, not done. But there is no SW2 Ports interface that I see?? You are mixing up interface with interface list.
One cannot put interface lists here!!

(5) Dont understand the bridge filter item and why its not applied only to BR-WAN. Never used bridge filters so not sure.
Also not sure why it has anything to do with fasstrack in the forward chain??

(6) THis would have to be modified too.
add bridge=bridge1 tagged=bridge1 untagged=\
ether9,ether10,ether11,ether12,ether13,ether14,ether15,ether16 vlan-ids=101

(7) So all your ethernet ports are hybrid ports???

(8) Find that you have gone overboard on firewall rules such that its near impossible to see teh forest for the trees.
Much better to get rid of all the pretent I am protecting my router BS and stick to two themes.
a. default rules
b. actual required rules for required traffic

(9) Many items are above my head in the config.....
c. drop all at end of both chains input/forward.
Last edited by anav on Fri Nov 11, 2022 2:36 pm, edited 1 time in total.
 
romrider
just joined
Topic Author
Posts: 10
Joined: Fri Oct 19, 2018 9:44 pm

Re: Wireguard Slow upload

Fri Nov 11, 2022 8:55 am

Thanks.

(1) Recommend move Vlanid=1 to vlan id=101 (dont use vlan1 for data as its the default vlan MT uses in the background
Right, but I don't think that's what slows down, my ingress just with wireguard


(2) Stemming from that, dont use the bridge for DHCP or anything else, just bridging.


On br-wan it's required:
1. My ISP requires internet to be on vlan 832
2. My ISP requires DHCP requests to be marked with COS 6 else you don't get an answer. Adding a bridge filter disables fasttrack as per routeros documentation. This is the only way to add COS to DHCP as DHCP requests uses raw sockets and skip the /ip firewall filter (so can't use postrouting for that)
3. This config is the only way to achieve 2. (some config is yet missing for VOD)
4. I've tried with the vlan832 not being part of the bridge (but being a slave of the bridge instead of being slave of the physical interface) and it doesn't change anything (the config is very different) but it wouldn't work with what's yet missing (aka VOD)


(3) So adding vlan101 into the mix there are SEVEN in total.

a. why only 4 pools
b. why only 4 dhcp servers
c. why only 4 dhcp server networks
Vlan 1,10,11,12 are devices vlan with dhcp.
Vlan 254 is the vlan used to access admin loopbacks of other network devices on my lan, no dhcp
Vlan255 is the vlan to access the LTE router, no dhcp required. Could be a /30
Vlan 832 is for internet, see above. Dhcp client only.

(4) Cannot wrap my head around these errors.........
/interface bridge port
add bridge=br-wan interface=vlan832
add bridge=bridge1 interface="SW2 Ports"

First, a vlan is not an interface to be identified in interface bridge ports, its one interface per line AKA an ethernet interface or a WLAN interface.
The second implies multiple interfaces on a single line, not done. But there is no SW2 Ports interface that I see?? You are mixing up interface with interface list.
One cannot put interface lists here!!
For vlan832, see my answer to 2)

For the list in an bridge, it's possible and supported as per routeros documentation:
Starting with RouterOS v6.41 it possible to add interface lists as a bridge port and sort them.


(5) Dont understand the bridge filter item and why its not applied only to BR-WAN. Never used bridge filters so not sure.
Also not sure why it has anything to do with fasstrack in the forward chain??


See my answer to 2)

(6) THis would have to be modified too.
add bridge=bridge1 tagged=bridge1 untagged=\
ether9,ether10,ether11,ether12,ether13,ether14,ether15,ether16 vlan-ids=101
Yes


(7) So all your ethernet ports are hybrid ports???


Not sure I understand this one?


(8) Find that you have gone overboard on firewall rules such that its near impossible to see teh forest for the trees.
Much better to get rid of all the pretent I am protecting my router BS and stick to two themes.
a. default rules
b. actual required rules for required traffic
c. drop all at end of both chains input/forward.


Maybe, I still need to simplify some things but overall it's tidy.

(9) Many items are above my head in the config.....
What can I say...
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Wireguard Slow upload

Fri Nov 11, 2022 2:39 pm

Understand only that your ISP needs a special connection config, the rest IMHO is bogus and not needed, but then again my knowledge is limited.
Unable to discern your wg issue.

Who is online

Users browsing this forum: johnson73, StephenDig and 77 guests