WG to WG slow, clumsy speed

RouterOS General
1

Hi
i have two WG tunnel concentrated on one WG HUB (CHR 7.11.2)
today i tried to transfer 6MB file over TFTP between two tunnels and … i was disappointed

so, starting server is Debian
TFTP running with --blocksize=1468

on same switch, Debian → PC (1Gbps)

tftp> get wrt.bin
getting from 192.168.0.2:wrt.bin to wrt.bin [octet]
Received 6680152 bytes in 2.2 seconds [24464269 bit/s]

good


Debian → WG1 tunnel WG1 → HUB

 /tool> fetch mode=tftp address=169.254.0.30 upload=no src-path="wrt.bin" 
      status: finished
  downloaded: 6523KiB
    duration: 36s

not so good !!!


Debian → WG1 tunnel WG1 → HUB → WG2 tunnel WG2 → consumer PC

tftp> bin       
mode set to octet
tftp> get wrt.bin
getting from 169.254.0.30:wrt.bin to wrt.bin [octet]
Received 6680152 bytes in 119.5 seconds [447207 bits/sec]

it is far from good :frowning:

2sec → 36sec → 120sec :frowning:

WG is running on public IPs, no nat, port fw and similar,
every link is at least 100Mbps, fiber
All WGs are MikroTiks 7.11.2

why it is so slow ?

2

Using TFTP as speed benchmarking tool is … innovative :wink:

You really should use a well established tool, like iperf3 … run on devices with ample resources so that they are not bottleneck. As iperf3 allows to test both using UDP and TCP, this sometimes comes handy at diagnosing certain aspects of connection (e.g. large delay jitter which can affect TCP quite considerably while it doesn’t bother UDP so much). Iperf3 also allows to use multiple parallel TCP streams which can sometimes point at processing bottlenecks on routers (ROS is known to process single-connection process using same CPU core, when router has multiple not-so-fast cores the single-connection benchmarks can show pretty low results while multi-connection benchmarks show much higher results.
Also run profiler tool on router while running throughput test to identify potential processing bottlenecks.

The way you describe the problem it could be just anything.

3

I dont see any MT config, so guessing the problem is non-MT and debian related, wrong forum. :stuck_out_tongue_winking_eye:

4

Hi

no, it is NOT innovative … it is must !
do i want to use TFTP ? No !
do i need to use TFTP ? yes !
do i have choice ? No !

so i have problem with WG tunnels and speed between them
as you could see, on first step, WG1 ↔ WG1 (HUB) the speed is rapidly dropped

@anav
configs?
very basic config

hub:

/interface wireguard
add disabled=no listen-port=8000 mtu=1420 name="wg: Hub (NMS)"
/interface wireguard peers
add allowed-address=fdff:255::2/128,fd00:2::/32,169.254.0.2/32,169.254.2.0/24 comment="02 - XXXX" disabled=no endpoint-address=redacted endpoint-port=8000 interface="wg: Hub (NMS)" persistent-keepalive= 15s public-key="XXX"
add allowed-address=fdff:255::3/128,fd00:3::/32,169.254.0.3/32,169.254.3.0/24 comment="03 - YYYY" disabled=no endpoint-address=redacted endpoint-port=8000 interface="wg: Hub (NMS)" persistent-keepalive= 15s public-key="YYY"

02:

# model = RB5009UG+S+
# serial number =
/interface wireguard
add disabled=no listen-port=8000 mtu=1420 name="wg: Mgmn (NMS)"
/interface wireguard peers
add allowed-address=fdff:255::/64,169.254.0.0/27 comment=hub disabled=no endpoint-address=redacted endpoint-port=8000 interface="wg: Mgmn (NMS)" persistent-keepalive=15s public-key="zzzzzz"

03:
same as “02”

@anav,
no, Debian → PC working fast, if you read my post, it is only 2sec between Debian and other PC on same switch

but, Debian → MKT_WG → MKT_WG is already 36sec
so, how could this be related to Debian ?

again, i am NOT measuring network speed
i need to use TFTP, no, not my idea
simply, i need to transfer bin files with TFTP from central TFTP server to many endpoints which is all Mikrotiks

5

I’m not questioning your urge to use TFTP. But I do see problems when using TFTP to argue about WG performance. Low app throughput can as well be due to increased latency … yes, WG does add latency, but most probably RTT between WG endpoints (measured over “plain” intetnet) is likely already much higher than inside LAN. So when pointing at particular implementation of WG it is necessary to rule out all other potential reasons for low performance. Which you didn’t (or so it seems).

6

If that is your config no wonder it doesnt work but since you seem to know best… good luck

7

ohh, mr guru :slight_smile:

if you think it is MTU-1420
and

TFTP running with --blocksize=1468

believe me, i tried to adjust TFTP server to fit in WG 1420
same thing
and tried to lower WG MTU, but same

8

Im no mtu guru, but I would ensure that both sides of any wireguard tunnel (in the wireguard settings) have the same MTU settings.
If there is an issue at the client MT router try this additional setting on the router itself.
/ip firewall mangle
add action=change-mss chain=forward comment=“Clamp MSS to PMTU for Outgoing packets” new-mss=clamp-to-pmtu out-interface=wireguard1 passthrough=yes protocol=tcp tcp-flags=syn

9

https://wiki.mikrotik.com/wiki/Manual:CHR#CHR_Licensing

1 mbit if no licence

10

it is paid lincenced CHR

hi @anav
yes, they are same
yes, i tried clamping, but clamping is for TCP, not UDP if i remember right …

so, as i see, only solution is to have independent TFTP servers at every remote locations and keep them synced with rsync …
whatever i tried, every L3 point in path will increase TFTP rtt and at the end, it will be unusable