Community discussions

MikroTik App
 
mmotti
just joined
Topic Author
Posts: 16
Joined: Thu Nov 17, 2022 9:50 pm

WireGuard on my RB5009 doesn't seem to play nice for any provider except Mullvad?

Fri Nov 25, 2022 7:26 pm

Hi,

I've been trying to experiment with some different WireGuard VPN providers and have been having difficulty with getting things working properly.

Mullvad works exactly as expected when entering the configuration details.

Prior to SurfShark I tried to use NordLynx (hack and slash to get it working on Linux first) and whilst I could browse the web I had some weird issues with Android push notifications not working for things like Gmail and Discord (although NordLynx worked fine using exactly the same config through the WireGuard app on my phone).

More recently I have been trying to get SurfShark working with mixed results (although, again, fine with WireGuard Android app). It wouldn't even allow me to connect until adding new-mss=clamp-to-pmtu which wasn't required for Mullvad or NordLynx. I have changed nothing in my configuration since switching from Mullvad with exception of the MSS setting, IP address of the interface, DNS address and endpoint IP / public key etc.

Since changing the MSS seting I am now "connected" through SurfShark and can browse the web, however the "upload" portion appears to be broken; speed tests indicate that my upload speed is 0Mbps and if I watch the WireGuard interface during a speedtest my Tx rate sits at 6-15Kbps.

I have noticed that the only way I can get my upload speed back to normal is by adding a routing rule for the destination address of my WG provider clients subnet to use the main routing table:
dst-address=192.168.5.0/24 action=lookup-only-in-table table=main
The problem is that I'm not exactly sure what this is doing or why it's affecting (or alleviating) the upload issue, or whether this is actually instead allowing my upload traffic to bypass the WireGuard tunnel. Again, this wasn't at all necessary with Mullvad.

Surely I shouldn't need anything like this with srcnat?!

Both NordLynx and SurfShark work absolutely fine with the WireGuard Android app using identical interface / peer configs as the router, yet not with my configuration on my Mikrotik Router.

Can anybody help me with troubleshooting what exactly is going on here? I'd really appreciate it.

My current setup:
Routing rule for src-address 192.168.5.0/24 to only use wg_provider table
Routing rule dst-address to 192.168.1.0/24 to only use main table so I can get back to LAN for clients using the wg_provider routing table
Firewall rules allowing interface list containing this subnet to use wg0
srcnat masq rule for WAN interface list (wg0 is in this list)

Here's my config:
/interface bridge
add admin-mac=XXXX auto-mac=no comment=defconf name=bridge \
    vlan-filtering=yes
/interface wireguard
add listen-port=61468 mtu=1420 name=wg0
add listen-port=13231 mtu=1420 name=wg1
/interface vlan
add interface=bridge name=vlan91 vlan-id=91
add interface=bridge name=vlan92 vlan-id=92
add interface=bridge name=vlan95 vlan-id=95
add interface=bridge name=vlan100 vlan-id=100
/interface pppoe-client
add add-default-route=yes disabled=no interface=ether1 name=\
    XXXX service-name=internet user=\
    XXXX
/interface list
add name=WAN
add name=LAN
add name=WG_VPN_Provider_Clients
add name=UNTRUSTED_LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=bridge ranges=192.168.88.100-192.168.88.199
add name=vlan92 ranges=192.168.2.100-192.168.2.199
add name=vlan93 ranges=192.168.3.100-192.168.3.199
add name=vlan94 ranges=192.168.4.100-192.168.4.199
add name=vlan95 ranges=192.168.5.100-192.168.5.199
add name=vlan91 ranges=192.168.1.100-192.168.1.199
add name=rescue ranges=192.168.89.100-192.168.89.199
/ip dhcp-server
add address-pool=bridge interface=bridge name=bridge
add address-pool=vlan95 interface=vlan95 name=vlan95
add address-pool=vlan92 interface=vlan92 name=vlan92
add address-pool=vlan91 interface=vlan91 name=vlan91
add address-pool=rescue interface=ether8 name=rescue
/routing table
add fib name=wg_provider
/interface bridge port
add bridge=bridge comment=defconf interface=ether2 pvid=91
add bridge=bridge comment=defconf interface=ether3 pvid=91
add bridge=bridge comment=defconf interface=ether4 pvid=91
add bridge=bridge comment=defconf interface=ether5 pvid=91
add bridge=bridge comment=defconf interface=ether6 pvid=91
add bridge=bridge comment=defconf interface=ether7 pvid=95
add bridge=bridge comment=defconf interface=sfp-sfpplus1 pvid=91
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface bridge vlan
add bridge=bridge tagged=bridge,ether3 untagged=ether7 vlan-ids=95
add bridge=bridge tagged=bridge,ether3 vlan-ids=92
add bridge=bridge tagged=bridge untagged=\
    ether2,ether3,ether4,ether5,ether6,ether7,sfp-sfpplus1 vlan-ids=91
/interface list member
add interface=bridge list=LAN
add interface=ether1 list=WAN
add interface=XXXXPPPOEXXXX list=WAN
add interface=vlan95 list=WG_VPN_Provider_Clients
add interface=wg0 list=WAN
add interface=vlan91 list=LAN
add interface=vlan92 list=UNTRUSTED_LAN
add interface=wg1 list=LAN
add interface=ether8 list=LAN
/interface wireguard peers
add allowed-address=0.0.0.0/0,::/0 endpoint-address=XXXXENDPOINTXXXX \
    endpoint-port=51820 interface=wg0 public-key=\
    "XXXXPUBKEYXXXX"
add allowed-address=192.168.10.2/32 interface=wg1 public-key=\
    "XXXXPUBKEYXXXX"
/ip address
add address=192.168.88.1/24 comment="bridge default" interface=bridge \
    network=192.168.88.0
add address=192.168.5.1/24 interface=vlan95 network=192.168.5.0
add address=192.168.2.1/24 interface=vlan92 network=192.168.2.0
add address=192.168.10.1/24 interface=wg1 network=192.168.10.0
add address=xxxx/16 interface=wg0 network=xxxx
add address=192.168.0.1/24 interface=ether1 network=192.168.0.0
add address=192.168.1.1/24 interface=vlan91 network=192.168.1.0
add address=192.168.89.1/24 comment="rescue port" interface=ether8 network=\
    192.168.89.0
/ip cloud
set ddns-enabled=yes update-time=no
/ip dhcp-client
add comment=defconf disabled=yes interface=ether1
/ip dhcp-server network
add address=192.168.1.0/24 comment=vlan91 dns-server=1.1.1.1,1.0.0.1 gateway=\
    192.168.1.1 netmask=24
add address=192.168.2.0/24 comment=vlan92 dns-server=1.1.1.1,1.0.0.1 gateway=\
    192.168.2.1
add address=192.168.5.0/24 comment=vlan95 dns-server=\
    XXXX,XXXX gateway=192.168.5.1 netmask=24
add address=192.168.88.0/24 comment=bridge dns-server=1.1.1.1,1.0.0.1 \
    gateway=192.168.88.1
add address=192.168.89.0/24 comment=rescue dns-server=1.1.1.1,1.0.0.1 \
    gateway=192.168.89.1 netmask=24
/ip dns
set allow-remote-requests=yes servers=1.1.1.1,1.0.0.1
/ip dns static
add address=192.168.1.1 comment=defconf name=router.lan
/ip firewall address-list
add address=192.168.1.10 list="Main PC"
/ip firewall filter
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment=\
    "defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=accept chain=input comment="allow WireGuard (Home)" dst-port=13231 \
    protocol=udp
add action=accept chain=input comment="Allow input chain from LAN interfaces" \
    in-interface-list=LAN
add action=drop chain=input comment="Drop remaining traffic on input chain"
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
    ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
    ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-state=established,related hw-offload=yes
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new in-interface-list=WAN
add action=accept chain=forward comment=\
    "Allow LAN interfaces to forward to any interface list" \
    in-interface-list=LAN out-interface-list=all
add action=accept chain=forward comment=\
    "Allow internet traffic for untrusted LAN interfaces" in-interface-list=\
    UNTRUSTED_LAN out-interface-list=WAN
add action=accept chain=forward comment=\
    "Allow specified clients out through the VPN Provider's Wireguard tunnel" \
    in-interface-list=WG_VPN_Provider_Clients out-interface=wg0
add action=accept chain=forward comment=\
    "Allow Steam Remote Play UDP from vlan95" dst-address-list="Main PC" \
    dst-port=27031-27036 in-interface=vlan95 protocol=udp
add action=accept chain=forward comment=\
    "Allow Steam Remote Play TCP from vlan95" dst-address-list="Main PC" \
    dst-port=27036-27037 in-interface=vlan95 protocol=tcp
add action=drop chain=forward comment=\
    "Drop remaining traffic on the forward chain"
/ip firewall mangle
add action=change-mss chain=forward comment="SurfShark specific MSS change" \
    new-mss=clamp-to-pmtu out-interface=wg0 passthrough=yes protocol=tcp \
    tcp-flags=syn
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" \
    ipsec-policy=out,none out-interface-list=WAN
/ip route
add dst-address=0.0.0.0/0 gateway=wg0 routing-table=wg_provider
/ipv6 address
add address=XXXX advertise=no interface=wg0
/ipv6 firewall address-list
add address=::/128 comment="defconf: unspecified address" list=bad_ipv6
add address=::1/128 comment="defconf: lo" list=bad_ipv6
add address=fec0::/10 comment="defconf: site-local" list=bad_ipv6
add address=::ffff:0.0.0.0/96 comment="defconf: ipv4-mapped" list=bad_ipv6
add address=::/96 comment="defconf: ipv4 compat" list=bad_ipv6
add address=100::/64 comment="defconf: discard only " list=bad_ipv6
add address=2001:db8::/32 comment="defconf: documentation" list=bad_ipv6
add address=2001:10::/28 comment="defconf: ORCHID" list=bad_ipv6
add address=3ffe::/16 comment="defconf: 6bone" list=bad_ipv6
/ipv6 firewall filter
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMPv6" protocol=\
    icmpv6
add action=accept chain=input comment="defconf: accept UDP traceroute" port=\
    33434-33534 protocol=udp
add action=accept chain=input comment=\
    "defconf: accept DHCPv6-Client prefix delegation." dst-port=546 protocol=\
    udp src-address=fe80::/10
add action=accept chain=input comment="defconf: accept IKE" dst-port=500,4500 \
    protocol=udp
add action=accept chain=input comment="defconf: accept ipsec AH" protocol=\
    ipsec-ah
add action=accept chain=input comment="defconf: accept ipsec ESP" protocol=\
    ipsec-esp
add action=accept chain=input comment=\
    "defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=input comment=\
    "defconf: drop everything else not coming from LAN" in-interface-list=\
    !LAN
add action=accept chain=forward comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop packets with bad src ipv6" src-address-list=bad_ipv6
add action=drop chain=forward comment=\
    "defconf: drop packets with bad dst ipv6" dst-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: rfc4890 drop hop-limit=1" \
    hop-limit=equal:1 protocol=icmpv6
add action=accept chain=forward comment="defconf: accept ICMPv6" protocol=\
    icmpv6
add action=accept chain=forward comment="defconf: accept HIP" protocol=139
add action=accept chain=forward comment="defconf: accept IKE" dst-port=\
    500,4500 protocol=udp
add action=accept chain=forward comment="defconf: accept ipsec AH" protocol=\
    ipsec-ah
add action=accept chain=forward comment="defconf: accept ipsec ESP" protocol=\
    ipsec-esp
add action=accept chain=forward comment=\
    "defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=forward comment=\
    "defconf: drop everything else not coming from LAN" in-interface-list=\
    !LAN
/routing rule
add action=lookup-only-in-table disabled=no dst-address=192.168.1.0/24 table=\
    main
add action=lookup-only-in-table comment="This rule shouldn't be here\?" \
    dst-address=192.168.5.0/24 table=main
add action=lookup-only-in-table disabled=no src-address=192.168.5.0/24 table=\
    wg_provider
/system clock
set time-zone-name=Europe/London
/system ntp client
set enabled=yes
/system ntp client servers
add address=0.uk.pool.ntp.org
add address=1.uk.pool.ntp.org
add address=2.uk.pool.ntp.org
add address=3.uk.pool.ntp.org
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN
Last edited by mmotti on Mon Nov 28, 2022 11:21 pm, edited 3 times in total.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: WireGuard on my RB5009 doesn't seem to play nice for any provider except Mullvad?

Fri Nov 25, 2022 9:09 pm

Once you mix apples and oranges (use bridge for DHCP instead of another vlan) the config gets easliy confusing for me, so I am going to change it, so then I can assess the rest of the config without feeling queasy................

Not that it will make a difference but I like to keep things to what I know works.
add address=10.14.0.2/24 interface=wg0 network=10.14.0.0 (you dont need 16 )..


If this is going out the wg why not put... speaking about dhcp-server network settings for vlan95
add address=192.168.5.0/24 comment=vlan95 dns-server=10.14.0.1 \ { or if that doesnt work, just 192.168.5.1 }
gateway=192.168.5.1 netmask=24


Purpose of this???? on the input chain?? ( assuming you must want to remote in on Wireguard directly from somewhere ?? ) Ie for the other wg interface the wireguard is acting as the recipient of the initial handshake?
add action=accept chain=input comment="allow WireGuard (Home)" dst-port=13231 \
protocol=udp


What is the intent of these rules.??? Assuming you have users on vlan95 that want to go where ? ---------> okay to a device on 192.168.1.0/24
THis is okay, but it will mean potentially some consideration when it comes to routing later on. EDIT> I see you have done that well!!
add action=accept chain=forward comment=\
"Allow Steam Remote Play UDP from vlan95" dst-address-list="Main PC" \
dst-port=27031-27036 in-interface=vlan95 protocol=udp
add action=accept chain=forward comment=\
"Allow Steam Remote Play TCP from vlan95" dst-address-list="Main PC" \
dst-port=27036-27037 in-interface=vlan95 protocol=tcp


Okay so you need a table, a route and a routing rule..............
check , check and check.
Get rid of the middle rule!!!

/routing rule
add action=lookup-only-in-table disabled=no dst-address=192.168.1.0/24 table=\
main
add action=lookup-only-in-table comment="This rule shouldn't be here\?" \
dst-address=192.168.5.0/24 table=main

add action=lookup-only-in-table disabled=no src-address=192.168.5.0/24 table=\
wg_provider
 
mmotti
just joined
Topic Author
Posts: 16
Joined: Thu Nov 17, 2022 9:50 pm

Re: WireGuard on my RB5009 doesn't seem to play nice for any provider except Mullvad?

Fri Nov 25, 2022 9:34 pm

Once you mix apples and oranges (use bridge for DHCP instead of another vlan) the config gets easliy confusing for me, so I am going to change it, so then I can assess the rest of the config without feeling queasy................
There's still a DHCP server attached to the Bridge as to be honest I wasn't exactly sure of the consequences of removing it. All ports (with exception of ether8) are assigned to a VLAN which provides DHCP and ether8 is just my "rescue" port with it's own DHCP. I guess I could remove the Bridge DHCP? All of my PVIDs are mostly set to VLAN91
Not that it will make a difference but I like to keep things to what I know works.
add address=10.14.0.2/24 interface=wg0 network=10.14.0.0 (you dont need 16 )..
OK - Will change :) /16 is just the config I was given by SurfShark.
If this is going out the wg why not put... speaking about dhcp-server network settings for vlan95
add address=192.168.5.0/24 comment=vlan95 dns-server=10.14.0.1 \ { or if that doesnt work, just 192.168.5.1 }
gateway=192.168.5.1 netmask=24
I tried both 10.14.0.1 and 192.168.5.1 but neither worked sadly - No internet until using the DNS servers they provided in the config file.
Purpose of this???? on the input chain?? ( assuming you must want to remote in on Wireguard directly from somewhere ?? ) Ie for the other wg interface the wireguard is acting as the recipient of the initial handshake?
add action=accept chain=input comment="allow WireGuard (Home)" dst-port=13231 \
protocol=udp
This is for wg1 which allows me to remote in externally from my phone to my router. I use wg0 for the VPN provider WireGuard :)
What is the intent of these rules.??? Assuming you have users on vlan95 that want to go where ? ---------> okay to a device on 192.168.1.0/24
THis is okay, but it will mean potentially some consideration when it comes to routing later on. EDIT> I see you have done that well!!
add action=accept chain=forward comment=\
"Allow Steam Remote Play UDP from vlan95" dst-address-list="Main PC" \
dst-port=27031-27036 in-interface=vlan95 protocol=udp
add action=accept chain=forward comment=\
"Allow Steam Remote Play TCP from vlan95" dst-address-list="Main PC" \
dst-port=27036-27037 in-interface=vlan95 protocol=tcp
So this is basically there because I wanted to isolate the 192.168.5.1 (VLAN95 -> wg0 --> external VPN provider) as best as I could, however I wanted my TV (VLAN95) to be able to still go out for internet access through the vpn provider but be accessible VIA Steam Link on my VLAN91 network or more specifically to one PC through specific ports.
Okay so you need a table, a route and a routing rule..............
check , check and check.
Get rid of the middle rule!!!
/routing rule
add action=lookup-only-in-table disabled=no dst-address=192.168.1.0/24 table=\
main
add action=lookup-only-in-table comment="This rule shouldn't be here\?" \
dst-address=192.168.5.0/24 table=main

add action=lookup-only-in-table disabled=no src-address=192.168.5.0/24 table=\
wg_provider
Yep! The middle rule is, or rather was, only there due to troubleshooting. It's literally the only way that my upload speed is more than 0-15Kbps. I can't figure out why :(
I genuinely have no idea where I'm going wrong
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: WireGuard on my RB5009 doesn't seem to play nice for any provider except Mullvad?

Sat Nov 26, 2022 12:18 am

Hi there,
No need to change bridge configuration etc. its just what I prefer to keep things consistent.
It should not affect your WG connection either way.

MY BAD also on DNS settings, I didnt realize the ones you had included came from the surfshark provider, so put those ones back in!!
I dont see any issues looking at it, you have the wg as part of WAN interface and the standard sourcenat rule so that should work........

Did by anychance you ever have any beta versions of firmware 7 on the router.
Sometimes one has to net install the latest version and start from a fresh clean install..........
 
mmotti
just joined
Topic Author
Posts: 16
Joined: Thu Nov 17, 2022 9:50 pm

Re: WireGuard on my RB5009 doesn't seem to play nice for any provider except Mullvad?

Sat Nov 26, 2022 1:52 am

Ahh okay :) tbf I could probably disable it as I don't use PVID 1 for any of the ports at present.

No problem! It's strange isn't it. For all intents and purposes it SHOULD work... It just.. doesn't. At least not properly.

I've only had the router around a week and it's my first Mikrotik device. I did try to install the latest beta today to see if it resolved the issue but reverted back as it didn't make any difference.

I'm not sure a netinstall would make too much difference tbh but might be worth a go. I tried it previously though and it absolutely refused to send any files to the router from my pc :(

Edit:
OK - Just successfully finished my first Netinstall with a PCI-e network adapter I found laying around as it flat-out refused to work with my onboard NICs. Sad news is the problem remains.

Mullvad:
Image
Surshark:
Image
Last edited by mmotti on Sat Nov 26, 2022 4:04 pm, edited 1 time in total.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: WireGuard on my RB5009 doesn't seem to play nice for any provider except Mullvad?

Sat Nov 26, 2022 6:25 am

Please post your latest config and will have a look tomorrow..............
 
mmotti
just joined
Topic Author
Posts: 16
Joined: Thu Nov 17, 2022 9:50 pm

Re: WireGuard on my RB5009 doesn't seem to play nice for any provider except Mullvad?

Sat Nov 26, 2022 2:57 pm

Sure, here's the config:
/interface bridge
add admin-mac=XXXX auto-mac=no comment=defconf name=bridge \
    vlan-filtering=yes
/interface wireguard
add listen-port=61468 mtu=1420 name=wg0
add listen-port=13231 mtu=1420 name=wg1
/interface vlan
add interface=bridge name=vlan91 vlan-id=91
add interface=bridge name=vlan92 vlan-id=92
add interface=bridge name=vlan95 vlan-id=95
add interface=bridge name=vlan100 vlan-id=100
/interface pppoe-client
add add-default-route=yes disabled=no interface=ether1 name=\
    XXXXPPPOEXXXX service-name=internet user=\
    XXXXPPPOEXXXX
/interface list
add name=WAN
add name=LAN
add name=WG_VPN_Provider_Clients
add name=UNTRUSTED_LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip dhcp-server option
add code=6 name="Mullvad DNS (Adblock)" value="'100.64.0.1'"
/ip pool
add name=bridge ranges=192.168.88.100-192.168.88.199
add name=vlan92 ranges=192.168.2.100-192.168.2.199
add name=vlan93 ranges=192.168.3.100-192.168.3.199
add name=vlan94 ranges=192.168.4.100-192.168.4.199
add name=vlan95 ranges=192.168.5.100-192.168.5.199
add name=vlan91 ranges=192.168.1.100-192.168.1.199
add name=rescue ranges=192.168.89.100-192.168.89.199
/ip dhcp-server
add address-pool=bridge disabled=yes interface=bridge name=bridge
add address-pool=vlan95 interface=vlan95 name=vlan95
add address-pool=vlan92 interface=vlan92 name=vlan92
add address-pool=vlan91 interface=vlan91 name=vlan91
add address-pool=rescue interface=ether8 name=rescue
/routing table
add fib name=wg_provider
/interface bridge port
add bridge=bridge comment=defconf interface=ether2 pvid=91
add bridge=bridge comment=defconf interface=ether3 pvid=91
add bridge=bridge comment=defconf interface=ether4 pvid=91
add bridge=bridge comment=defconf interface=ether5 pvid=91
add bridge=bridge comment=defconf interface=ether6 pvid=91
add bridge=bridge comment=defconf interface=ether7 pvid=95
add bridge=bridge comment=defconf interface=sfp-sfpplus1 pvid=91
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface bridge vlan
add bridge=bridge tagged=bridge,ether3 untagged=ether7 vlan-ids=95
add bridge=bridge tagged=bridge,ether3 vlan-ids=92
add bridge=bridge tagged=bridge untagged=\
    ether2,ether3,ether4,ether5,ether6,ether7,sfp-sfpplus1 vlan-ids=91
/interface list member
add interface=bridge list=LAN
add interface=ether1 list=WAN
add interface=dsl000724578@broadband.vodafone.co.uk list=WAN
add interface=vlan95 list=WG_VPN_Provider_Clients
add interface=wg0 list=WAN
add interface=vlan91 list=LAN
add interface=vlan92 list=UNTRUSTED_LAN
add interface=wg1 list=LAN
add interface=ether8 list=LAN
/interface wireguard peers
add allowed-address=0.0.0.0/0 endpoint-address=uk-lon.prod.surfshark.com \
    endpoint-port=51820 interface=wg0 public-key=\
    "XXXX"
add allowed-address=192.168.10.2/32 interface=wg1 public-key=\
    "XXXX"
/ip address
add address=192.168.88.1/24 comment="bridge default" interface=bridge \
    network=192.168.88.0
add address=192.168.5.1/24 interface=vlan95 network=192.168.5.0
add address=192.168.2.1/24 interface=vlan92 network=192.168.2.0
add address=192.168.10.1/24 interface=wg1 network=192.168.10.0
add address=xxxx/24 interface=wg0 network=xxxx
add address=192.168.0.1/24 interface=ether1 network=192.168.0.0
add address=192.168.1.1/24 interface=vlan91 network=192.168.1.0
add address=192.168.89.1/24 comment="rescue port" interface=ether8 network=\
    192.168.89.0
/ip cloud
set ddns-enabled=yes update-time=no
/ip dhcp-client
add comment=defconf disabled=yes interface=ether1
/ip dhcp-server network
add address=192.168.1.0/24 comment=vlan91 dns-server=1.1.1.1,1.0.0.1 gateway=\
    192.168.1.1 netmask=24
add address=192.168.2.0/24 comment=vlan92 dns-server=1.1.1.1,1.0.0.1 gateway=\
    192.168.2.1
add address=192.168.5.0/24 comment=vlan95 dns-server=\
    XXXX,XXXX gateway=192.168.5.1 netmask=24
add address=192.168.88.0/24 comment=bridge dns-server=192.168.88.1 gateway=\
    192.168.88.1 netmask=24
add address=192.168.89.0/24 comment=rescue dns-server=1.1.1.1,1.0.0.1 \
    gateway=192.168.89.1 netmask=24
/ip dns
set allow-remote-requests=yes servers=1.1.1.1,1.0.0.1
/ip dns static
add address=192.168.1.1 comment=defconf name=router.lan
/ip firewall address-list
add address=192.168.1.10 list="Main PC"
/ip firewall filter
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment=\
    "defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=accept chain=input comment="allow WireGuard (Home)" dst-port=13231 \
    protocol=udp
add action=accept chain=input comment="Allow input chain from LAN interfaces" \
    in-interface-list=LAN
add action=drop chain=input comment="Drop remaining traffic on input chain"
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
    ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
    ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-state=established,related hw-offload=yes
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new in-interface-list=WAN
add action=accept chain=forward comment=\
    "Allow LAN interfaces to forward to any interface list" \
    in-interface-list=LAN out-interface-list=all
add action=accept chain=forward comment=\
    "Allow internet traffic for untrusted LAN interfaces" in-interface-list=\
    UNTRUSTED_LAN out-interface-list=WAN
add action=accept chain=forward comment=\
    "Allow specified clients out through the VPN Provider's Wireguard tunnel" \
    in-interface-list=WG_VPN_Provider_Clients out-interface=wg0
add action=accept chain=forward comment=\
    "Allow Steam Remote Play UDP from vlan95" dst-address-list="Main PC" \
    dst-port=27031-27036 in-interface=vlan95 protocol=udp
add action=accept chain=forward comment=\
    "Allow Steam Remote Play TCP from vlan95" dst-address-list="Main PC" \
    dst-port=27036-27037 in-interface=vlan95 protocol=tcp
add action=drop chain=forward comment=\
    "Drop remaining traffic on the forward chain"
/ip firewall mangle
add action=change-mss chain=forward comment="SurfShark specific MSS change" \
    new-mss=clamp-to-pmtu out-interface=wg0 passthrough=yes protocol=tcp \
    tcp-flags=syn
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" \
    ipsec-policy=out,none out-interface-list=WAN
/ip route
add dst-address=0.0.0.0/0 gateway=wg0 routing-table=wg_provider
/ipv6 firewall address-list
add address=::/128 comment="defconf: unspecified address" list=bad_ipv6
add address=::1/128 comment="defconf: lo" list=bad_ipv6
add address=fec0::/10 comment="defconf: site-local" list=bad_ipv6
add address=::ffff:0.0.0.0/96 comment="defconf: ipv4-mapped" list=bad_ipv6
add address=::/96 comment="defconf: ipv4 compat" list=bad_ipv6
add address=100::/64 comment="defconf: discard only " list=bad_ipv6
add address=2001:db8::/32 comment="defconf: documentation" list=bad_ipv6
add address=2001:10::/28 comment="defconf: ORCHID" list=bad_ipv6
add address=3ffe::/16 comment="defconf: 6bone" list=bad_ipv6
/ipv6 firewall filter
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMPv6" protocol=\
    icmpv6
add action=accept chain=input comment="defconf: accept UDP traceroute" port=\
    33434-33534 protocol=udp
add action=accept chain=input comment=\
    "defconf: accept DHCPv6-Client prefix delegation." dst-port=546 protocol=\
    udp src-address=fe80::/10
add action=accept chain=input comment="defconf: accept IKE" dst-port=500,4500 \
    protocol=udp
add action=accept chain=input comment="defconf: accept ipsec AH" protocol=\
    ipsec-ah
add action=accept chain=input comment="defconf: accept ipsec ESP" protocol=\
    ipsec-esp
add action=accept chain=input comment=\
    "defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=input comment=\
    "defconf: drop everything else not coming from LAN" in-interface-list=\
    !LAN
add action=accept chain=forward comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop packets with bad src ipv6" src-address-list=bad_ipv6
add action=drop chain=forward comment=\
    "defconf: drop packets with bad dst ipv6" dst-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: rfc4890 drop hop-limit=1" \
    hop-limit=equal:1 protocol=icmpv6
add action=accept chain=forward comment="defconf: accept ICMPv6" protocol=\
    icmpv6
add action=accept chain=forward comment="defconf: accept HIP" protocol=139
add action=accept chain=forward comment="defconf: accept IKE" dst-port=\
    500,4500 protocol=udp
add action=accept chain=forward comment="defconf: accept ipsec AH" protocol=\
    ipsec-ah
add action=accept chain=forward comment="defconf: accept ipsec ESP" protocol=\
    ipsec-esp
add action=accept chain=forward comment=\
    "defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=forward comment=\
    "defconf: drop everything else not coming from LAN" in-interface-list=\
    !LAN
/routing rule
add action=lookup-only-in-table disabled=no dst-address=192.168.1.0/24 table=\
    main
add action=lookup-only-in-table disabled=no src-address=192.168.5.0/24 table=\
    wg_provider
/system clock
set time-zone-name=Europe/London
/system ntp client
set enabled=yes
/system ntp client servers
add address=0.uk.pool.ntp.org
add address=1.uk.pool.ntp.org
add address=2.uk.pool.ntp.org
add address=3.uk.pool.ntp.org
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN
I think the only real difference between the originally posted config was that I disabled the bridge DHCP server however have not removed the assigned IP addresses etc yet.

The only differences between the Mullvad and Surfshark config are as follows: wg0 IP Address, VLAN95 DNS Servers (Mullvad (looks to be private address) --> SurfShark (looks to be public address)), endpoint address/public key for wg0 peer and the SurfShark requiring the new-mss=clamp-to-pmtu . Otherwise, everything is identical.
Last edited by mmotti on Mon Nov 28, 2022 11:21 pm, edited 2 times in total.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: WireGuard on my RB5009 doesn't seem to play nice for any provider except Mullvad?

Sat Nov 26, 2022 4:40 pm

(1) When you say mullvad seems to be private and surfkshark public, not sure what you mean??

(2) Do they both give you a private key to use at your end ( so that the public key generated by it will already be in their possession), or do you have to send them a public key?

The only difference between the two should be.

a. private key supplied by them (to put on Wg0 which produces a public key they will then already have)
b. public key supplied by them (which you need to put on your peer settings for the third party VPN).
c. the endpoint
d. the endpoint port
 
mmotti
just joined
Topic Author
Posts: 16
Joined: Thu Nov 17, 2022 9:50 pm

Re: WireGuard on my RB5009 doesn't seem to play nice for any provider except Mullvad?

Sat Nov 26, 2022 5:00 pm

(1) When you say mullvad seems to be private and surfkshark public, not sure what you mean??
Sorry - I meant in terms of DNS address; 162.252.X.X/149.154.X.X (SurfShark) vs 10.64.0.1 (Mullvad)
(2) Do they both give you a private key to use at your end ( so that the public key generated by it will already be in their possession), or do you have to send them a public key?
So I use the same private key for both providers (of my wg0 interface) to generate key pairs. I can't remember exactly how I set it up but both Mullvad and SurfShark allow you to basically say "I already have a key" and they now have the public key of my wg0 interface as a managed WireGuard key / key pair (I can see my wg0 public key listed in both of Mullvad/SurfShark configuration area of their respective sites).
The only difference between the two should be.

a. private key supplied by them (to put on Wg0 which produces a public key they will then already have)
b. public key supplied by them (which you need to put on your peer settings for the third party VPN).
c. the endpoint
d. the endpoint port
As above, I supplied existing private or public key (can't remember which) to them instead of using a newly generated key pair and I can see my public key in both systems.

Otherwise yes all I change is what you've listed above in addition the the DNS server and interface address.

Also I don't run them at the same time. I have a router config for SurfShark and a separate one for Mullvad and I restore between the two. So wg0 is only ever active for one provider at a time.
Last edited by mmotti on Mon Nov 28, 2022 11:22 pm, edited 1 time in total.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: WireGuard on my RB5009 doesn't seem to play nice for any provider except Mullvad?

Sat Nov 26, 2022 5:05 pm

Okay so they actually asked for information from you, ( the public key your WG0 generated). Thats one way to do it.
The other way is when they dont ask for any information and just tell you which private key to use.

So for the one that doesnt work, you tried for DNS setting.
add address=192.168.5.0/24 comment=vlan95 dns-server=\
162.252.172.57,149.154.159.92 gateway=192.168.5.1 netmask=24

a. supplied DNS entries
b. wireguard gateway IP 10.0.14.1 ? or 10.0.14.2 ?
c. 1.1.1.1.

And nothing worked.

You tried setting DNS at your end to 1500? for wg0

You tried mss clamping??
 
mmotti
just joined
Topic Author
Posts: 16
Joined: Thu Nov 17, 2022 9:50 pm

Re: WireGuard on my RB5009 doesn't seem to play nice for any provider except Mullvad?

Sat Nov 26, 2022 5:10 pm

So for the one that doesnt work, you tried for DNS setting.

a. supplied DNS entries
b. wireguard gateway IP
c. 1.1.1.1.
Correct
You tried setting DNS at your end to 1500?
Sorry, would you mind elaborating on this one? I'm not exactly sure what you mean?
You tried mss clamping??
I enable the following when I switch to trying to use SurfShark (disabled currently as not necessary for Mullvad):
 3 X  ;;; SurfShark specific MSS change
      chain=forward action=change-mss new-mss=clamp-to-pmtu passthrough=yes 
      tcp-flags=syn protocol=tcp out-interface=wg0 log=no log-prefix=""
Note that I'm unable to use SurfShark AT ALL without this ^
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: WireGuard on my RB5009 doesn't seem to play nice for any provider except Mullvad?

Sat Nov 26, 2022 5:18 pm

/interface wireguard
add listen-port=xxxxx mtu=1420 name=my-wireguard

Okay, first
a. change wg0 mtu setting to 1500
b. disable the clamping rule.

THEN second try
a. change MTU to 1500
b. enable the clamping rule.
 
mmotti
just joined
Topic Author
Posts: 16
Joined: Thu Nov 17, 2022 9:50 pm

Re: WireGuard on my RB5009 doesn't seem to play nice for any provider except Mullvad?

Sat Nov 26, 2022 5:48 pm

OK I now have both WG tunnels active at the same time and a separate VLAN / SSID to test with so I don't have to keep restoring between router backups.

I changed the mangle rule slightly (added src address of new VLAN and new WG tunnel interface):
 3    ;;; SurfShark specific MSS change
      chain=forward action=change-mss new-mss=clamp-to-pmtu passthrough=yes 
      tcp-flags=syn protocol=tcp src-address=192.168.6.0/24 out-interface=wg2
      log=no log-prefix="" 
I hope this still works as intended.

Ran another speed test and the problem is immediately evident again which at the very least shows me I've got things up and running exactly the same way, just separated into a different VLAN, subnet, tunnel, key pairings etc.
Okay, first
a. change wg0 mtu setting to 1500
b. disable the clamping rule.
This results in a loss of internet connection. Trying to connect to speedtest.net and the first thing I saw was "this site does not support HTTPS" and trying to proceed anyway is infinite spinning wheel of death. Can't get to Google or anything else either.
THEN second try
a. change MTU to 1500
b. enable the clamping rule.
No internet once again.

Setting back to 1420 with clamping rule on returns the internet access.

Edit:
Not sure if relevant but my MTU to my ISP gateway is usually automatically set to 1492 as I use PPPoE over VDSL. I think 1420 should still leave room for the WireGuard header though? My ISP also only supports IPv4.

I double checked my MTU before fragmentation:
  • Not connected VIA WireGuard app on my PC: 1464 + 28 = 1492
  • Connected VIA WireGuard app on my PC: 1392 + 28 = 1420 (which is also what the WG app has set the MTU to).

Connecting directly through the WireGuard app on my PC or Android phone also allows speed tests to run properly.

The only place it doesn't seem to work is on the Mikrotik.
 
mmotti
just joined
Topic Author
Posts: 16
Joined: Thu Nov 17, 2022 9:50 pm

Re: WireGuard on my RB5009 doesn't seem to play nice for any provider except Mullvad?

Mon Nov 28, 2022 3:25 am

OK - After further investigation with a new provider (IVPN) I found something rather interesting.

My existing mangle rule specifically for the subnet that goes through IVPN:
chain=forward action=change-mss new-mss=clamp-to-pmtu passthrough=yes 
tcp-flags=syn protocol=tcp src-address=192.168.7.0/24 out-interface=wg3

With this particular provider and with mangle disabled I can access sites like Google and fast.com, but I can't access speedtest.net (sits with a spinning wheel of death - think I saw TLS handshake waiting). I couldn't access anything without mangle enabled with SurfShark. Enabling the mangle rule on my IVPN subet allowed me to access speedtest although I think it may load a little slower than normal.

I ran several speedtests on fast.com and speedtest.net, and the really interesting thing is that I get consistent upload speeds on fast.com, and consistent 0.0mbps through speedtest.

fast.com with mangle disabled: ** Can access without mangle
68/15, 67/19, 69/19
fast.com with mangle enabled: ** Can access without mangle
67/15, 67/19, 67/19
speedtest.net with mangle enabled ** Can't access without mangle
71.66/0.00, 71.64/0.00, 71.60/0.00

What this seems to suggest to me is:
  • If I could access the website without the need for the mangle rule, it seems to works as expected and my upload speed is fine
  • If I could not access the site prior to the mangle rule, enabling said rule as I have it doesn't seem to cover the "upload" portion of the traffic. Essentially it's dead. Download is fine.

Anyway, I discovered something else too. If I disregard my previous mangle/mss rule and use:
3    ;;; WireGuard Ext. MSS Change - OUT
      chain=forward action=change-mss new-mss=1380 passthrough=yes tcp-flags=syn protocol=tcp out-interface-list=WG_EXT_INT tcp-mss=!0-1380 

 4    ;;; WireGuard Ext. MSS Change - IN
      chain=forward action=change-mss new-mss=1380 passthrough=yes tcp-flags=syn protocol=tcp in-interface-list=WG_EXT_INT tcp-mss=!0-1380 
 

Having the mangle on both input AND output fixes my upload which I guess makes sense. If I use clamp-to-pmtu (only tested without tcp-mss=!0-1380) upload seems spotty still and will sometimes work, others not and will often be slower. Browsing also feels significantly slower.

So I guess my questions are:
1) Why?!
2) What is the proper fix for this? This feels very much like a bodge and I can't wrap my head around why all of this is required for SurfShark and IVPN, yet Mullvad just sails through without issue.
3) If this is the correct way to handle whatever is going on here, why do I need to explicitly specify an mss change for input and output? Everywhere I've come across this "solution" people seem to have only ever had to apply it to the out-interface?
 
Marino
Frequent Visitor
Frequent Visitor
Posts: 65
Joined: Sun Jun 14, 2015 7:26 pm

Re: WireGuard on my RB5009 doesn't seem to play nice for any provider except Mullvad?

Mon Feb 27, 2023 9:10 pm

add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
connection-state=established,related hw-offload=yes
Disable fasttrack in the firewall and see if your performance is better. I need to disable fasttrack to get Surfshark at speed.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: WireGuard on my RB5009 doesn't seem to play nice for any provider except Mullvad?

Mon Feb 27, 2023 10:28 pm

The only reason to disable fastrack is if one is using mangling in general, but not usually for just clamping but its worth a try............

Who is online

Users browsing this forum: rapix61, scoobyn8 and 45 guests