ProtonVPN Wiregaurd - no handshake

RouterOS General
1

Trying to set-up connection to ProtonVPN to route internet traffic. Am by no means an expert but have successfully set-up a site-to-site VPN (worked for years, now disabled) and device VPN connections which work just fine.

The guide on the Proton help site does in a couple of points (FW, routing) not make sense (to me), hence I tried to adapt my own previously working s2s configuration.

Now I have everything set-up except the routing but am unable to get the handshake from Proton working:

wguard-proton_ca160: [Proton VPN - CA#160] nLx6LJgXWOtnlKV65ruYMGjiw4DERImZGCpN3lolm1I=: Handshake for peer did not complete after 5 seconds, retrying (try 2)

If anybody could have a look at my configuration and point me into the direction of what I need to change….

Current configuration:

# 2025-11-01 12:36:03 by RouterOS 7.20.2
# software id = UJX4-HZLH
#
# model = RB5009UPr+S+
# serial number = XXXXXXXXX
/interface bridge
add name=bridge-docker port-cost-mode=short
add name=bridge-local port-cost-mode=short
/interface ethernet
set [ find default-name=ether1 ] poe-out=off
set [ find default-name=ether2 ] poe-out=off
set [ find default-name=ether4 ] advertise=10M-baseT-half,10M-baseT-full,100M-baseT-half,100M-baseT-full poe-out=off
set [ find default-name=ether6 ] poe-out=off
set [ find default-name=ether7 ] name=ether7-admin
set [ find default-name=ether8 ] name=ether8-wan poe-out=off
set [ find default-name=sfp-sfpplus1 ] disabled=yes
/interface ovpn-client
add auth=md5 cipher=aes128-cbc connect-to=ddns-1156rbr.asuscomm.com disabled=yes mac-address=02:07:84:F4:72:D5 name=ovpn-1156rbr protocol=udp user=apmo-administrator
/interface veth
add address=10.0.5.2/24 dhcp=no gateway=10.0.5.1 gateway6="" mac-address=40:74:E8:33:2B:78 name=veth-docker
/interface wireguard
add comment="Proton VPM - CA#160" listen-port=13231 mtu=1420 name=wguard-proton_ca160
add comment=XXXXXXXXXXXX.sn.mynetname.net:13232 listen-port=13232 mtu=1420 name=wguard-rw
add disabled=yes listen-port=13231 mtu=1420 name=wguard-s2s
/container mounts
add dst=/etc/pihole name=etc_pi-hole src=/pi-hole/etc_pi-hole
add dst=/etc/dnsmasq.d name=dnsmasq_pi-hole src=/pi-hole/dnsmasq_pi-hole
/disk
add parent=usb1 partition-number=1 partition-offset=65536 partition-size=4000000000 slot=swap swap=yes type=partition
add parent=usb1 partition-number=2 partition-offset=4000120832 partition-size=100000000000 slot=pi-hole smb-sharing=yes smb-user=pi-hole type=partition
/interface list
add name=listBridge
add name=iflist-LAN
add name=iflist-WAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=dhcp_pool0 ranges=10.0.4.100-10.0.4.199
/ip dhcp-server
add address-pool=dhcp_pool0 interface=bridge-local lease-time=10m name=dhcp-local
/ip smb users
set [ find default=yes ] disabled=yes
/user group
add name=DNSSync policy=ftp,read,write,test,!local,!telnet,!ssh,!reboot,!policy,!winbox,!password,!web,!sniff,!sensitive,!api,!romon,!rest-api
/container
add auto-restart-interval=10s envlists=pi-hole_envs interface=veth-docker logging=yes mounts=etc_pi-hole,dnsmasq_pi-hole name=pihole:latest root-dir=pi-hole start-on-boot=yes \
    workdir=/
/container config
set registry-url=https://registry-1.docker.io tmpdir=pi-hole/pull
/container envs
add key=DNSMASQ_USER list=pi-hole_envs value=root
add key=FTLCONF_webserver_api_password list=pi-hole_envs value=35en!MUrK%0Qkfze
add key=TZ list=pi-hole_envs value=America/Montreal
/interface bridge port
add bridge=bridge-local interface=ether2 internal-path-cost=10 path-cost=10
add bridge=bridge-local interface=ether3 internal-path-cost=10 path-cost=10
add bridge=bridge-local interface=ether4 internal-path-cost=10 path-cost=10
add bridge=bridge-local interface=ether5 internal-path-cost=10 path-cost=10
add bridge=bridge-local interface=ether6 internal-path-cost=10 path-cost=10
add bridge=bridge-local interface=ether1 internal-path-cost=10 path-cost=10
add bridge=bridge-docker interface=veth-docker internal-path-cost=10 path-cost=10
/ip firewall connection tracking
set udp-timeout=10s
/ip neighbor discovery-settings
set discover-interface-list=iflist-LAN lldp-med-net-policy-vlan=1
/ip settings
set tcp-syncookies=yes
/interface detect-internet
set detect-interface-list=all
/interface list member
add interface=bridge-local list=iflist-LAN
add interface=wguard-s2s list=iflist-LAN
add interface=ether8-wan list=iflist-WAN
add interface=wguard-rw list=iflist-LAN
add interface=wguard-proton_ca160 list=iflist-WAN
/interface ovpn-server server
add mac-address=FE:1B:47:4E:A3:0E name=ovpn-server1
/interface wireguard peers
add allowed-address=10.255.254.3/32 interface=wguard-rw name="XXXXXXXXXXXX" public-key="XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
add allowed-address=10.255.254.4/32 interface=wguard-rw name="XXXXXXXXXXXX" public-key="XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
add allowed-address=10.255.254.2/32 interface=wguard-rw name="XXXXXXXXXXXX" public-key="XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
add allowed-address=0.0.0.0/0,::/0 endpoint-address=176.113.74.82 endpoint-port=51820 interface=wguard-proton_ca160 name="Proton VPN - CA#160" persistent-keepalive=25s public-key=\
    "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
/ip address
add address=10.0.4.1/24 interface=bridge-local network=10.0.4.0
add address=10.255.255.2/30 disabled=yes interface=wguard-s2s network=10.255.255.0
add address=10.255.254.1/24 interface=wguard-rw network=10.255.254.0
add address=10.0.5.1/24 interface=bridge-docker network=10.0.5.0
add address=192.168.1.1/24 interface=ether7-admin network=192.168.1.0
add address=10.2.0.2/30 comment="Proton VPN - GA#160" interface=wguard-proton_ca160 network=10.2.0.0
/ip cloud
set ddns-enabled=yes ddns-update-interval=1h
/ip dhcp-client
add default-route-tables=main interface=ether8-wan use-peer-dns=no
/ip dhcp-server network
add address=10.0.4.0/24 dns-server=10.0.4.1 domain=1156rbr.ca gateway=10.0.4.1
/ip dns
set allow-remote-requests=yes servers=10.0.5.2,10.2.0.1
/ip firewall address-list
add address=hda086rwhf2.sn.mynetname.net list=titan.ecurie.ca
/ip firewall filter
add action=accept chain=input comment="accept established, related, untracked" connection-state=established,related,untracked
add action=drop chain=input comment="drop invalid" connection-state=invalid
add action=accept chain=input comment="allow all on admin interface" in-interface=ether7-admin
add action=accept chain=input comment="accept Proton VPN traffic" dst-port=13231 protocol=udp src-address=176.113.74.82
add action=accept chain=input comment="accept WireGuard VPN traffic from titan.ecurie.ca" disabled=yes dst-port=13231 protocol=udp src-address-list=titan.ecurie.ca
add action=accept chain=input comment="accept WireGuard traffic for road warrior clients" dst-port=13232 protocol=udp
add action=drop chain=input comment="block everything else from WAN" in-interface-list=iflist-WAN
add action=fasttrack-connection chain=forward comment="fast-track for established,related" connection-state=established,related hw-offload=yes
add action=accept chain=forward comment="accept established,connected" connection-state=established,related
add action=accept chain=forward comment="accept traffic to Proton VPN" in-interface-list=iflist-LAN out-interface=wguard-proton_ca160
add action=accept chain=forward comment="accept traffic from Proton VPN" in-interface=wguard-proton_ca160 out-interface-list=iflist-LAN
add action=accept chain=forward comment="accept traffic to site to site VPN" disabled=yes in-interface-list=iflist-LAN out-interface=wguard-s2s
add action=accept chain=forward comment="accept traffic from site to site VPN" disabled=yes in-interface=wguard-s2s out-interface-list=iflist-LAN
add action=accept chain=forward comment="accept traffic to RoadWarrior VPN" in-interface-list=iflist-LAN out-interface=wguard-rw
add action=accept chain=forward comment="accept traffic from RoadWarrior VPN" in-interface=wguard-rw out-interface-list=iflist-LAN
add action=drop chain=forward comment="drop invalid" connection-state=invalid
add action=drop chain=forward comment="drop access to clients behind NAT from WAN" connection-nat-state=!dstnat connection-state=new in-interface-list=iflist-WAN
/ip firewall nat
add action=masquerade chain=srcnat out-interface-list=iflist-WAN
add action=masquerade chain=srcnat src-address=10.0.5.0/24
add action=dst-nat chain=dstnat dst-address=10.0.4.1 dst-port=8080 protocol=tcp to-addresses=10.0.5.2 to-ports=80
/ip ipsec profile
set [ find default=yes ] dpd-interval=2m dpd-maximum-failures=5
/ip route
add comment="default internet traffic -> ProtonVPN CA#160" disabled=yes distance=1 dst-address=0.0.0.0/0 gateway=10.2.0.1 routing-table=main suppress-hw-offload=no
add comment="backup internet traffic -> Videotron" disabled=yes distance=2 dst-address=0.0.0.0/0 gateway=ether8-wan routing-table=main suppress-hw-offload=no
add comment="ProtonVPN CA#160 -> ether8-WAN" disabled=no distance=1 dst-address=176.113.74.82/32 gateway=ether8-wan routing-table=main suppress-hw-offload=no
/ip service
set telnet disabled=yes
/ip smb shares
set [ find default=yes ] directory=/pub
/ip ssh
set strong-crypto=yes
/system clock
set time-zone-name=America/Toronto
/system identity
set name=excalibur.agile-pmo.services
/tool mac-server
set allowed-interface-list=listBridge
/tool mac-server mac-winbox
set allowed-interface-list=listBridge

Thanks in advance…

2

Sure

  1. slight modification on address on peer and add persistent keep alive.

It should be:

/interface wireguard
add allowed-address=0.0.0.0/0  endpoint-address=PROTON_IP_ADDRESS  endpoint-port=PROTON-PORT  name=wguard-proton_ca160 public-key="++++++++++"  /
persistent-keep-alive=35

  1. why do you have an input chain rule for PROTON ??

  2. WHo is supposed to use the PROTON connection. I see no mangling or routing rules etc that would force users out wireguard proton vice local WAN??

  • did you want all of bridge to go out PROTON but not admin subnet?
  • The router still needs to establish the handshake with the proton server, so some IP DNS is still needed but I sense this part needs more attention and care.
  1. Recommend set this to none and it can cause issues.
/interface detect-internet
set detect-interface-list=all
3

thanks @anav

Re 1: the only difference i see is you removed the IP6 addresses and changed the keepalive to 35 - or did i miss anything ? Am I right that there is a ‘peer’ missing in your snippet….

Re 2: so UDP traffic in port 31231 from proton would get through had this from my s2s VPN and it worked - is this unnecessary ?

Re 3: eventually everything - working on the live router, so leaving the traffic flowing through ether8-wan until the tunnel is established.

Re 4: thx - wasn’t even aware this was on

Still don’t get any handshake - and have really no clue why….

4

Because your main internet connection uses DHCP client, both these two routes won't work (one is disabled, but if you enable it, then it won't work):

/ip route
add comment="backup internet traffic -> Videotron" disabled=yes distance=2 dst-address=0.0.0.0/0 gateway=ether8-wan routing-table=main suppress-hw-offload=no
add comment="ProtonVPN CA#160 -> ether8-WAN" disabled=no distance=1 dst-address=176.113.74.82/32 gateway=ether8-wan routing-table=main suppress-hw-offload=no

Specifying gateway=ether8-wan is not enough to have working routing, you'll need to specify the IP address of the gateway, that is provided by DHCP.

But because you'll also have other things running on the router (the other WG interfaces) that should also use the ISP gateway instead of being routed through ProtonVPN. You should only direct forwarded traffics from the bridges through ProtonVPN, not the traffics generated by the router itself (otherwise the needed configuration will be much more complicated).

You can improved your configuration like this:

  1. Add a new routing table name via-proton

    /routing table
add fib name=via-proton

  2. Remove the three /ip route entries added by you ("default internet traffic -> ProtonVPN CA#160", "backup internet traffic -> Videotron" and "ProtonVPN CA#160 -> ether8-WAN").

  3. Replace them with this route entry instead:

    /ip route
add comment="forwarded internet traffic -> ProtonVPN CA#160" distance=1 \
    dst-address=0.0.0.0/0 gateway=10.2.0.1@main check-gateway=ping routing-table=via-proton

    Please note that the routing table is via-proton, not main. Also enable check-gateway=ping on that route.

  4. Add the following routing rules to direct traffics from the bridges through VPN, with fall back to the ISP gateway.

    /routing rule
add action=lookup min-prefix=0 table=main
add action=lookup dst-address=0.0.0.0/0 interface=bridge-local table=via-proton
add action=lookup dst-address=0.0.0.0/0 interface=bridge-docker table=via-proton

  5. You should also add a mangle rule that adjust the MSS for TCP traffics going through the ProtonVPN tunnel:

    /ip firewall mangle
add action=change-mss chain=forward comment="reduce MSS for WG" new-mss=1380 \
    out-interface=wguard-proton_ca160 protocol=tcp tcp-flags=syn tcp-mss=1381-65535

That should do it. I've not checked the other parts of your configuration. Improvements can maybe be made to them, but that task's @anav speciality :slight_smile:.

5

Super @CGGXANNX - thanks for the detailed explanation.

Now I generated a new VPN configuration on Proton - same parameters but on a different endpoint - and did the setup as before with the exception of the input chain firewall rule.

→ AND IT WORKS - tunnel is up and I can access the server!! Still not clear why the other tunnel wasn’t working (suspect maybe on proton’s side or some weirdo name issues) but not too concerned right now; will figure this out with a spare/non live router later.

Have changed the routing along my ‘old’ train of thought with the changes of @CGGXANNX regarding the names of the gateway

→ AND IT WORKS i.e. have all internet traffic running over the VPN, have the DNS on the VPN server. Haven’t tested yet what happens if the VPN goes down (suspect traffic goes out via WAN but no DNS as it’s hardcoded to the VPN server) and have not tested the roadwarrior VPN yet.

Will tonight improve the configuration for the routing as per @CGGXANNX

6

Update: have it all working as per @CGGXANNX and @anav ‘s recommendations.

Only thing not working is the RoadWarrior VPN. I’m using Mikrotik’s DDNS service which now goes via the VPN, hence the RoadWarrior tunnel can not be established (or I would have to configure port forwarding on the VPN).

Question1: how can i force the router to use the direct internet providers IP (on ether8-wan in my case) instead of going through the VPN ?

Question2: can I (or HOW can I) exclude a specific machine (static IP in my network, but not on a isolated port on the router) from going through the VPN ?

7

Actually, if you did the configuration like I described above then:

  • In the main table the default route is to use the ISP (ether8-wan).

  • Only traffic coming from the two bridges are directed to use the via-proton table, due to these routing rules:

So, normally:

  • Traffic to MikroTik router to update DDNS is not affected, because not originating from the two bridges, and will still use the main table. Which means the DDNS IP address should still be correct. Can you verify the value of the DNS record?

  • Traffic to and from the Road Warrior WG port also uses the main table and uses the ISP directly. Please note that you have to create a separate WireGuard interface for RW, do not reuse the WG interface of ProtonVPN. Can you verify the Last Handshake field of the RW peers to see if they have non-zero durations?

  • Traffic inside the RW tunnel between the RW clients and the router should also use the main table.

  • When RW clients connect to resources in the LAN (members of the two bridges), then when those resources reply, the return traffic will also use the main table, because the destination should have a route with /24 or something similar destination, in that case this routing rule

    /routing rule
add action=lookup min-prefix=0 table=main

    will make sure that the main table is selected, and the two rules below are skipped.

So I don't know why your RW traffic is not working yet. Maybe you'll need to post the updated censored configuration export.

As for questions #2: You just add exception routing rules to the table, above the rules that select the via-proton table. Assuming you have a device 192.168.88.7 that you want to always use the ISP route, then insert a rule like this, so that the content of the routing rule table becomes:

/routing rule
add action=lookup min-prefix=0 table=main
add action=lookup-only-in-table src-address=192.168.88.7 table=main # <-- the inserted rule!
add action=lookup dst-address=0.0.0.0/0 interface=bridge-local table=via-proton
add action=lookup dst-address=0.0.0.0/0 interface=bridge-docker table=via-proton

Of course, if your list is dynamic, or contains hundreds of entries, then routing rules are no longer adequate. You'll need to switch to using mangle firewall rule for policy routing.

8

Figured out the issue - still had ‘add default route’ in the DHCP client for my internet provider on ‘no’. VPN still worked because I had the explicit route to redirect traffic to the VPN server via my internet provider. Have this route disabled and set the default route on the DHCP client to ‘yes’.

All Works like a charm now !! Thanks @CGGXANNX !