WireGuard - Routing to subnet not working

newpipe · February 2, 2022, 6:22pm

Hi,
I’ve set up a WireGuard VPN to connect two locations with each other, in order to access some network resources from a remote site.
This is, how my setup looks like.
The left side (remote) wants to access resources from the right side (local).

Now the problem is, that from the Mikrotik Router, where the WireGuard remote instance is running, I cannot ping any of the 192.168.2.x addresses of the local site (right side).
However, if I connect my Android phone with the same Wireguard config of the remote, I can easily access and ping 192.168.2.x resources of the local site.
I hope, someone can point me to what I did wrong on the Mikrotik router (left side).

This is the WireGuard config from the OpenWRT router of the local site (right side):

admin@verteiler:~# wg showconf wg_vpn_guest
[Interface]
ListenPort = 51908
PrivateKey = xxx

[Peer]
PublicKey = Pv14QyXfjgC7kRsnEmYr0PjbAI6rrYXt6HU4TjBN5Ts=
AllowedIPs = 192.168.15.2/32

[Peer]
PublicKey = LsS8UVYHN6vaVFM+z72RxC9QP4hlhAPOCMhHD2OiAyc=
AllowedIPs = 192.168.15.3/32

[Peer]
PublicKey = Wm2xuNzECP98a38gNHQDLuKi0euIehF3ocqBgC4WLz4=
AllowedIPs = 192.168.15.4/32

[Peer]
PublicKey = hjUGQJ8/hoR0A2lHsvo7ue2WxlQ7PEAeDd8Kyom8GSQ=
AllowedIPs = 192.168.15.5/32

[Peer]
PublicKey = tHqcViAZC0/xFWCaLd66Of2dWFGA2P7fzZnPJ78F91I=
AllowedIPs = 192.168.15.6/32

This is the config from the Mikrotik router of the remote site:

\[user2@RB3011\] /interface/wireguard> export 
# feb/02/2022 19:10:32 by RouterOS 7.1.1
#
# model = RB3011UiAS
/interface wireguard
add listen-port=13231 mtu=1420 name=wg0
/interface wireguard peers
add allowed-address=192.168.15.0/24,192.168.2.0/24 comment="VPN zu local" endpoint-address=185.68.227.63 endpoint-port=51908 interface=wg0 public-key=\
    "HycNIQFZSzR066fpsFNxJR0R59zhGVjPbnQXurCg0Rc="

This is the route that I added on the Mikrotik device of the remote site to route traffic for 192.168.2.x:

/ip route
add comment="Standard Gateway" disabled=no dst-address=0.0.0.0/0 gateway=192.168.5.1
add dst-address=192.168.2.0/24 gateway=wg0

This is the ip/address config of the remote Mikrotik:

/ip address
add address=192.168.88.1/24 comment=defconf interface=bridge network=192.168.88.0
add address=192.168.5.250/24 interface=bridge network=192.168.5.0
add address=192.168.15.6/24 comment="VPN zu local" interface=wg0 network=192.168.15.0

I’m really curious, if someone can see the problem in my config.

Thanks

anav · February 2, 2022, 7:27pm

MIKROTIK ( no major issues noted ref wireguard but check out your bridge !! )
OPENWRT ( NONE OF THE PEERS ALLOWS 192.168.5.0/24 ??? )

model = RB3011UiAS

/interface wireguard
add listen-port=13231 mtu=1420 name=wg0

/interface wireguard peers
add allowed-address=192.168.2.0/24 ( I dont use IP addresses for my wg interfaces but your setup with IP addresses should not be causing any issues )
endpoint-address=185.68.227.63
endpoint-port=51908 interface=wg0
public-key= From OpenWRT

Routes
dst-address=0.0.0.0/0 gwy=192.168.5.1 table=main
dst-address=192.168.2.0/24 gwy=wg0 table=main

There should be no problem with your routes, since both are table main, the more precise destination address (192.168.2.0/24) will take precedence when the user attempts to access the remote site. Nothing needs to change here.

/ip address
add address=192.168.88.1/24 comment=defconf interface=bridge network=192.168.88.0
add address=192.168.5.250/24 interface=bridge network=192.168.5.0
add address=192.168.15.6/24 comment="VPN zu local" interface=wg0 network=192.168.15.0 { I dont use IP addresses on my MT devices but it should neither cause problems if you do }.

newpipe · February 3, 2022, 6:32pm

Hi,
thanks much for your helpful reply. Unfortunately I didn’t have time yet to try your suggestions. However, I’ve some comments already.

Hmmm… Good point. However, I’m trying to ping directly from the command interface of the Mikrotik. I assume, it would simply use the wg0 IP (192.168.15.6) to send the ping from.
I successfully receive a reply, if I ping to 192.168.15.1 from the OpenWRT. I also see the ICMP packages, if I sniff the traffic from the wg interface of the OpenWRT. But I see no activity, if I ping to 192.168.2.1.
And again, if I establish the same VPN connection from an Android device to OpenWRT, I can successfully ping 192.168.15.1 and 192.168.2.1.

So I guess, there really must something wrong in the config of the Mikrotik device. Will try it tomorrow.

Thanks

anav · February 3, 2022, 6:48pm

Well fix up the bridge on the MT for sure!!
Then try pinging but dont try wg-0 try an allowed subnet address etc…
Dont forget to fix peer settings to the IP and not wg ip address.

newpipe · February 5, 2022, 12:59pm

I’ve no tried to apply your suggestions. But without success.
I can perfectly access all resources within the 192.168.5.0 network from the 192.168.2.0 network (OpenWRT side).

On the 192.168.5.0 network with the MT device, there is also a QNAP NAS.
If I set-up the same Wireguard tunnel on the QNAP, I can perfectly access resources within the 192.168.2.0 network of the OpenWRT side.
But if the tunnel is running on the MT device, no packets with destination 192.168.2.0 are leaving the MT device.
I also checked that with Wiresharking the wg0 interface on the OpenWRT. No packets coming. Pinging the wg0 OpenWRT interface on 192.168.15.1 is working.
So there really must be some setting on the MT device that I have wrong. Maybe something on the firewall?

These are my current firewall settings on the MT

/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment=test
add action=accept chain=forward dst-address=192.168.5.0/24
add action=accept chain=forward dst-address=192.168.2.0/24 out-interface=wg0
add action=accept chain=forward dst-address=192.168.15.0/24
add action=accept chain=input comment="defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=accept chain=input comment="Accept Wireguard" protocol=udp
add action=accept chain=forward comment="Forward Wireguard Traffic" src-address=10.15.15.0/24
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related hw-offload=yes
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=LAN

The routing table looks as follows:

/ip route
add comment="Standard Gateway" disabled=no distance=2 dst-address=0.0.0.0/0 gateway=192.168.5.1 pref-src="" routing-table=main suppress-hw-offload=no
add disabled=no distance=1 dst-address=192.168.2.0/24 gateway=wg0 pref-src="" routing-table=main suppress-hw-offload=no

The bridge settings are:

/interface bridge
add admin-mac=C4:XX:XX:.. auto-mac=no comment=defconf name=bridge
/interface bridge port
add bridge=bridge comment=defconf ingress-filtering=no interface=ether2
add bridge=bridge comment=defconf ingress-filtering=no interface=ether3
add bridge=bridge comment=defconf ingress-filtering=no interface=ether4
add bridge=bridge comment=defconf ingress-filtering=no interface=ether5
add bridge=bridge comment=defconf ingress-filtering=no interface=ether6
add bridge=bridge comment=defconf ingress-filtering=no interface=ether7
add bridge=bridge comment=defconf ingress-filtering=no interface=ether8
add bridge=bridge comment=defconf ingress-filtering=no interface=ether9
add bridge=bridge comment=defconf ingress-filtering=no interface=ether10
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp1

And the v4 address settings:

/ip address
add address=192.168.88.1/24 comment=defconf disabled=yes interface=bridge network=192.168.88.0
add address=192.168.5.250/24 interface=bridge network=192.168.5.0
add address=10.15.15.1/24 interface=wg0 network=10.15.15.0
add address=192.168.15.6/24 comment="VPN zu Michi" interface=wg0 network=192.168.15.0

And the WireGuard settings:

/interface wireguard
add listen-port=13231 mtu=1420 name=wg0
/interface wireguard peers
add allowed-address=10.15.15.2/32 comment="Endpoint Michi 1" interface=wg0 public-key="HLa3bCxpmIeQ2TrUJeZno4Z9tLCkFDhII+GIKOGmLhM="
add allowed-address=192.168.2.0/24,192.168.15.0/24 comment="VPN zu Michi" endpoint-address=185.68.227.63 endpoint-port=51908 interface=wg0 persistent-keepalive=25s public-key=\
    "HycNIQFZSzR066fpsFNxJR0R59zhGVjPbnQXurCg0Rc="

Do you see now what you wanted me to change? Anything else that you can think of?

anav · February 5, 2022, 3:10pm

I am confused, WHAT IS THE FIRST WIREGUARD PEER… I see no 10.15.15 network in your diagram and I thought that you couldnt use the MT as a server???

MT PEERS
/interface wireguard
add listen-port=13231 mtu=1420 name=wg0
/interface wireguard peer1
add allowed-address=10.15.15.2/32 comment=“Endpoint Michi 1” ??? DELETE!!
interface=wg0
public-key

/interface wireguard peer2
add allowed-address=192.168.2.0/24,192.168.15.0/24 comment=“VPN zu Michi” ( you are not passing data to a wg interface so remove)
endpoint-address=185.68.227.63 endpoint-port=51908
interface=wg0
persistent-keepalive=25s
public-key=\

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
/ip address
add address=192.168.88.1/24 comment=defconf disabled=yes interface=bridge network=192.168.88.0
add address=192.168.5.250/24 interface=bridge network=192.168.5.0
add address=10.15.15.1/24 interface=wg0 network=10.15.15.0
add address=192.168.15.6/24 comment=“VPN zu Michi” interface=wg0 network=192.168.15.0

Why do you have two addresses for WG-0 interface. DELETE BOTH DONT NEED EM, and this is a reason why I HATE THEM, they are never used properly and are not required in the end anyway.

+++++++++++++++++++++++++++++++++++++++

IP ROUTES ( the first one serves no purpose delete)
/ip route
add comment=“Standard Gateway” disabled=no distance=2 dst-address=0.0.0.0/0 gateway=192.168.5.1 pref-src=“” routing-table=main suppress-hw-offload=no???
add disabled=no distance=1 dst-address=192.168.2.0/24 gateway=wg0 pref-src=“” routing-table=main suppress-hw-offload=no oKAY!!

should look like
dst-address=0.0.0.0/0 gwy=ISP table=main (should already be in place - the usual default route )
dst-address=192.168.2.0/24 gwy=WG=0 table=main ( is the only extra route required to be added )

++++++++++++++++++++++++++++++++++++++++++++++++++++++++
IP FIREWALL RULES

ip firewall filter
add action=accept chain=input comment=“defconf: accept established,related,untracked” connection-state=established,related,untracked
add action=drop chain=input comment=“defconf: drop invalid” connection-state=invalid
add action=accept chain=input comment=“defconf: accept ICMP” protocol=icmp
add action=accept chain=input comment=test DELETE , does nothing.

add action=accept chain=input comment=“defconf: accept to local loopback (for CAPsMAN)” dst-address=127.0.0.1
add action=accept chain=input comment=“Accept Wireguard” protocol=udp
add action=drop chain=input comment=“defconf: drop all not coming from LAN” in-interface-list=!LAN

Fix order FOR THE REST… input with input, forward with forward.

add action=accept chain=forward comment=“defconf: accept in ipsec policy” ipsec-policy=in,ipsec
add action=accept chain=forward comment=“defconf: accept out ipsec policy” ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment=“defconf: fasttrack” connection-state=established,related hw-offload=yes
add action=accept chain=forward comment=“defconf: accept established,related, untracked” connection-state=established,related,untracked
add action=drop chain=forward comment=“defconf: drop invalid” connection-state=invalid
add action=drop chain=forward comment=“defconf: drop all from WAN not DSTNATed” connection-nat-state=!dstnat connection-state=new in-interface-list=WAN

/ip firewall nat WRONG see fix.
add action=masquerade chain=srcnat comment=“defconf: masquerade” ipsec-policy=out,none out-interface-list=LAN .. use → WAN

All these rules below, delete not required Yur firewall rules do not block any LAN to LAN type traffic. Also they do not block LAN to WAN traffic, did you make a rule for that, NO! So no need for all these others!! Besides the fact I dont like your firewall rule setup but its functional with the changes I have pointed out and should work.

add action=accept chain=forward dst-address=192.168.5.0/24
add action=accept chain=forward dst-address=192.168.2.0/24 out-interface=wg0
add action=accept chain=forward dst-address=192.168.15.0/24
add action=accept chain=forward comment=“Forward Wireguard Traffic” src-address=10.15.15.0/24

++++++++++++++++++++

Make sure on the pEER settings on the OPENWRT Wireguard server, that the PEER setting for the MT you use-----> allowed ips =192.168.5.0/24

anav · February 5, 2022, 3:17pm

By the way, I said all of this on the first post, and you are not listening??

holvoetn · February 5, 2022, 3:23pm

Maybe you didn’t say it clear or nice enough ?

Sob · February 5, 2022, 3:34pm

Didn’t you sort of agree to not purge addresses on sight? I’m with you on 10.15.15.1/24, that seems pointless (edit: actually, no, that’s just another peer, there’s nothing wrong with that). But how do you know that 192.168.15.6/24 isn’t required? It could be useful if there should be access from other WG peers connected to server (if server allows it). You can’t know that, so you shouldn’t touch it.

Sob · February 5, 2022, 3:47pm

As for the problem, one obvious mistake is peer’s definition on server, it should have (as already suggested):

AllowedIPs = 192.168.15.6/32,192.168.5.0/24

Another problem is that when RB is just device in LAN and not default gateway for other devices in same LAN, when they want to send anything to 192.168.2.x, they will send it to FritzBox as their default gateway, not to RB. So you need to add static route on FritzBox to 192.168.2.0/24 with 192.168.5.250 as gateway. Other direction (192.168.2.x → 192.168.5.x) may work if there’s srcnat on RB.

anav · February 5, 2022, 4:23pm

Ahh Sob, excellent point, on the fritz box.

Internet traffic heading from the wg tunnel from the MT device will go out the Fritz box to the internet and return.
When it hits the Fritz box, the Fritz will look DEEP into the returning traffic and find a source address IT DOES NOT HAVE on its networks.
So you are saying there is two techniques here at play.

(1) STATIC ROUTE METHOD:
Static Route on the FRITZ BOX to ensure that traffic with source address of an uknown subnet ( coming from the MT routers wireguard tunnel ) is routed back to the MT Router.

(2) SOURCE NAT METHOD - PREFERRED WHEN UNABLE TO ACCESS FIRST ROUTER.
Ensures all traffic is source natted with the WANIP of the MT device (second router). By that we mean the actual SOURCE IP of the traffic!!.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
As for the 10.15 etc… yes its most likely another peer and tunnel we know nothing about, AND if you are going to put that in the config and not reflect it in the diagram I will flag it!!!

Sob · February 5, 2022, 4:43pm

Srcnat helps only with one direction (WG->LAN), because it makes it all look as if it’s from RB (192.168.5.250). That’s btw what you most likely broke with your suggestion to fix existing masquerade rule. It doesn’t help with the other (LAN->WG), for that you need route on default gw, or routes on individual LAN devices (but that’s not good from management perspective).

anav · February 5, 2022, 4:46pm

THis look right??
When a Mikrotik Router is behind another MT Router (best case) or behind an ISP Router (not as good) , where one can still create static routes and port forwardings or behind an ISP router (worst case) that is ‘closed’ off, the admin will have to take extra steps during the config process. Without the ability to forward ports, on the ISP Router, one cannot use the MT as a Server end for connection purposes and without static routes, on the ISP Router, we can run into other issues with internet bound traffic.

Consider the case where the MT Wireguard connection is being used by REMOTE USERS to go out the internet of the MT, through the ‘closed’ ISP router.
The issue that arises is that returning traffic from that internet connection still carries the SOURCE ADDRESS of the users on the remote subnet. These IP addresses will not be
recognized by the ISP router and it will not know where to send them and they will be dropped. There are two solutions:

(1) Static Route: If the ISP router is capable of static routes then the following should be constructed:
dst=address=subnet_of_users gwy=WANIP of second router table=main

(2) Source NAT: With a completely closed ISP router, then the preferred solution is to SRC NAT the traffic to the WANIP of the Mikrotik device. In essence we are replacing the source address of the subnet users with the WAN IP of the MT device. Since this is also the LANIP of the MT device wrt the ISP router, all returning internet traffic will get passed to the indicated source address (MT Device) and the MT device will un SRC-NAT the source address back to the subnet user IPs.
add chain=src-nat action=masquerade out-interface-list=WAN src-address=‘subnet_of_remote_users’

newpipe · February 11, 2022, 12:49pm

Hi,
I thank all of you for the helpful contriibution.
I’m sorry that it took so long to reply. Altough the topic is important to me, it’s currently difficult to find the right amount of time to focus on it.

So let me first of all provide an updated network diagram. I’ve added the QNAP NAS that i mentioned in one of the earlier posts.

Let me clarifie some things that I’ve seen in your posts.

The role of the Mikrotik Routerboard is to be a switch. The ISP gateway of the network is the “FritzBox”, which is the Standardgateway of the Mikrotik device (192.168.5.1).
Now the Mikrotik device should get a new role assigned. Being the Wireguard gateway for the network to another private network.
The Mikrotik device is establishing the connection to the OpenWRT device (192.168.2.x).
On the FritzBox it is possible to define static routes and create port forwards. Later I will create the static route to forward 192.168.2.x traffic to the Mikrotik device.
The ping tests into the remote 192.168.2.x network I’ve done directly on the Mikrotik device. So no question of static routes on the FritzBox.
When moving the same Wireguard config to the QNAP NAS for testing, I can successfully ping targets in the remote 192.168.2.x network from the QNAP NAS.
I can successfully ping from any 192.168.2.x device to any 192.168.5.x device. But just not the other way round. Even not directly on the Mikrotik device, where the tunnel is established.

The peer that you see with the 10.x address that you see in the Mikrotik config was just for testing. But in the described scenario, this peer is not used and no tunnel with this peer is alive.

Because of the above, it must be some misconfiguration on my Mikrotik device. Maybe I’m missing some fundamental firewall rule, some Bridge config or some IP settings, which currently looks as follows:

 ip/settings/print
              ip-forward: yes
          send-redirects: yes
     accept-source-route: no
        accept-redirects: no
        secure-redirects: yes
               rp-filter: no
          tcp-syncookies: no
    max-neighbor-entries: 8192
             arp-timeout: 30s
         icmp-rate-limit: 10
          icmp-rate-mask: 0x1818
             route-cache: yes
         allow-fast-path: yes
   ipv4-fast-path-active: no
  ipv4-fast-path-packets: 0
    ipv4-fast-path-bytes: 0
   ipv4-fasttrack-active: yes
  ipv4-fasttrack-packets: 57
    ipv4-fasttrack-bytes: 29536

Btw. the below is now the latest WG config.
OpenWRT device (192.168.2.1):

 wg showconf wg_vpn_guest
[Interface]
ListenPort = 51908
PrivateKey = xxx

[Peer]
PublicKey = Pv14QyXfjgC7kRsnEmYr0PjbAI6rrYXt6HU4TjBN5Ts=
AllowedIPs = 192.168.15.2/32

[Peer]
PublicKey = LsS8UVYHN6vaVFM+z72RxC9QP4hlhAPOCMhHD2OiAyc=
AllowedIPs = 192.168.15.3/32

[Peer]
PublicKey = Wm2xuNzECP98a38gNHQDLuKi0euIehF3ocqBgC4WLz4=
AllowedIPs = 192.168.15.4/32

[Peer]
PublicKey = hjUGQJ8/hoR0A2lHsvo7ue2WxlQ7PEAeDd8Kyom8GSQ=
AllowedIPs = 192.168.15.5/32

[Peer] <this is the peer for the Mikrotik device>
PublicKey = tHqcViAZC0/xFWCaLd66Of2dWFGA2P7fzZnPJ78F91I=
AllowedIPs = 192.168.15.6/32, 192.168.5.0/24

The Mikrotik device (192.168.5.250):

/interface wireguard
add listen-port=13231 mtu=1420 name=wg0
/interface wireguard peers
add allowed-address=192.168.2.0/24,192.168.15.0/24 comment="VPN zu Michi" endpoint-address=<pub IP of OpenWRT> endpoint-port=51908 interface=wg0 \
    persistent-keepalive=25s public-key="HycNIQFZSzR066fpsFNxJR0R59zhGVjPbnQXurCg0Rc="

Thanks

Sob · February 11, 2022, 1:24pm

The route should help.

You have no problem when testing with WG on QNAP, because it’s just QNAP itself, it has direct link to 192.168.2.x, it can’t get lost. But when WG tunnel is on RB, then the same QNAP (when it doesn’t have own tunnel anymore) can’t reach 192.168.2.x, because it has no idea that gateway to 192.168.2.x is RB. So it sends those packets to FritzBox, and it doesn’t know anything about 192.168.2.x either, so it may send it to internet or block it (doesn’t really matter which one it is, because in both cases it doesn’t work). But if FritzBox has the route, it will forward those packets to RB.

newpipe · February 11, 2022, 2:59pm

Thanks for your quick reply.

Actually it’s a bit different.

When I create the Wireguard tunnel directly on the Mikrotik device:
I can ping from the remote end (192.168.2.x) to devices that are connected to the Mikrotik device (192.168.5.x).
- It is not possible to ping from the Console of the Mikrotik device (remember, the tunnel is established on the same device) to remote devices (192.168.2.x).
When I create the Wireguard tunnel on the QNAP device with the same config (inside the same network), just for testing:
I can ping from the Linux Shell of the QNAP to devices to the remote end (192.168.2.x). Remember, this is not working on the Mikrotik device

.
You see, I’m even not yet testing to route traffic within the 192.168.5.x network via the Mikrotik to the remote 192.168.2.x network. All tests so far to reach the remote 192.168.2.x network are still executed at the same device that is establishing the Wireguard tunnel.

Sob · February 11, 2022, 4:06pm

Access to 192.168.2.x from RB should work. You can check using Tools->Torch on WG interface that packets go out to OpenWRT at the other end. Maybe there could be problem with source address, RB will choose 192.168.15.6 and either OpenWRT or target devices may not like it, so check their firewalls.