WireGuard - multi-site not passing traffic

RouterOS
1

I have 3 sites configured to work with WireGuard, Office 1 (hub), Office 2, Office 3.
Office 1 ←→ Office 2 works fine, all traffic is passed.

Office 1 ←→ Office 3 works fine, all traffic is passed.

However, when I try to go from Office 2 ←→ Office 3, there is no traffic passed.

Obviously this is a routing issue, unfortunately, I’m unsure as to where the routing issue lies - in the hub or the peers.

office3.rsc (7.3 KB)

office2.rsc (7.0 KB)

office1.rsc (7.3 KB)

(Note: these are setup with DHCP, pulling from 192.168.5.1, and that is where the statics (endpoints) of the wireguard peers are at)

2
  • On the "Office 2" router, you also need to add 192.168.66.0/24 to the list of allowed-address of the WG peer.
  • On the "Office 3" router, you also need to add 192.168.88.0/24 to the list of allowed-address of the WG peer.

Although if "Office 2" has a connection to "Office 3" on the public IP side, then it would be better for the performance if you don't use a hub configuration (with "Office 1" as hub), but make a mesh instead, with "Office 2" also having a separate WG peer for "Office 3", and "Office 3" having a WG for "Office 2". In this configuration, instead of the change mentioned above:

  • On "Office 2":

    • Current WG peer to "Office 1" keeps allowed-address=192.168.77.0/24,10.255.255.1/32.
    • New peer to "Office 3" has (just copy from the entry already on "Office 1"):
      add allowed-address=192.168.66.0/24,10.255.255.3/32 endpoint-address=\
    192.168.5.34 endpoint-port=13231 interface=wireguard1 name="Office 3" \
    persistent-keepalive=10s public-key=\
    "xBbqzO+wlxDNauQPCg+cq3ufTwjhv5dsizc33lBcYyg="

  • On "Office 3":

    • Current WG peer to "Office 1" keeps allowed-address=192.168.77.0/24,10.255.255.1/32.
    • New peer to "Office 2" has (just copy from the entry already on "Office 1"):
      add allowed-address=192.168.88.0/24,10.255.255.2/32 endpoint-address=\
    192.168.5.25 endpoint-port=13231 interface=wireguard1 name="Office 2"\
    persistent-keepalive=10s public-key=\
    "3Tdyf+pTfocoMFMF8pjE+1IKD2MAvPJZ7RRQIKw20VY="

That should give you a triangular mesh between the 3 routers.

3

Thanks for your reply. I did try what you had suggested about the allowed-address on the WG peers, but I still could not pass traffic from Office 2 ←→ Office 3.

I had already considered a peer link from O2 - O3, but since the data is only going to O1 (this is a camera setup), felt the added complexity isn’t warranted. Eventually, I’ll have the Main Office and 5 peers, so the ever-increasing complexity of having a peer-peer connectivity could become a maintenance nightmare.

Currently, connectivity from Office 2 ←→ Office 3 is not necessary, I was kind of surprised that it didn’t “just work” as I had specified the routes to go out onto the WG interface.

I was hoping it would “just work” and was asking if I had missed anything simple that would allow O2←→O3 traffic.

Setting up a torch on Office 1 watching the wireguard interface, when I ping from O2 or O3 to O1, I can see traffic. However, when I try and ping O2 to O3 (or O3 to O2), I don’t see any traffic going through the wireguard interface on O1 - which I would expect. Even with static routes for O2 / O3 to wireguard .. it still doesn’t seem to be able to pass traffic

4

It should work, with "Office 1" as hub, with these settings:

  • On "Office 2":

    • allowed-address=192.168.66.0/24,192.168.77.0/24,10.255.255.1/32 on the only peer that you currently have.
    • Active route for dst-address=192.168.66.0/24 gateway=wireguard1 under /ip route (besides dst-address=192.168.77.0/24 gateway=wireguard1). Your exported configuration already has this.
    • Firewall is not blocking the traffic in the forward chain between the subnets. Regarding this, your exported firewall configuration is OK.

  • On "Office 3":

    • allowed-address=192.168.88.0/24,192.168.77.0/24,10.255.255.1/32 on the only peer that you currently have.
    • Active route for dst-address=192.168.88.0/24 gateway=wireguard1 under /ip route (besides dst-address=192.168.77.0/24 gateway=wireguard1). Your exported configuration already has this.
    • Firewall is not blocking the traffic in the forward chain between the subnets. Regarding this, your exported firewall configuration is OK.

  • On "Office 1" (the hub):

    • Two separate WG peers like you already configured as seen in the export
    • Active route for dst-address=192.168.66.0/24 gateway=wireguard1 and dst-address=192.168.88.0/24 gateway=wireguard1 under /ip route as you've already configured.
    • Firewall not blocking forwarded traffics between the subnets, also already done.
5

Thanks for the info. I did try all of the additions to the allowed-addresses and routes. (see attached configs)

When I run a ping from Office 2 to Office 1, I can run a torch on the WireGuard interface on Office 1 and watch the traffic flow. Likewise from Office 3 to Office 1. I was expecting to see some traffic passthrough the Office 1 router as it went from Office 2 → Office 1 → Office 3, when I would setup a ping from 2 to 3. However I did not.

On the Office 2 router, I see:

image
image668×392 30.8 KB

(NOTE: 10.255.255.2 = Office 2, 192.168.66.1 = Office 3 - this image is from Office 2’s side trying to ping Office 3)

In the routing tables, I’ve even tried specifying the 10.255.255.x as the gateway IP in addition to the wireguard interface.

Additionally, from Office 2, I cannot ping 10.255.255.3 of Office 3 - I get the same result as the Office 3 internal network. Note that from Office 1, I can ping either Office’s 10.255.255.x with no issues, and I can ping the internal network with no issues (and vice-versa).

office3.rsc (7.4 KB)

office2.rsc (7.3 KB)

office1.rsc (7.7 KB)

6

I think of it as an advantage and more robust setup. Im managing 8 locations with WG in star topology. And also peers between sites (mesh) but with select allowed IPs/32.

Imagine one compromised host with unrestricted access to all your subnets, that’s a real nightmare

7

Hi, the allowed-address field of the peer in office2.rsc is not correct, it currently contains:

  • 192.168.77.0/24,192.168.88.0/24,10.255.255.1/32

However, 192.168.88.0/24 is the subnet of "Office 2" itself. Here you are supposed to list the subnets of "Office 1" and "Office 3", which means the correct value should be:

  • 192.168.77.0/24,192.168.66.0/24,10.255.255.1/32

with 192.168.66.0/24 being the subnet at "Office 3".

Same with office3.rsc, you should use 192.168.88.0/24 instead of 192.168.66.0/24. So the correct value has to be:

  • 192.168.77.0/24,192.168.88.0/24,10.255.255.1/32

Once again, in allowed-address you need to list the subnet of the other routers, not of the current router.

Your /ip route entries do the right thing and lists the correct remote subnets. You can see the logic in the following recap:

  • Office 1: has /ip address subnet 192.168.77.0/24, has routes to destination 192.168.66.0/24 and 192.168.88.0/24, Have WG peers with allowed subnets 192.168.66.0/24 and 192.168.88.0/24.

  • Office 2: has /ip address subnet 192.168.88.0/24, has routes to destination 192.168.66.0/24 and 192.168.77.0/24, Have WG peers with allowed subnets 192.168.66.0/24 and 192.168.77.0/24.

  • Office 3: has /ip address subnet 192.168.66.0/24, has routes to destination 192.168.88.0/24 and 192.168.77.0/24, Have WG peers with allowed subnets 192.168.88.0/24 and 192.168.77.0/24.

8

Thanks for the info. I was able to get it to ping back and forth between Office 2 and Office 3. Albeit, it had a TTL error. As I was looking at that, the traceroute showed the ping from Office 3, bouncing between Office 1 and Office 2. This also happened when I ran a trace between 2 and 3.

At this point, I appreciate your help and insight, but I’m going to shelve this lab for now.