WinBox access over multi-peer WireGuard network

Buongiorno,

I have build a network with 3 MikroTik routers the names are:

GreatWall: LAN 172.16.0.0/26 - WG ADD 10.10.12.1
HellsGate: LAN 172.16.10.0/26 - WG ADD 10.10.12.2
FortKnox: LAN 172.16.20.0/27 - WG ADD 10.10.12.3

This “mesh” connection is called Fireloop.

Communication between devices across the different LANs works perfectly — for example, 172.16.0.10 can reach any device in the other LANs through the Fireloop tunnels.

The issue concerns WinBox management: I’m unable to access one of the other routers via WinBox through the Fireloop tunnels.

For clarification, when I use my Mac, I can establish a dedicated tunnel between the Mac and one router at a time (non-mesh configuration). In that setup, WinBox access works correctly when the proper filter rule is applied.

4    ;;; Allow WinBox over WG Fireloop
      chain=input action=accept protocol=tcp in-interface-list=HWG dst-port=8291 log=no log-prefix=""

(HWG is the interface lists of all my WireGuard interfaces)

But when the Mac doesn’t have a dedicated (“point-to-point”) WireGuard tunnel active and is instead using the routers’ mesh network, I can only access the router that manages the LAN I’m physically connected to.

Is it possible to access the others as well? What am I missing?

EDIT: All these routers are behind a NAT, one with public IP and two no. All attempts to login via WinBox are aborting with “Wrong password” but of course is not correct. I’m pretty sure that the problem is here, the when a ping GreatWall from a device of the FortKnox LAN is the FortKnox itself to reply (<1ms)

FortKnox
Flags: X - disabled, I - invalid; D - dynamic
0   ;;; defconf: masquerade
chain=srcnat action=masquerade out-interface-list=WAN ipsec-policy=out,none
GreatWall
Flags: X - disabled, I - invalid; D - dynamic 
 0    ;;; defconf: masquerade
      chain=srcnat action=masquerade out-interface-list=WAN log=no log-prefix="" ipsec-policy=out,none 

 1    ;;; HTTP -> 172.16.0.16
      chain=dstnat action=dst-nat to-addresses=172.16.0.16 to-ports=80 protocol=tcp dst-address=192.168.30.2 dst-port=80 log=no log-prefix="" 

 2    ;;; HTTPS -> 172.16.0.16
      chain=dstnat action=dst-nat to-addresses=172.16.0.16 to-ports=443 protocol=tcp dst-address=192.168.30.2 dst-port=443 log=no log-prefix=""

EDIT2: The clients of the LANs are able to ping each other but the routers aren’t able to ping both routers and clients, this is strange…

Can you confirm that connectivity between the routers is provided by Wireguard VPN tunnels. I am not keen on using the word mesh as that is probably not what you have. Wireguard is a point to point protocol and you have more likely a hub and spoke type setup. . Meshing could be possible but only if all three sites have public IP addresses, I think.

Regardless, please post config of all three sites for review.
/export file=anynameyouwish ( minus router serial #, any public WANIP information, keys, dhcp lease lists)

I’m pretty sure, the way to achive that is that the device without a public IP should start the tunnel, once created, with a persistent keep alive it can be maintained, is not the first time that I’m using this solution.I’m not saying that is correct, but it’s working, otherwise i wouldn’t been able to communicate.

FORTKNOX

/interface bridge
add admin-mac=xxxxxxxxxxx auto-mac=no comment=defconf name=bridge
/interface wireguard
add listen-port=51820 mtu=1420 name=wg-fireloop
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
add comment="Home WireGuard tunnel" name=HWG
add name=MGMT
/ip pool
add name=dhcp-lan ranges=172.16.20.10-172.16.20.30
/ip dhcp-server
add address-pool=dhcp-lan interface=bridge name=dhcp1
/disk settings
set auto-media-interface=bridge auto-media-sharing=yes auto-smb-sharing=yes
/interface bridge port
add bridge=bridge comment=defconf interface=ether2
add bridge=bridge comment=defconf interface=ether3
add bridge=bridge comment=defconf interface=ether4
add bridge=bridge comment=defconf interface=ether5
add bridge=bridge comment=defconf interface=sfp1
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=ether1 list=WAN
add interface=wg-fireloop list=HWG
add interface=wg-fireloop list=MGMT
add interface=bridge list=MGMT
/interface wireguard peers
add allowed-address=10.10.12.1/32,172.16.0.0/26 comment=GreatWall \
    endpoint-address=[GreatWall Public IP] endpoint-port=51820 interface=\
    wg-fireloop name=peer5 persistent-keepalive=25s public-key=\
    "xxxxxxxx"
add allowed-address=10.10.12.4/32 comment="other client" interface=\
    wg-fireloop name=peer6 persistent-keepalive=25s public-key=\
    "xxxxxxxx="
/ip address
add address=172.16.20.1/27 comment=defconf interface=bridge network=\
    172.16.20.0
add address=10.10.12.3/24 interface=wg-fireloop network=10.10.12.0
add address=172.16.0.1/26 interface=wg-fireloop network=172.16.0.0
/ip dhcp-client
add comment=defconf interface=ether1
/ip dhcp-server network
add address=172.16.20.0/27 comment=defconf dns-server=172.16.20.1 gateway=\
    172.16.20.1 netmask=27
/ip dns
set allow-remote-requests=yes cache-size=40000KiB servers=1.1.1.1,1.0.0.1
/ip firewall filter
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="Allow WG Fireloop mesh" disabled=yes \
    dst-port=51820 protocol=udp
add action=accept chain=input comment="Allow WinBox over WG Fireloop" \
    dst-port=8291 in-interface-list=HWG protocol=tcp
add action=accept chain=forward comment="Allow intercomm WG Fireloop" \
    in-interface-list=HWG out-interface-list=LAN
add action=accept chain=forward in-interface-list=LAN out-interface-list=HWG
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    in-interface-list=!LAN
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-state=established,related disabled=yes hw-offload=yes
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" \
    ipsec-policy=out,none out-interface-list=WAN

/system clock
set time-zone-name=Europe/Rome
/system identity
set name=FortKnox

/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=MGMT


GREATWALL

/interface bridge
add name=bridge-dmz
add admin-mac=xxxxxxxxxx auto-mac=no comment=defconf name=bridge-lan
/interface ethernet
set [ find default-name=ether1 ] comment=WAN
set [ find default-name=sfp-sfpplus1 ] l2mtu=9018 mtu=9000
/interface wireguard
add comment="Access GreatWall <-> HellsGate <-> FortKnox" listen-port=51820 \
    mtu=1420 name=wg-fireloop
add comment="WireGuard to VVF network" listen-port=43864 mtu=1420 name=wg-vvf
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
add name=DMZ
add name=MGMT
add name=HWG
/ip dhcp-server option
add code=12 name=HOSTNAME
/ip pool
add name=dhcp-lan ranges=172.16.0.40-172.16.0.62
add name=dhcp-dmz ranges=172.16.1.20-172.16.1.30
/ip dhcp-server
add address-pool=dhcp-lan comment=defconf interface=bridge-lan name=dhcp-lan
add address-pool=dhcp-dmz interface=bridge-dmz name=dhcp-dmz
/interface bridge port
add bridge=bridge-lan comment=defconf interface=ether2
add bridge=bridge-lan comment=defconf interface=ether3
add bridge=bridge-lan comment=defconf interface=ether4
add bridge=bridge-dmz comment=defconf interface=ether5
add bridge=bridge-dmz comment=defconf interface=ether6
add bridge=bridge-dmz comment=defconf interface=ether7
add bridge=bridge-dmz comment=defconf interface=ether8
add bridge=bridge-lan comment=defconf interface=sfp-sfpplus1
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface list member
add comment=defconf interface=bridge-lan list=LAN
add comment=defconf interface=ether1 list=WAN
add interface=bridge-dmz list=DMZ
add interface=wg-vvf list=MGMT
add interface=wg-fireloop list=MGMT
add interface=bridge-lan list=MGMT
add comment="Home WireGuard tunnels" interface=wg-fireloop list=HWG

/interface wireguard peers
add allowed-address=10.10.10.3/32,192.168.99.0/24 comment=MAIN-VVF \
    endpoint-address=xxxxx.xxxx.xxx endpoint-port=51820 interface=wg-vvf \
    name=peer8 persistent-keepalive=25s public-key=\
    "xxxxxxxx"
add allowed-address=10.10.12.5/32 comment=OtherDev interface=wg-fireloop \
    name=peer16 public-key="xxxxxxxxxxx"
add allowed-address=10.10.12.3/32,172.16.20.0/27 comment=FortKnox interface=\
    wg-fireloop name=peer18 persistent-keepalive=25s public-key=\
    "xxxxxxxxx"
add allowed-address=10.10.12.4/32 comment="Other dev" interface=\
    wg-fireloop name=peer19 persistent-keepalive=25s public-key=\
    "xxxxxxxxxxxxxx="
add allowed-address=10.10.12.6/32 comment=OtherDev interface=wg-fireloop \
    name=peer20 public-key="xxxxxxxxxxx="
/ip address
add address=172.16.0.1/26 comment=defconf interface=bridge-lan network=\
    172.16.0.0
add address=172.16.1.1/27 interface=bridge-dmz network=172.16.1.0
add address=10.10.12.1/24 interface=wg-fireloop network=10.10.12.0
add address=192.168.30.2/30 comment=WAN interface=ether1 network=192.168.30.0
add address=10.10.10.3/24 interface=wg-vvf network=10.10.10.0
add address=172.16.20.1/27 interface=wg-fireloop network=172.16.20.0
/ip dhcp-client
add comment=defconf interface=ether1

/ip dhcp-server network
add address=172.16.0.0/26 comment=defconf dns-server=\
    172.16.0.1,1.1.1.1,1.0.0.1 gateway=172.16.0.1 netmask=26
add address=172.16.1.0/27 dns-server=172.16.1.1,1.1.1.1,1.0.0.1 gateway=\
    172.16.1.1
/ip dns
set allow-remote-requests=yes cache-size=45000KiB servers=1.1.1.1,1.0.0.1

/ip firewall address-list
add address=172.16.0.0/26 list=Fireloop_LAN
add address=172.16.10.0/26 list=Fireloop_LAN
add address=172.16.20.0/27 list=Fireloop_LAN
add address=10.10.12.0/24 list=Fireloop_LAN
/ip firewall filter
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="Allow WG Fireloop mesh" dst-port=51820 \
    protocol=udp
add action=accept chain=input comment="Allow WinBox over WG Fireloop" \
    dst-port=8291 in-interface-list=HWG protocol=tcp
add action=accept chain=input comment="Allow WinBox over WG Fireloop" \
    dst-port=8291 protocol=tcp src-address-list=Fireloop_LAN
add action=accept chain=input comment="Allow SSH over WG Fireloop" dst-port=\
    22 in-interface-list=HWG protocol=tcp
add action=accept chain=input comment="Allow DNS over WG Fireloop" dst-port=\
    53 in-interface-list=HWG protocol=udp
add action=accept chain=forward comment="Allow intercomm WG Fireloop" \
    in-interface-list=HWG out-interface-list=LAN
add action=accept chain=forward in-interface-list=LAN out-interface-list=HWG
add action=accept chain=forward comment="Allow intercomm WG VVF" \
    in-interface=wg-vvf out-interface-list=LAN
add action=accept chain=forward in-interface-list=LAN out-interface=wg-vvf
add action=accept chain=forward comment="Allow intercomm WG VVF-Fireloop" \
    in-interface=wg-vvf out-interface-list=HWG
add action=accept chain=forward in-interface-list=HWG out-interface=wg-vvf
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    in-interface-list=!LAN
add action=accept chain=forward comment="DMZ forward in" in-interface=\
    bridge-dmz
add action=accept chain=forward comment="DMZ forward out" out-interface=\
    bridge-dmz
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-state=established,related disabled=yes hw-offload=yes
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" \
    ipsec-policy=out,none out-interface-list=WAN
add action=dst-nat chain=dstnat comment="HTTP -> 172.16.0.16" dst-address=\
    192.168.30.2 dst-port=80 protocol=tcp to-addresses=172.16.0.16 to-ports=\
    80
add action=dst-nat chain=dstnat comment="HTTPS -> 172.16.0.16" dst-address=\
    192.168.30.2 dst-port=443 protocol=tcp to-addresses=172.16.0.16 to-ports=\
    443
/ip ipsec profile
set [ find default=yes ] dpd-interval=2m dpd-maximum-failures=5
/ip route
add disabled=no distance=1 dst-address=192.168.99.0/24 gateway=wg-vvf \
    routing-table=main scope=30 suppress-hw-offload=no target-scope=10
add comment="DEFAULT GATEAWAY" disabled=no distance=1 dst-address=0.0.0.0/0 \
    gateway=192.168.30.1 routing-table=main scope=30 suppress-hw-offload=no \
    target-scope=10
/ip service
set winbox address=0.0.0.0/0

/system clock
set time-zone-name=Europe/Rome
/system identity
set name=GreatWall
/system ntp server
set use-local-clock=yes
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=MGMT

The third device is actually not available, but is not different than GreatWall. In anycase is a conceptual problem that i need to understand, i want to been able to recreate it.

In another environment I’ve been able to do it without problem but the equivalent of GreatWall wasn’t behind a NAT, the equivalent of FortKnox was behind the same conditions instead.

That is correct. The general approach is that both remote routers (without a public IP) connect to the MAIN router with a public IP. These are direct ptp connections.
Through routes and firewall rules we can ensure that traffic between all of them flows.

What is best is to have a clear set of requirements.
USERS,
which subnets on Router1 need access to subnets on routers 2,3
which subnets on Router2 need access to subnets on routers 1,3
which subnets on Router3 need access to subnets on routers 1,2

Is the sole admin on Router1?
Will the admin on Router1 travel to Router2 and Router3 from time to time and need config access.
Will the admin have also remote devices (laptop/ipad/smartphone) and may wish to reach all three routers for config purposes or all subnets for some monitoring
Are there any workers that will need wireguard to reach subnets.

If any of the above is true, it should be communicated and helps resolve any issues

The idea is that all the possible clients that have access to one of the LANs should been able to reach each site without the requirement of activating any kind of tunnel or other stuffs.

The admins instead can be client of the network and access all the routers for configurations, they have also the ability to access the whole network from outside connecting directly to one of the routers.

What I’m not able to do is to access from the one of the LANs the other routers!

I’m adding the fact that is I’m using the IP of greatwall 172.16.0.1 to access from a client on the LAN of FortKnox, using the IP of GreatWall and the password of FortKnox I can open a winbox to fortknox and the header is presenting the IP of greatwall. Is a NAT problem, I’m pretty sure, help me to understand…

I’ve been able to solve everything!

It was a routing problem, the table has an entry with gateaway “wg-fireloop” the problem was that it was not specific enough, I was required to change the gateaway with the specific IP of the router of the network.

Good thing you fixed it, because to be frank I don't understand much of what you were trying to communicate. :slight_smile:
I will take a look at the configs to see if there are other possible issues.

1 Like

Thanks in any case for the support!

It is a mess and i like to play with it!

Not that I understand much of your setup/configurations, but if I may, you should use some methods to keep the configurations of those three routers "in-sync".

I quickly checked the configurations you posted on my (shameless plug) Config Compare spreadsheet:

and found the lack of symmetry between the two configs you posted ... inconveniencing.

Yes, to fix this will take hours, and there is too much lacking information as I got mired down.

The idea is simple, 3 routers, 3 LAN, 1 wireguard tunnel, each device will have the other two as peers.
ANY device of any LANs can access ANY other device in any other LANs.

Of course this is the simple version, now i will add all the other little specs.

Maybe is a little bit more clear! In any case sorry for my bad configs but I’m a newbie in the mikrotik world but I’m really interesting!

There are many significant errors in the configs.........
You have a second wireguard interface not mentioned and that complicates any fixes as it was an unknown. For any config help at a detail level, I can work with well stated requirements.

a. identify all user(s)/device(s), including admins
b. identify all traffic needed (and be specific subnet 1 on R1 to subnet 2 on R2 etc.....)

Great wall is wireguard server for handshake but then connects to another router as client??
To make matters worse the endpoint port for that fourth Router ( acting as server for handshake with Great wall as client for handshake uses the same endpoint port your clients use to connect to Great Wall main router.

Nothing you are doing is difficult, but it is complex enough that one has to get the details correct.

In this original configuration that you posted, these were the actual cause of the problem:

  • This address entry on FORTKNOX:

    /ip address
    add address=172.16.0.1/26 interface=wg-fireloop network=172.16.0.0
    

    Here you are assigning the IP address 172.16.0.1 to FORTKNOX, and add a route with destination 172.16.0.0 gateway wg-fireloop. The second part is correct and needed, but the 1st part is wrong, the address 172.16.0.1 should belong to GREATWALL (on its bridge-lan interface). You cannot reach the intended remote 172.16.0.1 when you are inside the LAN of FORTKNOX, because 172.16.0.1 is configured to be the FORTKNOX router.

    The fix here is to remove this entry and replace it with only the route:

    /ip route
    add dst-address=172.16.0.0/26 gateway=wg-fireloop
    
  • Same exact issue on GREATWALL, with this entry:

    /ip address
    add address=172.16.20.1/27 interface=wg-fireloop network=172.16.20.0
    

    With this, beside the needed route, the IP address 172.16.20.1 is assigned to GREATWALL and you can't use it to reach FORTKNOX. The fix is to remove that IP -> Address entry and only add the route:

    /ip route
    add dst-address=172.16.20.0/27 gateway=wg-fireloop