VLAN + wireguard connectivity

Hi all,

We have been working for a couple of weeks now to make a set-up where we have a virtual network that is accessible on the local and remote site. After many, many, iterations we are out of options/clues to solve this issue. Really hope one of you can help us.

System architecture:

The local site consists of a simple CHR.
The remote site is a RB5009 with an cAP-ax connected to eth8.
The cAP-ax is configured as ‘bridge’, hence all the routing and DHCP is done on the RB5009
These is a Wireguard tunnel between the CHR and RB5009, on network 10.123.0.1/24, with the Wireguard on the CHR being 10.123.0.1 and on the RB5009 10.123.0.2
The local network is 192.168.130.100/24
The remote is 192.168.131.1/24 on the bridge of the RB5009
VLAN 20 on 192.168.132.1/24 (tagged on eth8)
VLAN 80 (guest) on 192.168.133.1/24 (tagged on eth8)

The problem:

From the bridge on the RB5009 (network 192.168.131.1/24) we can access the local/CHR Wireguard and the network with it’s clients perfectly, as it is supposed to be.
But from VLAN20 (e.g. WLAN client connected to the cAP-ax) we can not. ← this is the problem

Without the VLAN’s in place it actually worked well. So we expect the issue is related to the VLAN configuration.

All the config files are attached (passwords, serials, wireguard keys and WAN ip’s removed).

We really, really, hope someone can give a clue in the right direction to solve this issue. It must be something simple, but unfortunately we where not able to spot it.
20240922_CHR.rsc (2.36 KB)
20240922_AP.rsc (2.96 KB)
20240922_RB5009.rsc (7.49 KB)
arch.drawio.png

Treat the capac as an AP not a router.........
Assuming vlan20 is the trusted subnet.
There is a clue in the error in your bridge port settings *A
Assuming ether1 is trunk port to Router, will use ether2 as an off bridge backup or easy management access.
Remove the IP dns static setting...... /ip dns static add address=192.168.88.1 comment=defconf name=router.lan
Missing address of the AP, in this case on the trusted vlan20, will assuming its .5

EDIT: seeing on the 5009 that most likely vlan20 is not the trusted subnet but instead the 131 subnet which I will call vlan10 and make the necessary changes below.
......................

model = cAPGi-5HaxD2HaxD

/interface ethernet
set [ find default-name=ether2 ] name=OffBridge2

/interface bridge
add admin-mac=D4:01:C3:F5:56:BA auto-mac=no comment=defconf name=bridge
vlan-filtering=yes
/interface wifi
set [ find default-name=wifi1 ] channel.band=5ghz-ax .skip-dfs-channels=
10min-cac .width=20/40/80mhz configuration.country=Netherlands .mode=ap
.ssid=WLAN_5GHz disabled=no security.authentication-types=wpa2-psk .ft=
yes .ft-over-ds=yes
set [ find default-name=wifi2 ] channel.band=2ghz-ax .skip-dfs-channels=
10min-cac .width=20/40mhz configuration.country=Netherlands .mode=ap
.ssid=WLAN_24GHz disabled=no security.authentication-types=wpa2-psk .ft=
yes .ft-over-ds=yes

/interface vlan
add interface=bridge name=vlan10 vlan-id=10

/interface list
add name=TRUSTED

/interface list members
add interface=vlan10 list=TRUSTED
add interface=OFFbridge2 list=TRUSTED

/interface wifi security
add authentication-types=wpa2-psk disabled=no management-protection=disabled
name=guest
/interface wifi
add configuration.mode=ap .ssid=WLAN_Guest_24GHz datapath.client-isolation=
yes disabled=no mac-address=D6:01:C3:F5:56:BC master-interface=wifi2
name=guest_wlan security=guest
/disk settings
set auto-media-interface=bridge auto-media-sharing=yes auto-smb-sharing=yes

/interface bridge port
add bridge=bridge ingress-filtering=yes frame-types=admit-vlan-tagged interface=ether1 comment="Trunk port to Router"
add bridge=bridge ingress-filtering=yes frame-types=admit-priority-and-untagged interface=wifi1 pvid=20
add bridge=bridge ingress-filtering=yes frame-types=admit-priority-and-untagged interface=wifi2 pvid=20
add bridge=bridge ingress-filtering=yes frame-types=admit-priority-and-untagged interface=guest_wlan pvid=80
/ip neighbor discovery-settings
set discover-interface-list=TRUSTED

/ip address
add address=192.168.131.5/24 interface=vlan10 network=192.168.131.0

add address=192.168.55.1/30 interface=OffBridge2 network=192.168.55.0
/ipv6 settings
set disable-ipv6=yes

/interface bridge vlan
add bridge=bridge tagged=bridge,ether1 vlan-ids=10
add bridge=bridge tagged=ether1 untagged=wifi1,wifi2 vlan-ids=20
add bridge=bridge tagged=ether1 untagged=guest_wlan vlan-ids=80

/ip dns
add server=192.168.131.1
/ip route
add dst-address=0.0.0.0/0 gateway=192.168.131.1 routing-table=main
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set api disabled=yes
set api-ssl disabled=yes
/ipv6 firewall address-list
add address=2001:db8::/32 comment="defconf: documentation" list=bad_ipv6
add address=2001:10::/28 comment="defconf: ORCHID" list=bad_ipv6
/system identity
set name=ClientAP
/system note
set show-at-login=no
/system routerboard mode-button
set enabled=yes on-event=dark-mode
/system script
add comment=defconf dont-require-permissions=no name=dark-mode owner=*sys
policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon
source="\r
\n :if ([system leds settings get all-leds-off] = "never") do={\r
\n /system leds settings set all-leds-off=immediate \r
\n } else={\r
\n /system leds settings set all-leds-off=never \r
\n }\r
\n "

/tool mac-server
set allowed-interface-list=NONE

/tool mac-server mac-winbox
set allowed-interface-list=TRUSTED

/tool romon
set enabled=yes id=00:00:00:00:00:03

/system ntp client
add address=192.168.131.1

++++++++++++++++++++++++++++++++++++++++++++++

5009

  1. You have three pools and only two vlans, once you go vlans its better to go all vlans and not mix apples and oranges......
    This also makes me think that vlan20 is not the trusted subnet and its really the 192.168.131 subnet, so will go back and change the CAP above
    to reflect this..............

/interface vlan
add interface=bridge name=vlan10 vlan-id=10
add interface=bridge name=vlan20 vlan-id=20
add interface=bridge name=vlan80 vlan-id=80

/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
add name=TRUSTED

/interface bridge port
add bridge=bridge ingress-filtering=yes frame-types=admit-priority-and-untagged interface=ether2
add bridge=bridge ingress-filtering=yes frame-types=admit-priority-and-untaggedinterface=ether3
add bridge=bridge ingress-filtering=yes frame-types=admit-priority-and-untagged interface=ether4
add bridge=bridge ingress-filtering=yes frame-types=admit-priority-and-untagged interface=ether5
add bridge=bridge ingress-filtering=yes frame-types=admit-priority-and-untagged interface=ether6
add bridge=bridge ingress-filtering=yes frame-types=admit-priority-and-untagged interface=ether7
add bridge=bridge ingress-filtering=yes frame-types=admit-only-vlan-tagged interface=ether8 comment="Trunk Port to Cap"
add bridge=bridge ingress-filtering=yes frame-types=admit-priority-and-untagged interface=sfp-sfpplus1

/ip neighbor discovery-settings
set discover-interface-list=TRUSTED

/interface list member
add comment=defconf interface=ether1 list=WAN
add interface=vlan10 list=LAN
add interface=vlan20 list=LAN
add interface=vlan80 list=LAN
add interface=wireguard_Remote list=LAN
add interface=vlan10 list=TRUSTED

add interface=wireguard_Remote list=TRUSTED

Note: assuming one of your requirements is for the admin to be able to remotely connect to the CHR, the 5009 and the Access point.
For example, if that is the case then one would add the wireguard interface to the trusted List.

  1. Wireguard Peer settings appear to be bang on!.

  2. /ip address
    add address=192.168.131.1/24 interface=vlan10 network=192.168.131.0
    add address=192.168.132.1/24 interface=vlan20 network=192.168.132.0
    add address=192.168.133.1/24 interface=vlan80 network=192.168.133.0
    add address=10.0.123.2/24 interface=wireguard_Remote network=10.0.123.0

  3. Remove this old default static setting.
    /ip dns static
    add address=192.168.88.1 comment=defconf name=router.lan

  4. I dont see a manual route for the MAIN ISP, so assuming you have default route selected in IP DHCP client settings.

  5. /tool mac-server
    set allowed-interface-list=NONE
    /tool mac-server mac-winbox
    set allowed-interface-list=TRUSTED

NOW For firewall rules: First thing is to organize it, keep chains together for ease of reading and troubleshooting.

/ip firewall filter
add action=accept chain=input connection-state=established,related,untracked
add action=drop chain=input connection-state=invalid
add action=accept chain=input protocol=icmp
add action=accept chain=input dst-port=13231 protocol=udp
add action=accept chain=input comment="admin access" in-interface-list=TRUSTED
add action=accept chain=input comment="user access to services" in-interface-list=LAN dst-port=53,123 protocol=udp
add action=accept chain=input comment="user access to services" in-interface-list=LAN dst-port=53 protocol=tcp
add action=drop chain=input comment="drop all else"
{ add as last rule on the input chain after other rules in place or will lock yourself out }

(Note1: consider adding an off bridge port for backup configuration/access similar to cap)
(Note2: If you want to limit access to the router within VLAN10 ( aka limit to only admin as one should ) or not all wireguard remote users, then create a firewall address
/ip firewall address-list
add address=10.0.123.X list=Authorized comment="remote admin laptop"
add address=10.0.123.Y list=Authorized comment="remote admin smartphone"
add address=192.168.130.Z list=Authorized comment="admin on chr router lan"
add address=192.168.131.A list=Authorized comment="admin on local 5009"

add action=accept chain=input comment="admin access" src-address-list=Authorized

NOW for the forward chain:

add action=fasttrack-connection chain=forward connection-state=established,related
add action=accept chain=forward connection-state=established,related,untracked
add action=drop chain=forward connection-state=invalid
add action=accept chain=forward comment="internet traffic" in-interface-list=LAN out-interface-list=WAN
add action=accept chain=forward comment="remote wg users to LAN" in-interface=wireguard_Remote out-interface-list=LAN dst-address=!192.168.133.0/24
add action=accept chain=forward comment="access CHR" in-inteface-list=LAN out-interface-list=wireguard_Remote src-address=!192.168.133.0/24
add action=accept chain=forward comment="port forwarding" connection-nat-state=dstnat { disable or remove if not required }
add action=drop chain=forward comment="drop all else"

CHR

  1. Missing local network…in address
    /ip address
    add address=10.0.123.1/24 interface=wireguard_CHR network=10.0.123.0
    add address=192.168.130.1/24 interface=bridge network=192.168.130.0

  2. Adjust, simplify NAT, too messy…
    /ip firewall nat
    add action=masquerade chain=srcnat out-interface=ether1

  3. Routes missing one, should be:
    /ip route
    add dst-address=192.168.131.0/24 gateway=wireguard_CHR routing-table=main
    add dst-address=192.168.132.0/24 gateway=wireguard_CHR routing-table=main

  4. Routing Rules… basic I guess…
    /ip firewall filter
    add action=reject chain=forward dst-address=192.168.130.1 in-interface=
    wireguard_CHR reject-with=icmp-network-unreachable
    add action=accept chain=forward dst-address=192.168.131.0/24 src-address=
    192.168.130.0/24
    add action=accept chain=forward dst-address=192.168.132.0/24 src-address=
    192.168.130.0/24
    add action=accept chain=forward dst-address=192.168.130.0/24 src-address=
    192.168.131.0/24
    add action=accept chain=forward dst-address=192.168.132.0/24 src-address=
    192.168.131.0/24
    add action=accept chain=forward dst-address=192.168.130.0/24 src-address=
    192.168.132.0/24
    add action=accept chain=forward dst-address=192.168.131.0/24 src-address=
    192.168.132.0/24

/ip firewall address-list
add address=10.0.123.X list=Authorized comment=“remote admin laptop”
add address=10.0.123.Y list=Authorized comment=“remote admin smartphone”
add address=192.168.130.Z list=Authorized comment=“admin on chr router lan”
add address=192.168.131.A list=Authorized comment=“admin on local 5009”

/ip firewall filter
add action=accept chain=input connection-state=established,related,untracked
add action=drop chain=input connection-state=invalid
add action=accept chain=input protocol=icmp
add action=accept chain=input dst-port=51888 protocol=udp
add action=accept chain=input comment=“admin access” src-address-list=Authorized
add action=accept chain=input comment=“user access to services” src-address=192.168.130.0/24 dst-port=53,123 protocol=udp
add action=accept chain=input comment=“user access to services” src-address=192.168.130.0/24 dst-port=53 protocol=tcp
add action=drop chain=input comment=“drop all else” { add as last rule on the input chain after other rules in place or will lock yourself out }
++++++++++++++++++++++++++++++
add action=fasttrack-connection chain=forward connection-state=established,related
add action=accept chain=forward connection-state=established,related,untracked
add action=drop chain=forward connection-state=invalid
add action=accept chain=forward comment=“internet traffic” src-address=192.168.130.0/24 out-interface-list=WAN
add action=accept chain=forward comment=“local access to 5009” src-address=192.168.130.0/24 out-interface-list=wireguard_CHR
add action=accept chain=forward comment=“remote wg users to local LAN” in-interface=wireguard_CHR dst-address=192.168.130.0/24
add action=accept chain=forward comment=“RELAY remote users to 5009” in-interface=wireguard_CHR out-interface=wireguard CHR
add action=accept chain=forward comment=“port forwarding” connection-nat-state=dstnat { disable or remove if not required }
add action=drop chain=forward comment=“drop all else”

Thank you so much @anav for this very elaborate answer! I am going to study on it, and then implement and test this. Will let you know the outcome. Thanks again.