VLAN separation using new Bridge VLAN Filtering feature

jahudka · Sun Aug 18, 2019 12:26 pm

Hello, I have a CRS328-24P-4S+ router which I'm using as the central network node for a small network between a couple of apartments. I'm not currently using the SFP+ ports, only the 24 ethernet ports, which are divided into groups of two or three for each apartment. I'd like each apartment to be on a separate subnet. There's a hAP ac2 in each apartment that I'm controlling via CAPsMAN in local forwarding mode; I'd also like to be able to have the hAPs broadcast some of the other apartments' wifi networks wherever it might help coverage. On top of that I have a dedicated management ethernet port and I'd like to restrict management access to all the routers to this port. This is what I've come up with:

Each apartment will have its own VLAN (untagged on the apartments' own ports, tagged on other apartments' ports)
There will be a management VLAN and any management access will be firewalled to be only possible from this VLAN
Obviously the CRS328 will be configured as a NAT, as well as serving DHCP for each VLAN

I made a first-attempt configuration, but then I read the Common Misconfigurations page and decided to give it a do-over. So I put all the ethernet ports into a single bridge, assigned appropriate

pvid

s, defined all the appropriate rules in /interface bridge vlan, added VLAN interfaces on the bridge and configured DHCP on them, and turned bridge VLAN filtering on.. and it almost works the way I need it to :-) everything is hardware offloaded, all the networks have internet access, management access works.. the only thing that I don't like is that the apartments' subnets aren't separated - I can ping apartment B's router (and possibly other devices on apt. B's network) from apartment A.

These are the pertinent parts of /export hide-sensitive:

On the CRS328 (filtered to show only two apartments for brevity):

/interface bridge add name=bridge-lan vlan-filtering=yes

/interface vlan
add interface=bridge-lan name=vlan11 vlan-id=11 # VLAN 11 is management
add interface=bridge-lan name=vlan21 vlan-id=21 # VLAN 21 is apt. A
add interface=bridge-lan name=vlan22 vlan-id=22 # VLAN 22 is apt. B

/ip pool
add name=pool-11 ranges=172.28.11.55-172.28.11.254
add name=pool-21 ranges=172.28.21.55-172.28.21.254
add name=pool-22 ranges=172.28.22.55-172.28.22.254

/ip dhcp-server network
add address=172.28.11.0/24 gateway=172.28.11.1
add address=172.28.21.0/24 gateway=172.28.21.1
add address=172.28.22.0/24 gateway=172.28.22.1

/ip address
add address=172.28.11.1/24 interface=vlan11 network=172.28.11.0
add address=172.28.21.1/24 interface=vlan21 network=172.28.21.0
add address=172.28.22.1/24 interface=vlan22 network=172.28.22.0

/ip dhcp-server
add address-pool=pool-11 disabled=no interface=vlan11 name=server-11
add address-pool=pool-21 disabled=no interface=vlan21 name=server-21
add address-pool=pool-22 disabled=no interface=vlan22 name=server-22

/interface bridge port
add bridge=bridge-lan interface=ether1 pvid=21
add bridge=bridge-lan interface=ether2 pvid=21
add bridge=bridge-lan interface=ether3 pvid=22
add bridge=bridge-lan interface=ether4 pvid=22
add bridge=bridge-lan interface=ether23 pvid=11

/interface bridge vlan
add bridge=bridge-lan tagged="ether1,ether2,ether3,ether4,bridge-lan" untagged="ether23" vlan-ids=11
add bridge=bridge-lan tagged="ether3,bridge-lan" untagged="ether1,ether2" vlan-ids=21
add bridge=bridge-lan tagged="ether1,bridge-lan" untagged="ether3,ether4" vlan-ids=22

/ip firewall filter
add action=accept chain=input protocol=icmp
add action=drop chain=input src-address=!172.28.11.0/24 # disables management access unless it's coming from the management subnet

/ip firewall nat
add action=masquerade chain=srcnat out-interface=ether24 src-address=172.28.11.0/24
add action=masquerade chain=srcnat out-interface=ether24 src-address=172.28.21.0/24
add action=masquerade chain=srcnat out-interface=ether24 src-address=172.28.22.0/24

And this is an excerpt of apt. A's hAP ac2's config (this hAP is connected to ether1 on the CRS328):

/interface bridge
add name=bridge-lan-A
add name=bridge-lan-B

/interface wireless
set wlan1 mode=ap-bridge band=2ghz-b/g/n ssid="Apt A"
set wlan2 mode=ap-bridge band=5ghz-a/n/ac ssid="Apt A"
add name=wlan3 master-interface=wlan1 ssid="Apt B"
add name=wlan4 master-interface=wlan2 ssid="Apt B"

# VLANs the old way, since bridge VLAN filtering on non-CRS3xx devices kills hw offloading
/interface vlan
add interface=ether1 name=vlan11 vlan-id=11
add interface=ether1 name=vlan22 vlan-id=22

/interface bridge port
add bridge=bridge-lan-A interface=ether1
add bridge=bridge-lan-A interface=ether2
add bridge=bridge-lan-A interface=ether3
add bridge=bridge-lan-A interface=ether4
add bridge=bridge-lan-A interface=ether5
add bridge=bridge-lan-A interface=wlan1
add bridge=bridge-lan-A interface=wlan2
add bridge=bridge-lan-B interface=wlan3
add bridge=bridge-lan-B interface=wlan4
add bridge=bridge-lan-B interface=vlan22

/ip address add address=172.28.21.2/24 network=172.28.21.0 interface=bridge-lan-A
/ip dhcp-client add interface=vlan11 disabled=no

Configuration for B is similar. I've intentionally switched back from CAPsMAN for now to keep things simple(r).

With this setup I'm able to /ping 172.28.21.2 from apt. B's hAP (which has a static address 172.28.22.2 and another obtained from the DHCP client on the management subnet, e.g. 172.28.11.253). I should very much like to not be able to ping it :-) any ideas?

tdw · Sun Aug 18, 2019 2:12 pm

Your VLANs have isolated the various apartments ethernet / layer 2 networks, however without firewall rules to prevent forwarding the CRS will be routing traffic between the subnets on those VLANs.

The CRS devices are designed for switching plus the odd service function and a some routing, the CPU is not up to routing high volumes of traffic although the CRS3xx are significantly better than the CRS1xx - you may need a more capable router depending on the available WAN bandwidth.

sindy · Sun Aug 18, 2019 2:14 pm

@little_strawberry, separation of VLANs at L2 level works fine in your setup. What you actually need is a set of IP firewall rules to drop traffic between IP subnets living in the distinct VLANs - or, even better, a set of firewall rules dropping all traffic which is not explicitly allowed. So roughtly it would be something like

chain=input action=accept connection-state=established,related,untracked comment="let the connection tracking do what it is intended for"
chain=input action=drop connection-state=invalid
chain=input action=accept in-interface=your-management-vlan-name comment="allow full access to the CRS itself from anything connected to the management port"
chain=input action=accept protocol=udp dst-port=53 in-interface-list=!WAN comment="allow access to DNS/UDP from anywhere but WAN"
chain=input action=accept protocol=tcp dst-port=53 in-interface-list=!WAN comment="allow access to DNS/TCP from anywhere but WAN"
chain=input action=drop comment="drop the rest"

chain=forward action=fasttrack-connection connection-state=established,related
chain=forward action=accept connection-state=established,related,untracked
chain=forward action=drop connection-state=invalid
chain=forward action=accept in-interface-list=!WAN out-interface-list=WAN comment="allow access to internet from anywhere"
chain=forward action=accept in-interface=your-management-vlan-name comment="allow management connections regardless the destination"
chain=forward action=drop comment="drop the rest"

On the hAP ac², the chain=input would look the same, and no rules in chain=forward would be necessary as the devices would actually not be used as routers.

Other remarks are:

use of the CRS as the uplink router is not a good idea as it is a nice switch but a poor router due to its (relatively) weak CPU. So unless your uplink is less than, say, 20 Mbit/s, it is much better to use one of the hAP ac² for routing.
hardware accelerated bridging only improves performance where there is some traffic between the ports of the same switch. So a cable-connected NAS in an apartment used by cable-connected device in the same apartment, or an IPTV set connected to the uplink via an IPTV VLAN common to all apartments are examples of connections where hw-accelerated bridging makes sense; connection of a device in an apartment to internet via router is an example of a connecion where hw-acceleration has no effect.
don't forget about mac-telnet and mac-winbox, these bypass the IP firewall, so you need to disable it completely or use a dedicated interface list to restrict mac-access to.

jahudka · Sun Aug 18, 2019 3:34 pm

@sindy, @tdw, thanks for your input, this is kind of what I feared - I had hoped it would be possible to separate the networks at the hardware level without involving the CPU, but as soon as IP firewall is in play that won't be the case. My uplink is currently 50Mbps/50Mbps, which is probably going to be above what the router can handle. I'll monitor the network and if it turns out to be an issue, I'll get either another hAP ac2 or a hEX to do the routing and have the CRS328 do just the switching.

sindy · Sun Aug 18, 2019 3:56 pm

I had hoped it would be possible to separate the networks at the hardware level without involving the CPU, but as soon as IP firewall is in play that won't be the case.

I don't think it is anything worth fearing. All the routing, including routing between the subnets in the VLANs, is done by the CPU anyway, so by preventing the inter-subnet routing from happening using IP firewall rules, you actually save the CPU. The firewall rules eat no bread if no packets pass through them, and as you need to protect the internal network from the "filth from the net", you have to use some firewall rules anyway. And the first rule in each chain, action=accept connection-state=established,related, in accord with fasttracking, will handle 95+ % of all traffic regardless from where the connection has been initially originated. So putting the firewall together in a way which, as a "side effect", also prevents routing between the apartments' subnets, will cost you no additional CPU power. Only the attempts of the apartment tenants to reach each other's network will.

If you wanted it hardcore, you could use /interface ethernet switch rule to drop frames carrying unwanted traffic already at ingress, but as these rules would have to be set individually for ports serving each apartment, you would waste your brain power instead of the CPU.

Out of curiosity, would you mind to name the city/region?

jahudka · Sun Aug 18, 2019 4:12 pm

@sindy Prague, Czech Republic

yeah, you're right about saving CPU this way, didn't think of that.. Okay, I'll see how well the CRS can perform and go from there

ordnit · Thu Apr 22, 2021 10:57 am

Hi there,
thank you for this post, I replicated this solution on GNS3 to try to address an issue I have at home.

The configuration I created on the simulator is pretty much the same as the one in the post:
bridge-lan is an interface which "holds" vlan100 and vlan109 sub-interfaces.
Physical interfaces are member of bridge-lan.

The problem I have is that when I try to ping any host in the simulator, it seems as if the router was unable
to process incoming arp frames. Under ip - arp I have no entries and only one "incomplete" for default router.

What I'm missing?

Mikrotik is CHR running 6.48.2

jahudka · Sat Apr 24, 2021 3:47 pm

Hi there,
thank you for this post, I replicated this solution on GNS3 to try to address an issue I have at home.

The configuration I created on the simulator is pretty much the same as the one in the post:
bridge-lan is an interface which "holds" vlan100 and vlan109 sub-interfaces.
Physical interfaces are member of bridge-lan.

The problem I have is that when I try to ping any host in the simulator, it seems as if the router was unable
to process incoming arp frames. Under ip - arp I have no entries and only one "incomplete" for default router.

What I'm missing?

Mikrotik is CHR running 6.48.2

Hi, I'm not sure but I just noticed that in my original post there was a mistake in the

/interface bridge vlan

section - all the VLANs that you want to be able to route must include

bridge-lan

(or whatever your bridge is called) in the

tagged

list, otherwise the VLANs won't ever reach the bridge and cannot be routed. In my original configuration I had mistakenly included the

tagged=

parameter twice, first with

bridge-lan

and second with the name of the interface where the VLAN was supposed to appear tagged. Sorry for the confusion.

sindy · Sat Apr 24, 2021 4:45 pm

This post may explain why the configuration changes mentioned by @jahudka are necessary.

anav · Sat Apr 24, 2021 8:01 pm

From the first post.
/interface vlan
add interface=bridge-lan name=vlan11 vlan-id=21 # VLAN 21 is management
add interface=bridge-lan name=vlan21 vlan-id=21 # VLAN 21 is apt. A
add interface=bridge-lan name=vlan22 vlan-id=22 # VLAN 22 is apt. B

Missing IP address, DHCP server, DHCP server network, IP Pool for vlan 11.

Post an updated config (all parts not some parts) if you have issues.

VLAN separation using new Bridge VLAN Filtering feature

VLAN separation using new Bridge VLAN Filtering feature

Re: VLAN separation using new Bridge VLAN Filtering feature

Re: VLAN separation using new Bridge VLAN Filtering feature

Re: VLAN separation using new Bridge VLAN Filtering feature

Re: VLAN separation using new Bridge VLAN Filtering feature

Re: VLAN separation using new Bridge VLAN Filtering feature

Re: VLAN separation using new Bridge VLAN Filtering feature

Re: VLAN separation using new Bridge VLAN Filtering feature

Re: VLAN separation using new Bridge VLAN Filtering feature

Re: VLAN separation using new Bridge VLAN Filtering feature

Who is online