VLANed home network, 2nd try

Hi there,

The previous topic was quite a mess, due to multiple sequential failing plans and a lot of confusion caused by this. Let’s have a fresh start.

Situation:

A fresh try for my home network. I moved from a Ubiquiti Unifi network, having 4 VLANs

  • VLAN1 : default (trusted network for all clients),
  • guest: unknown VLAN-id
  • VLAN3: iot
  • VLAN4: camera

To a new partially Mikrotik infrastructure. As you can see I don't know the VLAN-id for the Ubiquiti guest wifi. I don't think it's reelevant since we're redesigning everything.

The migration was started a bit too fast since my Ubiquiti switch broke and I decided not to invest into Ubiquiti anymore, instead buying MT.

Current situation:

Currently the network is functioning without VLANs on both MT and Ubiquiti hardware:

My current infrastructure:

  • CCR2004-16G-2S+ (hostname: r1): It's my main router, between my ISP and my network. It's working fine and currently doing:
    • Routing and NAT
    • Wireguard incoming connections
    • DHCP
    • Note: My ISP requires me to have VLAN300 configured on the WAN-side.
  • CRS32CRS328-24P-4S+ (sw-zolder):
    • Main switch: It's the hub in the hub-and-spoke network. Both the router and all other switches are connected to this one. Also some servers and client devices connected directly.
  • hap ac2 (sw-bureau):
    • On my personal desk. Being misued as a switch (currently all ports except for ether5 in a bridge), if I misconfigure it it's not much of a problem, only my wired desktops/laptops are connected.
    • Wifi is turned off.
  • hEXPoE (sw-schuur)
    • It provides switching and PoE to a bunch of outside PoE devices. The hEXPoE itself is inside btw. Also being used as just a switch.
    • No wifi either.
  • 2 Ubiquiti switches
    • (sw-mk-links and sw-mk-rechts). I’ll get MT when these break down.
  • 6 Ubiquiti AP-AC-Pro's.
    • They are serving 4 SSIDs, each for a different Ubiquiti VLAN.

Since there are no VLANs configured everything is on the default VLAN.

I don't mind buying more/new hardware but if I can make thing work with the current set, it would be great.

On all MT devices I've created an OffBridge Port, on the highest port available. Except for "sw-schuur" since I need all available ports.

Current IP plan:

  • r1: 192.168.1.1
  • sw-bureau: 192.168.1.11
  • sw-mk-rechts: 192.168.1.12
  • sw-mk-links: 192.168.1.31
  • sw-schuur: 192.168.1.14

(everything on a /24)

192.168.1.2-25 are reserved for infrastructure/servers/NAS. 192.168.1.100 and above is for regular clients.

All ports on all switches are in 1 bridge. On r1 ether1 is in a bridge (on it’s own, which is a bit silly, I just saw ….) and sfp-sfpplus1+ether14-16 are in their own bridge (bridge_LAN).

New design:

VLANs/IP-ranges:

For now I envision the following VLANs:

  • VLAN10, 192.168.10.0/24 : Standard clients
  • VLAN20, 192.168.20.0/24 : IOT stuff.
  • VLAN30, 192.168.10.0/24 : Camera's. I'd like to have those separated.
  • VLAN40, 192.168.40.0/24 : Guest network
  • VLAN99, 192.168.99.0/24 : Management VLAN. Only for wired access.

On all networks x.x.x.1 should be the router (r1). Since I’m a bit lazy I decided to use /24 for all networks, despite not having 254 hosts on any network at all.

In some future I might add some networks for tinkering et cetera, but for now I'd be very very very happy to get this up and running. My main priorities would be VLANs 10, 20, 30 and 99. Currently, I’ve got a nice hotspot for Ubiquiti’s guest network. Creating a new one has no priority for now.

DHCP functionality will be provided by r1, r1 will also act as a DNS-recursor; requests will be forwarded to an internal pi-hole (192.168.1.2).

Access points should serve VLAN10, VLAN20, VLAN30 and VLAN40, each having it’s own SSID. I prefer to keep VLAN99 wired.

Regarding inter-VLAN communication:

I think there should be no inter-VLAN communication except for communication via my router. I see people creating multiple (virtual) interfaces on devices to make sure these devices are in multiple VLANs, which might be useful for Home-Assistant (being in the standard client vlan, and the iot vlan at the same time). To me it feels more secure to have my Home assistant box on VLAN10, and adding an fw rule (in the router) to allow certain devices in VLAN20 to communicate on certain ports with it. The router should be able to handle this easily.

Topology/connections:

All connections between router/switches/APs should be trunks, since all of these devices will serve multiple VLANs.

Okay that’s the global overview, but how about the combination of ports and VLANs?

There you go:

Ports & VLANs

r1: (192.168.99.1)

  • ether1: Uplink to ISP. Forced to be VLAN300. DHCP
  • sfp-sfpplus1: downlink to sw-zolder: Trunk.
    • (this is a 1 meter DAC-cable)
  • ether16: Offbridge.

sw-zolder: (192.168.99.5)

  • sfp-sfpplus1: uplink to r1: Trunk.
  • ether1: sw-mk-links: Trunk
  • ether2: sw-mk-rechts : Trunk
  • ether3: Reserved for future use: Trunk
  • ether4: AP : Trunk
  • ether6: sw-bureau: Trunk
  • ether9-ether16: Trusted machines: VLAN10
    • Ports 11+12 are an 802.3ad bonding to my NAS
  • ether17: camera: VLAN30
  • ether18: camera: VLAN30
  • ether20: iot: VLAN20
  • ether24: Offbridge.

The rest of the ports are unused. Since the switch is in a trusted location I think having the other ports in VLAN10 is fine.

sw-schuur: (192.168.99.14)

  1. uplink to sw-mk-links: trunk
  2. camera: VLAN30
  3. camera: VLAN30
  4. access point: trunk
  5. iot: VLAN20

sw-bureau: (192.168.99.11)

  1. uplink to sw-zolder: trunk
  2. pc: VLAN10
  3. pc: VLAN10
  4. management: VLAN99
  5. Offbridge port: (also VLAN99?)

sw-mk-links (ubiquiti, 192.168.99.12):

  1. iot: VLAN20
  2. access point: trunk
  3. access point: trunk
  4. downlink to sw-schuur: trunk
  5. empty
  6. empty
  7. empty
  8. uplink to sw-zolder: trunk

sw-mk-rechts(ubiquiti, 192.168.99.13):

  1. uplink to sw-zolder: trunk
  2. empty
  3. access point: trunk
  4. empty
  5. empty
  6. access point: trunk
  7. empty
  8. empty

All devices have the Offbridge port their highest port (except for sw-schuur and the ubiquiti’s).

All trunks will serve all VLANs, since there’s cameras and iot stuff on all APs. The APs will not create an SSID for the management-vlan.

All current configs have been attached to this post (serials are removed, as well as private stuff such as wireguard peers) . I understand having a non-existing gateway on the Offport-network is not going to work, but I just need to be able to connect to the directly-attached machine.

For now I think I’ve done my homework properly and I’ll start writing scripts for creating the proper VLANs on all devices after this (and some coffee :slight_smile: ) .

Tables as suggest by @Buckeye in my previous topic.

Despite this I’m not to sure what to do with the bridge on the devices. If I look at @Buckeye’s examples, he makes the bridge untagged for a VLAN if there’s only untagged ports on that VLAN, and tagged if there’s tagged ports for that VLAN.

I’ll change that in my spreadsheet.

Configs before implementing VLANs

sw-zolder-config.rsc (2.4 KB)

sw-schuur-config.rsc (1.1 KB)

r1-config.rsc (2.4 KB)

sw-bureau-config.rsc (1.4 KB)

Open questions:

  1. How should I treat the bridge in my spreadsheet-overview? Just make it untagged if there are no tagged ports for this specific VLAN in the bridge, and otherwise tagged?

  2. My Ubiquiti APs and use VLAN1 as default VLAN. I think I can’t change this. I’ll keep it for now. Will that become a problem with my current setup?

Todo:

Tonight I’ll create appropriate configs for all devices.

Take your time over this. It is a huge project for starting on vLANs. So rather than try to configure on a per device basis, implement it 1 vLAN at a time.

1 Like

Looking at the examples both @Buckeye and I gave, on the bridge, it is only vLAN 1 which is untagged, because it is the default vLAN, which is not explicitly tagged and its Single Broadcast Domain and DHCP can be allowed to spread anywhere which is not tagged via the bridge. Like you never tag for a basic pre vLAN starter network. The others must be tagged, else they will 'leak' via the bridge, whereas you really want their transition on and off the bridge managed by the largely autogenerated routing table [IP -> Routes]. Others might have a better explanation.

Take your time over this. It is a huge project for starting on vLANs. So rather than try to configure on a per device basis, implement it 1 vLAN at a time.

Yeah this time I’ll try not to choke myself on the workload.

I’ll take the management-VLAN as an example.

Here's a link to the previous thread if others need more context. Moderator: Can you please add a link from the other thread to this thread? Best in OP, but if not there, the last post where the thread was locked.

I am having trouble understanding how those work together. Unifi (by default, and required when adopting) use the untagged connection for management. You can use the untagged connection for an SSID (and I believe for a guest id that will use the same subnet, but will block traffic to "private" address space at the AP L2/L3 layers).

The bridge on the CCR2004-16G-2S+ (r1), hap ac2 (sw-bureau), hEXPoE (sw-schuur) are all currently configured without vlan-filtering enabled, so they are currently "vlan transparent", meaning that the bridge will just relay ethernet frames as is, using only the mac addresses for forwarding decisions, and ignoring the EtherType. In this configuration the bridge will allow vlan tagged frames to pass-through, but will never add or remove vlan tags. This is similar to all "dumb" non-managed switches made in the last 20 years.

The bridge in the CRS32CRS328-24P-4S+ (sw-zolder) switch is configured with vlan-filtering on, but with all bridge ports in the same vlan (the default vlan 1), and with all ports configured as access ports for vlan 1 (pvid=1). These aren't displayed in the standard export, since they are default values, if you export verbose you will see all the settings (and it will be overwhelming).

Because the CRS32CRS328-24P-4S+ (sw-zolder) switch is configured with vlan-filtering on (and the default ingress-filtering when vlan-filtering is enabled), any tagged frames for other vlans will be dropped, so the switch will no longer allow other vlans (tagged) to pass through. So only untagged frames will reach the APs directly attached to the sw-zolder switch (as it is currently configured).

The more general way to set up SSIDs is to have vlans tied to SSID's, e.g. a separate vlan for each SSID. Then you can use the router's firewall to block access to other subnets, but still allow access to the internet, e.g. the IoT devices could be allowed to communicate with each other within the same vlan, and be allowed access to the internet.

Are all of the 4 SSIDs working at this time? Or is the only one working the one associtated with the "main" 192.168.1.0/24 untagged subnet. I did see you mentioned that you had a hot-spot for the guest network.

Well it’s after 11PM over here… but things are starting to work the way they should. I’m quite happy and even a bit proud of myself.

I’ve created a small management VLAN (VLAN99) on which all MT stuff will have a VLAN interface for management.

So I’ve added this to R1:


#######################################
#
# -- Trunk Ports --
#
#######################################

/interface bridge vlan
add bridge=bridge_LAN tagged=sfp-sfpplus1,bridge_LAN vlan-ids=99


# this is currenltly still being done by VLAN1...
#/interface vlan add interface=bridge_LAN name=MGT_VLAN vlan-id=99
#/ip address add address=192.168.99.1/24 interface=MGT_VLAN


# provide DNS for all local networks. 
/ip dns set allow-remote-requests=yes servers="192.168.1.2"

#######################################
# IP Services
#######################################

/interface vlan add interface=bridge_LAN name=MGT_VLAN vlan-id=99
/ip address add interface=MGT_VLAN address=192.168.99.1/24
/ip pool add name=MGT_POOL ranges=192.168.99.100-192.168.99.200
/ip dhcp-server add address-pool=MGT_POOL interface=MGT_VLAN name=MGT_DHCP disabled=no
/ip dhcp-server network add address=192.168.99.0/24 dns-server=192.168.99.1 gateway=192.168.99.1



#######################################
# VLAN Security
#######################################

# Only allow packets with tags over the Trunk Ports
#/interface bridge port
#set bridge=BR1 ingress-filtering=yes frame-types=admit-only-vlan-tagged [find interface=sfp-sfpplus1]



/interface bridge set bridge_LAN vlan-filtering=yes

I think that I’ll break the current default VLAN stuff if I enable… so I’ll do it later on.

# Only allow packets with tags over the Trunk Ports
#/interface bridge port
#set bridge=BR1 ingress-filtering=yes frame-types=admit-only-vlan-tagged [find interface=sfp-sfpplus1]

On sw-zolder:

/interface bridge port
remove [ find interface=ether23 ]

add bridge=bridge_LAN interface=ether23 pvid=99


/interface bridge vlan
add bridge=bridge_LAN tagged=bridge_LAN,sfp-sfpplus1,ether6 vlan-ids=99

#######################################
# IP Addressing & Routing
#######################################

/interface vlan add interface=bridge_LAN name=MGT_VLAN vlan-id=99
/ip address add address=192.168.99.5/24 interface=MGT_VLAN

# The Router's IP this switch will use
/ip route add distance=1 gateway=192.168.99.1

set bridge=bridge_LAN ingress-filtering=yes frame-types=admit-only-untagged-and-priority-tagged [find interface=ether23]


#set bridge=bridge_LAN ingress-filtering=yes frame-types=admit-only-vlan-tagged [find interface=sfp-sfpplus1]
/interface bridge set bridge_LAN vlan-filtering=yes


So I’ve got a trunk on ether6 (I actually locked myself out by forgetting to add ether6, and had to use the OffBridge interface. Really good to have one!), and a tagged port on 23 (might be useful later on).

And on sw-bureau:

/interface bridge port
remove [ find interface=ether2 ]

add bridge=bridge_LAN interface=ether2 pvid=99

/interface bridge vlan
add bridge=bridge_LAN tagged=bridge_LAN,ether1 vlan-ids=99

#######################################
# IP Addressing & Routing
#######################################

# LAN facing Switch's IP address on a BASE_VLAN
/interface vlan add interface=bridge_LAN name=MGT_VLAN vlan-id=99
/ip address add address=192.168.99.11/24 interface=MGT_VLAN

# The Router's IP this switch will use
/ip route add distance=1 gateway=192.168.99.1

set bridge=bridge_LAN ingress-filtering=yes frame-types=admit-only-untagged-and-priority-tagged [find interface=ether2]


#set bridge=bridge_LAN ingress-filtering=yes frame-types=admit-only-vlan-tagged [find interface=sfp-sfpplus1]
/interface bridge set bridge_LAN vlan-filtering=yes

This way the trunks between r1 ←→ sw-zolder ←→ sw-bureau will use either default VLAN or VLAN99.

Connecting a laptop to ether3, I got a regular (default VLAN) IP from r1, when using ether2 I got a VLAN99 ip from R1. Yay!

Thank you @Buckeye and @DuctView for your patience. I’ll move on tomorrow for creating the camera and iot vlan. Then I’ll have to make sure ubiquiti-stuff also works.

I will have a look later but buckeye has a point. By default ubiquiti adores hybrid ports where the management vlan is untagged and the rest of the vlans tagged, and I am talking the port leading to the next smart device down the line. MT is more sane and expects communication between smart device is done over trunk ports ( all tagged ).

Vlan1 ID should not be used for any data or management vlans it simply behind the scenes doing its bridge thing and can be safely ignored. So suggest you use vlan11 for whatever you thought you needed vlan1 for.

Also off bridge port for each MT device approach is a good one to safely configure vlan filtering on bridges.

When you factory reset an MT router and initialize with either "no default configuration" or with the default configuration, your first connection to the device is using untagged traffic. How is this different than Unifi's AP's which start off only responding to untagged traffic, but allow you to change to a management vlan that is tagged? I will agree that most unifi networks do continue to use "hybrid" links to the access points, because you still need to protect your trunk links, and you will at least until MACsec becomes more common.

I prefer to leave the management untagged at home. It is a lot more convenient and I am willing to take the risk for a network that is in my house. It is already a trunk link (by trunk I mean more than one vlan sharing the same wire, probably "Hybrid link" is a more "proper" name, because it explicitly specifies that untagged traffic will be connected to a vlan specified by the pvid. Because it carries multiple vlans, it already needs to be protected, because anything with access to the trunk/hybrid link already has the ability to access any of the vlans it carries, a tagged only trunk only provides protection against the unsophisticated attacker, it's the equivalent of using a non-standard port for RDP.

Since you have a non-critical device to work with (the hap ac2), why don't you use it for learning and playing?

As long as you don't follow @anav's advice to remove all use of vlan 1 and removing an ip address from the bridge itself, switching between vlan-filtering=no and vlan-filtering=yes will have little effect, other than no longer forwarding tagged frames for non-configured vlans. See this thread: Once and for all COMPLETE Offbridge Port setup, specifically Once and for all COMPLETE Offbridge Port setup - #14 by CGGXANNX and Once and for all COMPLETE Offbridge Port setup - #31 by Amm0

  1. Traffic sent from the bridge will be untagged on the internal trunk link to the virtual swich, vlan interface traffic will have a tag applied. That does not imply that you can't send untagged traffic from a vlan interface, but the tag removal is then being done by the "virtual switch" personality of the bridge. My spreadsheet was only a mock-up. It assumed that the bridge was in vlan-aware mode, i.e. vlan-filtering=yes and that frame-types=admit-all. Also, I never set the pvid to a vlan-id that is the same as a tagged vlan on the same port. So for me, if a vlan is marked as U it is assumed the frames sent on that port will be untagged for the vlan, and that the pvid is the same as the vlan id for the row. I first learned vlans on Cisco, and there (at least on the version I learned on), the pvid of the port (the native vlan) also meant that frames for the native vlan would be sent without a tag. On MikroTik, these two are controlled by different parts of the config; the ingress behavior is controlled by a combination of pvid (which specifies what vlan within the switch a received untagged ethernet frame will be associated with), the frame-types (which will control which frames are allowed to be processed), and ingress-filtering (which controls whether ingress frames are pre-filtered to accept only frames for vlans ids which the port is a member of). What controls the tagging of frames sent from a port is the /interface/bridge/vlan line that that has a list of ports that will send the specified vlan-ids as untagged and another list that has the ports the frames will be sent tagged. In Unifi you don't have the ability to easily shoot yourself in the foot, and specify that frames for a specific port will be sent tagged for a vlan, but received untagged for that same vlan. With recent version (7.16 and above) of ROS, the need to manually configure the bridge port in the /interface/bridge/vlan section is greatly diminished; when a vlan interface is created under the bridge, then the bridge will be added as tagged (on the internal "virtual trunk link" between the "virtual switch" and the "routing block's vlan interface") That's all discussed in the RouterOS bridge mysteries explained - General - MikroTik community forum and Question related to "RouterOS bridge mysteries explained" threads.
    But coming from EdgeOS (vyatta) on the Ubiquiti ER-X to ROS on the RB760iGS (hEX S) was not easy for me. It tooks a while to figure out how things are done on ROS. But it also took time to learn EdgeOS when coming from Cisco. But I was overwhelmed by all the details that ROS exposes compared to vyatta. After you understand how things are done in ROS, it makes sense, but ROS has many more knobs to turn than Unifi, and many fewer blade guards. E.g. see this post on Tom Lawrence's forum that I wrote back in Feb 2022 (shortly after my first exposure to ROS).

  2. The Unifi APs really only care about the ethernet being untagged when the access points are adopted. Once they have been adopted, you can change the management vlan to a tagged vlan. And yes, Unifi makes the assumption that vlan 1 will be untagged, and that if you specify a different vlan for management, that it will be tagged (i.e. the AP will then use a vlan subinterface which is tagged). But the access points will work with any untagged vlan, i.e. I have mine connected with a hybrid link on a switch that is using a vlan id other than 1 as the pvid on switch-port connected to the AP. Interally the AP's bridge may be using pvid 1, but that doesn't imply that the other has to be the same vlans. See this for what I mean.

One can use unifi APs as is, it simply a hybrid port setup to them.
The managment vlan, lets say 99, where they unifi or any smart device gets its IP address from, is sent to the unifi as untagged and the rest of the data/user vlans are sent on the port tagged.
This is not rocket science and NO NEED to event think about vlan1, KISS

ex.
port 2 goes to a smart switch
port 2 goes to unifi AP
port 3 goes to home pc.
vlan10 is home vlan
vlan20 is guest vlan
vlan99 is management or base vlan

add bridge=bridge frame-types=admit-only-vlan-tagged interface=ether2  comment="to smart switch"
add bridge=bridge frame-types=admit-all   interface=ether3 pvid=99  comment="hybrid to unifi AP"
add bridge=bridge frame-types=admit-only-untagged-and-prioirity  interface=ether4 pvid=10 \
 comment="home pc"

add bridge=bridge tagged=bridge,ether2  untagged=ether3  vlan-id=99 comment="management vlan"
add bridge=bridge tagged=bridge,ether2,ether3  untagged=ether4 vlan-id=10  comment="home vlan"
add bridge=bridge tagged=bridge,ether2,ether3  vlan-id=20  comment="guest vlan"