Need help with VLAN on a CRS125/hEX

Hi Mikrotik Forums! I need your help with VLAN'n. I have little hair left and continue to pull it out.

Over the path month, I have read everything I can about VLAN with Mikrotik (especially on the CRS125's chipset) but fail to have any kind of success making it work with my hEX setup. I hail from the Dell and Cisco IOS world where configuring a VLAN is somewhat different. I understand VLAN in principal on that platform but found out (and read) it's different on the Mikrotik side.

Prior to setting up the switch, I just had everything bridged in one large bucket. I have an ESXi server now, and wish to have different VLAN's going to it in a single trunk port (ether20-ether24). I've allocated ether20-ether24 as I may put ESXi on a secondary or tertiary machine that I have laying around and load balance the VM's I need across those other machines.

The hEX in my home routes traffic for the caps-man datapath bridge, connection to ISP, and lan traffic. It also handles all the typical stuff you'd find in a home router such as DHCP and DNS and does work quite well. Speeds tests from cogeco -> https://cogeco-on.speedtestcustom.com/result/2105b300-358b-11eb-9eb4-8dc6a31190c1. 800/30 is not far from what I pay for (1gig/30mb down) but I'm sure theres something I haven't done right with fasttrack.

ether1 = Cable Modem
ether2 = basement RB962UiGS-5HacT2HnT AP/Switch
ether3 = kitchen RBcAPGi-5acD2nD AP
ether4 = office RB962UiGS-5HacT2HnT AP/Switch
ether5 = Trunk to CRS
sfp1 = trunk (not used)

Configuration for Router/hEX:

dec/03/2020 11:59:20 by RouterOS 6.47.7

model = RB760iGS

/interface bridge
add name=bridge1 protocol-mode=none vlan-filtering=yes
/interface ethernet
set [ find default-name=ether1 ] comment=router name=ether1-wan
set [ find default-name=ether2 ] comment=basement name=ether2-basement-ap
set [ find default-name=ether3 ] comment=kitchen name=ether3-kitchen-ap
set [ find default-name=ether4 ] comment=office name=ether4-office-ap
set [ find default-name=ether5 ] comment=switch name=ether5-switch
/interface vlan
add interface=bridge1 name=vlan10 vlan-id=10
add interface=bridge1 name=vlan20 vlan-id=20
add interface=bridge1 name=vlan30 vlan-id=30
add interface=bridge1 name=vlan99 vlan-id=99
/caps-man datapath
add arp=enabled bridge=bridge1 client-to-client-forwarding=yes
local-forwarding=yes name=dp_bridge
/caps-man security
add authentication-types=wpa2-psk encryption=aes-ccm group-encryption=aes-ccm
group-key-update=5m name=wpa2
/caps-man configuration
add channel.band=2ghz-b/g/n channel.control-channel-width=20mhz country=
"united states3" datapath=dp_bridge distance=indoors hw-retries=4
max-sta-count=255 mode=ap multicast-helper=full name=WIFI-11 security=
wpa2 ssid=WIFI-11
add channel.band=5ghz-a/n/ac channel.control-channel-width=20mhz country=
"united states3" datapath=dp_bridge distance=indoors hw-retries=4
max-sta-count=255 mode=ap multicast-helper=full name=WIFI-5G security=
wpa2 ssid=WIFI-5G
/interface list
add name=wan
add name=vlans
add name=mgmt
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile
set [ find default=yes ] html-directory=flash/hotspot
/ip pool
add name=BASE_POOL ranges=172.16.0.10-172.16.0.254
add name=vpn ranges=172.16.2.2-172.16.2.254
/ip dhcp-server
add address-pool=BASE_POOL disabled=no interface=vlan99 name=BASE_DHCP
/ppp profile
set *FFFFFFFE dns-server=172.16.0.1 local-address=172.16.2.1 remote-address=
vpn use-upnp=no
/user group
set full policy="local,telnet,ssh,ftp,reboot,read,write,policy,test,winbox,pas
sword,web,sniff,sensitive,api,romon,dude,tikapp"
/caps-man access-list
add action=accept allow-signal-out-of-range=10s disabled=yes interface=all
signal-range=-80..120 ssid-regexp=""
add action=reject allow-signal-out-of-range=10s disabled=yes interface=all
signal-range=-120..-81 ssid-regexp=""
add action=accept allow-signal-out-of-range=10s disabled=yes interface=all
signal-range=-80..120 ssid-regexp=""
add action=reject allow-signal-out-of-range=10s disabled=yes interface=all
signal-range=-120..-81 ssid-regexp=""
/caps-man manager
set ca-certificate=auto certificate=auto enabled=yes
/caps-man provisioning
add action=create-dynamic-enabled hw-supported-modes=gn master-configuration=
WIFI-11 name-format=identity
add action=create-dynamic-enabled hw-supported-modes=ac master-configuration=
WIFI-5G name-format=identity
/interface bridge port
add bridge=bridge1 frame-types=admit-only-vlan-tagged ingress-filtering=yes
interface=sfp1
add bridge=bridge1 interface=ether2-basement-ap pvid=99
add bridge=bridge1 interface=ether3-kitchen-ap pvid=99
add bridge=bridge1 interface=ether4-office-ap pvid=99
add bridge=bridge1 frame-types=admit-only-vlan-tagged ingress-filtering=yes
interface=ether5-switch
/ip neighbor discovery-settings
set discover-interface-list=mgmt
/interface bridge vlan
add bridge=bridge1 tagged=bridge1,ether5-switch,sfp1 vlan-ids=10
add bridge=bridge1 tagged=bridge1,ether5-switch,sfp1 vlan-ids=20
add bridge=bridge1 tagged=bridge1,ether5-switch,sfp1 vlan-ids=30
add bridge=bridge1 tagged=bridge1,ether5-switch,sfp1 untagged=
ether2-basement-ap,ether3-kitchen-ap,ether4-office-ap vlan-ids=99
/interface l2tp-server server
set authentication=mschap1,mschap2 enabled=yes use-ipsec=yes
/interface list member
add interface=ether1-wan list=wan
add interface=vlan99 list=vlans
add interface=vlan99 list=mgmt
add interface=vlan20 list=vlans
add interface=vlan30 list=vlans
add interface=vlan10 list=vlans
/ip address
add address=172.16.0.1/24 interface=vlan99 network=172.16.0.0
/ip cloud
set ddns-enabled=yes
/ip dhcp-client
add disabled=no interface=ether1-wan
/ip dhcp-server lease
add address=172.16.0.136 mac-address=00:17:88:19:81:3F server=BASE_DHCP
add address=172.16.0.103 comment="fish tank light" mac-address=
D8:97:60:05:FF:AE server=BASE_DHCP
add address=172.16.0.121 client-id=1:0:4:20:ee:c6:51 mac-address=
00:04:20:EE:C6:51 server=BASE_DHCP
add address=172.16.0.102 comment="fish tank light" mac-address=
D8:97:60:06:25:B8 server=BASE_DHCP
add address=172.16.0.240 client-id=1:1c:ca:e3:78:cc:e2 mac-address=
1C:CA:E3:78:CC:E2 server=BASE_DHCP
/ip dhcp-server network
add address=172.16.0.0/24 dns-server=172.16.0.1 gateway=172.16.0.1
/ip dns
set allow-remote-requests=yes
/ip firewall address-list
add address=172.16.0.0/24 list=allowed_to_router
/ip firewall filter
add action=accept chain=input comment="allow established, related"
connection-state=established,related
add action=accept chain=input comment="allow L2TP VPN (ipsec-esp)"
in-interface=ether1-wan protocol=ipsec-esp
add action=accept chain=input comment="allow Winbox from Internet" dst-port=
8291 in-interface=ether1-wan log=yes log-prefix=EXTRN protocol=tcp
add action=accept chain=input comment="allow L2TP VPN (500,4500,1701/udp)"
dst-port=500,1701,4500 in-interface=ether1-wan protocol=udp
add action=accept chain=input comment="Allow from MGMT VLAN99"
in-interface-list=mgmt
add action=accept chain=input comment="allow icmp" protocol=icmp
add action=drop chain=input comment="drop everything else"
add action=fasttrack-connection chain=forward connection-state=
established,related
add action=accept chain=forward connection-state=established,related
/ip firewall mangle
add action=mark-connection chain=forward comment=
"mark ipsec connections to exclude them from fasttrack" ipsec-policy=
out,ipsec new-connection-mark=ipsec
add action=mark-connection chain=forward comment=
"mark ipsec connections to exclude them from fasttrack" ipsec-policy=
in,ipsec new-connection-mark=ipsec
/ip firewall nat
add action=masquerade chain=srcnat comment="Default masquerade"
out-interface-list=wan
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh address=172.16.0.0/12
set api disabled=yes
set winbox address=0.0.0.0/0
set api-ssl disabled=yes
/ppp secret
add name=iredden profile=default-encryption remote-address=172.16.2.15
service=l2tp
add name=iredden_iphone profile=default-encryption service=l2tp
/system clock
set time-zone-name=America/Toronto
/system identity
set name=router
/tool mac-server
set allowed-interface-list=mgmt
/tool mac-server mac-winbox
set allowed-interface-list=mgmt

Switch CRS125

ether1 = router
ether2-19 = various computers (access ports vlan 99)
ether20-24 = trunk (esxi plugged into ether24 <-> vmnic3 trunk / ether9 <-> vmnic0 (management ip))
sfp1 = trunk (not used)

dec/03/2020 12:06:38 by RouterOS 6.47.7

model = CRS125-24G-1S

/interface bridge
add name=bridge1 protocol-mode=none
/interface vlan
add interface=ether1 name=vlan99 vlan-id=99
/interface list
add name=wan
add name=lan
add name=mgmt
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/interface bridge port
add bridge=bridge1 interface=ether1
add bridge=bridge1 interface=ether2
add bridge=bridge1 interface=ether3
add bridge=bridge1 interface=ether4
add bridge=bridge1 interface=ether5
add bridge=bridge1 interface=ether6
add bridge=bridge1 interface=ether7
add bridge=bridge1 interface=ether8
add bridge=bridge1 interface=ether9
add bridge=bridge1 interface=ether10
add bridge=bridge1 interface=ether11
add bridge=bridge1 interface=ether12
add bridge=bridge1 interface=ether13
add bridge=bridge1 interface=ether14
add bridge=bridge1 interface=ether15
add bridge=bridge1 interface=ether16
add bridge=bridge1 interface=ether17
add bridge=bridge1 interface=ether18
add bridge=bridge1 interface=ether19
add bridge=bridge1 interface=ether20
add bridge=bridge1 interface=ether21
add bridge=bridge1 interface=ether22
add bridge=bridge1 interface=ether23
add bridge=bridge1 interface=ether24
add bridge=bridge1 interface=sfp1
/ip neighbor discovery-settings
set discover-interface-list=mgmt
/interface ethernet switch egress-vlan-tag
add tagged-ports=ether1,sfp1,switch1-cpu vlan-id=99
add tagged-ports=ether1,ether20,ether21,ether22,ether23,ether24,sfp1 vlan-id=
10
add tagged-ports=ether1,ether20,ether21,ether22,ether23,ether24,sfp1 vlan-id=
20
add tagged-ports=ether1,ether20,ether21,ether22,ether23,ether24,sfp1 vlan-id=
30
/interface ethernet switch ingress-vlan-translation
add comment="access ports" customer-vid=0 new-customer-vid=99 ports="ether2,et
her3,ether4,ether5,ether6,ether7,ether8,ether9,ether10,ether11,ether12,eth
er13,ether14,ether15,ether16,ether17,ether18,ether19"
/interface list member
add interface=ether1 list=wan
add interface=ether2 list=lan
add interface=ether3 list=lan
add interface=ether4 list=lan
add interface=ether5 list=lan
add interface=ether6 list=lan
add interface=ether7 list=lan
add interface=ether8 list=lan
add interface=ether9 list=lan
add interface=ether10 list=lan
add interface=ether11 list=lan
add interface=ether12 list=lan
add interface=ether13 list=lan
add interface=ether14 list=lan
add interface=ether15 list=lan
add interface=ether16 list=lan
add interface=ether17 list=lan
add interface=ether18 list=lan
add interface=ether19 list=lan
add interface=ether20 list=lan
add interface=ether21 list=lan
add interface=ether22 list=lan
add interface=ether23 list=lan
add interface=ether24 list=lan
add interface=sfp1 list=lan
add interface=vlan99 list=mgmt
/ip address
add address=172.16.0.2/24 interface=vlan99 network=172.16.0.0
/ip dns
set servers=172.16.0.1
/ip route
add distance=1 gateway=172.16.0.1
/system clock
set time-zone-name=America/Toronto
/system identity
set name=switch
/tool mac-server
set allowed-interface-list=mgmt
/tool mac-server mac-winbox
set allowed-interface-list=mgmt

This seems to work until I issue:
/interface ethernet switch set forward-unknown-vlan=no
... and then everything goes dead. I cannot ping anything including my nas, esxi box, router, Internet, etc ...

Sometimes the switch will completely 'brick' and will have to reset-configuration after I issue a '/interface ethernet switch ingress-vlan-translation add' command. I think thats due to an order thing....

The configuration on my AP's is straight from caps-man mode auto configuration. The next step is to get them onto a trunk port so I can run various vlan's from wifi but one thing at a time. When I set that up as a test, the caps-man wouldn't connect.

Please help!

sorry you are using switch chips, I only use vlan filtering method .

For CRS125: according to manual, config is missing appropriate list of vlan/port membership in /interface ethernet switch vlan … so I guess all VLANs are somehow unknown.

Are you saying it’s somehow a broken implementation?

Whats the performance impact going to bridged vlans?

I’m saying that your configuration is incomplete. And it’s not about bridge vlans, it’s still switch vlans.

Missing something like this

/interface ethernet switch vlan
add ports=ether1,ether20,ether21,ether22,ether23,ether24,sfp1 vlan-id=10 learn=yes
add ports=ether1,ether20,ether21,ether22,ether23,ether24,sfp1 vlan-id=20 learn=yes
add ports=ether1,ether20,ether21,ether22,ether23,ether24,sfp1 vlan-id=30 learn=yes
add ports=ether1,sfp1,switch1-cpu,ether2,ether3,ether4,\ 
                   ether5,ether6,ether7,ether8,ether9,ether10,ether11,\
                   ether12,ether13,ether14,ether15,ether16,ether17,\
                   ether18,ether19 vlan-id=99 learn=yes

Note it is /interface ethernet switch vlan, so no software/bridge stuff here.

Yes, it does seem redundant, but it seems to be mandatory, no shortcuts allowed.

/interface ethernet switch vlan
add ports=ether1,ether20,ether21,ether22,ether23,ether24,sfp1 vlan-id=10 learn=yes
add ports=ether1,ether20,ether21,ether22,ether23,ether24,sfp1 vlan-id=20 learn=yes
add ports=ether1,ether20,ether21,ether22,ether23,ether24,sfp1 vlan-id=30 learn=yes
add ports=ether1,sfp1,switch1-cpu,ether2,ether3,ether4,\ 
                   ether5,ether6,ether7,ether8,ether9,ether10,ether11,\
                   ether12,ether13,ether14,ether15,ether16,ether17,\
                   ether18,ether19 vlan-id=99 learn=yes

I added the above, and I can now access machines on those ports however, from the switch’s interface on vlan99 (which has a 172.16.0.2/24) ip, it can’t access another machine on vlan99 (172.16.0.10). Yet, my PC which is connected through the router on ether4 can access it.

/interface vlan set interface=bridge1 fixed it.