Cannot connect to the Internet after setting up VLAN

RouterOS Beginner Basics
1

Hi all,

In short, I cannot connect to the Internet after setting up VLAN. I am just a beginner. Your help will be appreciated.

Context:
I currently run a k3s homelab cluster behind a opnsense firewall and a CRS326-24G-2S+IN switch WITHOUT VLAN. A https service is exposed to the Internet through port forwarding + NAT reflection/hairpin NAT. Two reverse proxies(traefik) and a DNS is being run by the k3s cluster. Cilium + metalLB with BGP are being used in the k3s network(CNI) setup. Tailscale is used. The opnsense firewall is behind a router+modem from my ISP. I set the router+modem in IP pass-through mode already. My computer across the other room is directly connected to the switch in the rack.

Objective:

  1. Multiple VLANs are setup. And I can fine tune the traffic between them.
  2. Assigned 172.20.0.0/22 to VLAN10. MetalLB will reserve IPs from 172.20.0.0/17 while the physical hosts will use IPs from 172.20.128.0/17 via DHCP.
    Computers in VLAN10 can connect to 172.20.0.0/17 freely. And computers from other VLANs can access 172.20.0.0/17 via individual firewall rules. The firewall’s forward chain rules in my config might explain it better.
  3. A https service is exposed to the Internet through port forwarding via hairpin NAT. Dynamic IP method is preferred.
  4. Keep using Cilium + metalLB with BGP setup.

Sub-Objective:

  1. Add a small switch between my computer and the CRS326-24G-2S+IN switch, so multiple devices can share a cable in another room.
  2. Getting rid of opnsense server and use the CRS326-24G-2S+IN switch as router directly or buying another mikrotik router.

Before investing more money, I would like to try my VLAN setup in mikrotik first.

Thanks a lot to http://forum.mikrotik.com/t/using-routeros-to-vlan-your-network/126489/1 I just have to modify a little part of it for my VLAN setup. After I reset my router, without choosing router or bridge mode in the quick setup, I import my config and here is the export in stage 1:


# aug/04/2023 14:02:55 by RouterOS 7.6
# software id = KI2X-PHFS
#
# model = CRS326-24G-2S+
# serial number = SOME_ID
/interface bridge
add admin-mac=aa:bb:cc:dd:ee:ff auto-mac=no comment=defconf name=BR1 \
    vlan-filtering=yes
/interface vlan
add interface=BR1 name=CAMERA_VLAN vlan-id=30
add interface=BR1 name=GUEST_VLAN vlan-id=20
add interface=BR1 name=IOT_VLAN vlan-id=40
add interface=BR1 name=MGMT_VLAN vlan-id=99
add interface=BR1 name=TRUST_VLAN vlan-id=10
/interface list
add name=WAN
add name=VLAN
add name=MGMT
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile
set [ find default=yes ] html-directory=hotspot
/ip pool
add name=TRUST_POOL ranges=172.20.0.2-172.20.255.254
add name=GUEST_POOL ranges=172.21.0.2-172.21.255.254
add name=CAMERA_POOL ranges=172.22.0.2-172.22.255.254
add name=IOT_POOL ranges=172.23.0.2-172.23.255.254
add name=MGMT_POOL ranges=172.30.0.2-172.30.255.254
/ip dhcp-server
add address-pool=TRUST_POOL interface=TRUST_VLAN name=TRUST_DHCP
add address-pool=GUEST_POOL interface=GUEST_VLAN name=GUEST_DHCP
add address-pool=CAMERA_POOL interface=CAMERA_VLAN name=CAMERA_DHCP
add address-pool=IOT_POOL interface=IOT_VLAN name=IOT_DHCP
add address-pool=MGMT_POOL interface=MGMT_VLAN name=MGMT_DHCP
/port
set 0 name=serial0
/interface bridge port
add bridge=BR1 comment=defconf interface=ether1
add bridge=BR1 comment=defconf frame-types=admit-only-vlan-tagged interface=\
    ether2
add bridge=BR1 comment=defconf frame-types=admit-only-vlan-tagged interface=\
    ether3
add bridge=BR1 comment=defconf frame-types=admit-only-vlan-tagged interface=\
    ether4
add bridge=BR1 comment=defconf frame-types=admit-only-vlan-tagged interface=\
    ether5
add bridge=BR1 comment=defconf frame-types=admit-only-vlan-tagged interface=\
    ether6
add bridge=BR1 comment=defconf frame-types=admit-only-vlan-tagged interface=\
    ether7
add bridge=BR1 comment=defconf interface=ether8
add bridge=BR1 comment=defconf interface=ether9
add bridge=BR1 comment=defconf interface=ether10
add bridge=BR1 comment=defconf interface=ether11
add bridge=BR1 comment=defconf interface=ether12
add bridge=BR1 comment=defconf interface=ether13
add bridge=BR1 comment=defconf interface=ether14
add bridge=BR1 comment=defconf interface=ether15
add bridge=BR1 comment=defconf interface=ether16
add bridge=BR1 comment=defconf interface=ether17
add bridge=BR1 comment=defconf interface=ether18
add bridge=BR1 comment=defconf interface=ether19
add bridge=BR1 comment=defconf interface=ether20
add bridge=BR1 comment=defconf interface=ether21
add bridge=BR1 comment=defconf interface=ether22
add bridge=BR1 comment=defconf interface=ether23
add bridge=BR1 comment=defconf interface=ether24
add bridge=BR1 comment=defconf frame-types=admit-only-vlan-tagged interface=\
    sfp-sfpplus1
add bridge=BR1 comment=defconf interface=sfp-sfpplus2
/interface bridge vlan
add bridge=BR1 tagged=\
    BR1,ether2,ether3,ether4,ether5,ether6,ether7,sfp-sfpplus1 vlan-ids=10
add bridge=BR1 tagged=\
    BR1,ether2,ether3,ether4,ether5,ether6,ether7,sfp-sfpplus1 vlan-ids=20
add bridge=BR1 tagged=\
    BR1,ether2,ether3,ether4,ether5,ether6,ether7,sfp-sfpplus1 vlan-ids=30
add bridge=BR1 tagged=\
    BR1,ether2,ether3,ether4,ether5,ether6,ether7,sfp-sfpplus1 vlan-ids=40
add bridge=BR1 tagged=\
    BR1,ether2,ether3,ether4,ether5,ether6,ether7,sfp-sfpplus1 vlan-ids=99
/interface list member
add interface=ether1 list=WAN
add interface=MGMT_VLAN list=VLAN
add interface=TRUST_VLAN list=VLAN
add interface=GUEST_VLAN list=VLAN
add interface=CAMERA_VLAN list=VLAN
add interface=IOT_VLAN list=VLAN
add interface=MGMT_VLAN list=MGMT
/ip address
add address=172.30.0.1/22 interface=MGMT_VLAN network=172.30.0.0
add address=172.20.0.1/22 interface=TRUST_VLAN network=172.20.0.0
add address=172.21.0.1/22 interface=GUEST_VLAN network=172.21.0.0
add address=172.22.0.1/22 interface=CAMERA_VLAN network=172.22.0.0
add address=172.23.0.1/22 interface=IOT_VLAN network=172.23.0.0
/ip dhcp-client
add comment="to connect RG" interface=BR1
/ip dhcp-server network
add address=172.20.0.0/22 dns-server=172.20.0.1 gateway=172.20.0.1
add address=172.21.0.0/22 dns-server=172.21.0.1 gateway=172.21.0.1
add address=172.22.0.0/22 dns-server=172.22.0.1 gateway=172.22.0.1
add address=172.23.0.0/22 dns-server=172.23.0.1 gateway=172.23.0.1
add address=172.30.0.0/22 dns-server=172.30.0.1 gateway=172.30.0.1
/ip dns
set allow-remote-requests=yes servers=9.9.9.9
/ip firewall filter
add action=accept chain=input comment="Allow Estab & Related" \
    connection-state=established,related
add action=accept chain=input comment="Allow VLAN" in-interface-list=VLAN
add action=accept chain=input comment="Allow MGMT_VLAN Full Access" \
    in-interface=MGMT_VLAN
add action=accept chain=forward comment="Allow Estab & Related" \
    connection-state=established,related
add action=drop chain=forward comment="Drop Camera from Internet" \
    in-interface=CAMERA_VLAN out-interface-list=WAN
add action=accept chain=forward comment="VLAN Internet Access only" \
    connection-state=new in-interface-list=VLAN out-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="Default masquerade" \
    out-interface-list=WAN
/system clock
set time-zone-name=America/Los_Angeles
/system identity
set name=router
/system routerboard settings
set boot-os=router-os

Most of the things look ok so far, but I noticed my computer’s IP is still in the subnet of router+modem. I guess it is because I only has one switch and the above config only set ether2-7 as trunk. So I tried to apply the following config to simulate a “router and switch environment”. (My computer is connected via ether15). Here is the export in stage 2:


# aug/04/2023 14:26:56 by RouterOS 7.6
# software id = KI2X-PHFS
#
# model = CRS326-24G-2S+
# serial number = SOME_ID
/interface bridge
add admin-mac=aa:bb:cc:dd:ee:ff auto-mac=no comment=defconf name=BR1 \
    vlan-filtering=yes
/interface vlan
add interface=BR1 name=CAMERA_VLAN vlan-id=30
add interface=BR1 name=GUEST_VLAN vlan-id=20
add interface=BR1 name=IOT_VLAN vlan-id=40
add interface=BR1 name=MGMT_VLAN vlan-id=99
add interface=BR1 name=TRUST_VLAN vlan-id=10
/interface list
add name=WAN
add name=VLAN
add name=MGMT
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile
set [ find default=yes ] html-directory=hotspot
/ip pool
add name=TRUST_POOL ranges=172.20.0.2-172.20.255.254
add name=GUEST_POOL ranges=172.21.0.2-172.21.255.254
add name=CAMERA_POOL ranges=172.22.0.2-172.22.255.254
add name=IOT_POOL ranges=172.23.0.2-172.23.255.254
add name=MGMT_POOL ranges=172.30.0.2-172.30.255.254
/ip dhcp-server
add address-pool=TRUST_POOL interface=TRUST_VLAN name=TRUST_DHCP
add address-pool=GUEST_POOL interface=GUEST_VLAN name=GUEST_DHCP
add address-pool=CAMERA_POOL interface=CAMERA_VLAN name=CAMERA_DHCP
add address-pool=IOT_POOL interface=IOT_VLAN name=IOT_DHCP
add address-pool=MGMT_POOL interface=MGMT_VLAN name=MGMT_DHCP
/port
set 0 name=serial0
/interface bridge port
add bridge=BR1 comment=defconf interface=ether1
add bridge=BR1 comment=defconf frame-types=admit-only-vlan-tagged interface=\
    ether2
add bridge=BR1 comment=defconf frame-types=admit-only-vlan-tagged interface=\
    ether3
add bridge=BR1 comment=defconf frame-types=admit-only-vlan-tagged interface=\
    ether4
add bridge=BR1 comment=defconf frame-types=admit-only-vlan-tagged interface=\
    ether5
add bridge=BR1 comment=defconf frame-types=admit-only-vlan-tagged interface=\
    ether6
add bridge=BR1 comment=defconf frame-types=admit-only-vlan-tagged interface=\
    ether7
add bridge=BR1 comment=defconf interface=ether8
add bridge=BR1 comment=defconf interface=ether9
add bridge=BR1 comment=defconf interface=ether10
add bridge=BR1 comment=defconf interface=ether11
add bridge=BR1 comment=defconf interface=ether12
add bridge=BR1 comment=defconf interface=ether13
add bridge=BR1 comment=defconf interface=ether14
add bridge=BR1 comment=defconf frame-types=\
    admit-only-untagged-and-priority-tagged interface=ether15 pvid=10
add bridge=BR1 comment=defconf interface=ether16
add bridge=BR1 comment=defconf interface=ether17
add bridge=BR1 comment=defconf interface=ether18
add bridge=BR1 comment=defconf interface=ether19
add bridge=BR1 comment=defconf interface=ether20
add bridge=BR1 comment=defconf interface=ether21
add bridge=BR1 comment=defconf interface=ether22
add bridge=BR1 comment=defconf interface=ether23
add bridge=BR1 comment=defconf interface=ether24
add bridge=BR1 comment=defconf frame-types=admit-only-vlan-tagged interface=\
    sfp-sfpplus1
add bridge=BR1 comment=defconf interface=sfp-sfpplus2
/interface bridge vlan
add bridge=BR1 tagged=\
    BR1,ether2,ether3,ether4,ether5,ether6,ether7,sfp-sfpplus1 vlan-ids=10
add bridge=BR1 tagged=\
    BR1,ether2,ether3,ether4,ether5,ether6,ether7,sfp-sfpplus1 vlan-ids=20
add bridge=BR1 tagged=\
    BR1,ether2,ether3,ether4,ether5,ether6,ether7,sfp-sfpplus1 vlan-ids=30
add bridge=BR1 tagged=\
    BR1,ether2,ether3,ether4,ether5,ether6,ether7,sfp-sfpplus1 vlan-ids=40
add bridge=BR1 tagged=\
    BR1,ether2,ether3,ether4,ether5,ether6,ether7,sfp-sfpplus1 vlan-ids=99
/interface list member
add interface=ether1 list=WAN
add interface=MGMT_VLAN list=VLAN
add interface=TRUST_VLAN list=VLAN
add interface=GUEST_VLAN list=VLAN
add interface=CAMERA_VLAN list=VLAN
add interface=IOT_VLAN list=VLAN
add interface=MGMT_VLAN list=MGMT
/ip address
add address=172.30.0.1/22 interface=MGMT_VLAN network=172.30.0.0
add address=172.20.0.1/22 interface=TRUST_VLAN network=172.20.0.0
add address=172.21.0.1/22 interface=GUEST_VLAN network=172.21.0.0
add address=172.22.0.1/22 interface=CAMERA_VLAN network=172.22.0.0
add address=172.23.0.1/22 interface=IOT_VLAN network=172.23.0.0
/ip dhcp-client
add comment="to connect RG" interface=BR1
/ip dhcp-server network
add address=172.20.0.0/22 dns-server=172.20.0.1 gateway=172.20.0.1
add address=172.21.0.0/22 dns-server=172.21.0.1 gateway=172.21.0.1
add address=172.22.0.0/22 dns-server=172.22.0.1 gateway=172.22.0.1
add address=172.23.0.0/22 dns-server=172.23.0.1 gateway=172.23.0.1
add address=172.30.0.0/22 dns-server=172.30.0.1 gateway=172.30.0.1
/ip dns
set allow-remote-requests=yes servers=9.9.9.9
/ip firewall filter
add action=accept chain=input comment="Allow Estab & Related" \
    connection-state=established,related
add action=accept chain=input comment="Allow VLAN" in-interface-list=VLAN
add action=accept chain=input comment="Allow MGMT_VLAN Full Access" \
    in-interface=MGMT_VLAN
add action=accept chain=forward comment="Allow Estab & Related" \
    connection-state=established,related
add action=drop chain=forward comment="Drop Camera from Internet" \
    in-interface=CAMERA_VLAN out-interface-list=WAN
add action=accept chain=forward comment="VLAN Internet Access only" \
    connection-state=new in-interface-list=VLAN out-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="Default masquerade" \
    out-interface-list=WAN
/system clock
set time-zone-name=America/Los_Angeles
/system identity
set name=router
/system routerboard settings
set boot-os=router-os

Unfortunately, my computer cannot reach the Internet after the above patch. When I ping google.com, I get “Destination unreachable: Address unreachable”.

So here are my questions:

  1. Is my VLAN setup correct? Or do I really need a router and a switch to test the config in http://forum.mikrotik.com/t/using-routeros-to-vlan-your-network/126489/1
  2. Suppose I buy another switch and installed it between my computer and the original switch, it must be a managed switch right? Will the hEX lite enough? Out of curiosity, does the new switch must be mikrotik switch? I also consider CSS610-8G-2S+IN too for my future 10G upgrade, but I want to prove that I handle this mikrotik setup first.
  3. Suppose my camera VLAN traffic (VLAN30) is around 100-134 MB/s from a few devices, so each link’s traffic is around 34MB/s. Since the camera is in its own VLAN (VLAN30) and the servers which interference/store/compute are in VLAN10. Does that traffic considered as routing traffic? My original plan is assigning multiple NICs for the server to handle both VLANs, but it seems k3s doesn’t support multiple NICs unfortunately.
  4. If the traffic is considered as routing traffic. The test results of CRS326-24G-2S+IN in 25 simple queues (1518 byte) is 1266.6 Mbps or 158 MB/s, but that’s for all ports, right. Does it mean each port can only do 7MB/s for routing traffic? Or the switch can handle all traffic under 158 MB/s in only a few ports.
  5. Will my Cilium + metalLB with BGP’s setup affect my VLAN setup?

I’ve tried it couple days already, any help will be appreciate. Thanks for your time.

2

Please provide the standard config view.

/export file=anynameyouwish (minus router serial number and any public WANIP information)

3

Thanks for the tips. I thought the rsc format is also acceptable. I already edited my original post and replace the rsc config with the standard export config format. Let me know if I need to provide more information. Thanks.