Home Lab VLAN/Routing Help

I live in an apartment where the building has a Ruckus AP that does the building’s wifi and has a 1G out (before they used this they used to provide a dumb switch to network to the den, living room and bedroom).

I wanted to upgrade my homelab setup so I am trying to make my life as difficult as possible by having a CRS305 in this small network hovel, that uplinks to a CRS317 I got, to then uplink to my RB5009 I’ve been using.

I acquired 3 mini PCs and have a dual 10G NIC for my NAS system so I can have all these systems have 2 SFP+ uplinks that I can bond, or have 1 be tagged with multiple VLANs for a hypervisor and the other be tagged with a SAN VLAN for iSCSI connections.

I have spent 2 weekends now, doing nothing but failing to appropriately configure VLANs, and I have no clue what I am doing wrong.

I am recreating this 3 RouterOS layout in Cisco’s Modeling Lab and I have no clue what I am doing wrong.

This is my home layout, I feel that This should be very simple, but I cannot make it work for the life of me.
home-layout.drawio.png
RB5009

/interface bridge
add name=rb5009br vlan-filtering=yes
/interface vlan
add interface=rb5009br name=vlan10 vlan-id=10
add interface=rb5009br name=vlan99 vlan-id=99
/interface bridge port
add bridge=rb5009br interface=sfp-plus1
add bridge=rb5009br interface=ether1 pvid=10
/interface bridge vlan
add bridge=rb5009br tagged=sfp-plus1 vlan-ids=10
add bridge=rb5009br tagged=sfp-plus1,rb5009br vlan-ids=99
/ip address
add address=10.0.10.1/24 interface=vlan10 network=10.0.10.0
add address=10.0.99.1/24 interface=vlan99 network=10.0.99.0
/ip route
add distance=5 gateway=10.0.99.3

CRS317

/interface bridge
add name=crs317br vlan-filtering=yes
/interface vlan
add interface=crs317br name=vlan99 vlan-id=99
/interface bridge port
add bridge=crs317br interface=sfp-plus[1-14] pvid=whatever-this-isnt-important
add bridge=crs317br interface=sfp-plus15 
add bridge=crs317br interface=sfp-plus16
/interface bridge vlan 
add bridge=crs317br tagged=sfp-plus15,sfp-plus16 vlan-ids=10
add bridge=crs317br tagged=sfp-plus15,sfp-plus16,crs317br vlan-ids=99
/ip address
add address=10.0.99.2/24 interface=vlan99 network=10.0.99.0

CRS305

/interface bridge
add name=crs305br vlan-filter=yes
/interface vlan
add interface=crs305br name=vlan99 vlan-id=99
/interface bridge port
add bridge=crs305br interface=sfp-plus4 pvid=10
add bridge=crs305br interface=sfp-plus1
/interface bridge vlan
add bridge=crs305br tagged=sfp-plus1 vlan-ids=10
add bridge=crs305br tagged=sfp-plus1,crs305br vlan-ids=99
/ip address
add address=10.0.99.3/24 interface=vlan99 network=10.0.99.0
/ip dhcp-client
add interface=ether1

This is me typing it all out by hand, but as I said, I can’t even get something simple running in CML. I can do frame-types=vlan-tagged-only on the trunk ports but I lose all connection and my windows DHCP on the mini-pcs fail.

In CML if I were to have a pvid on the CHR running as a switch with pvid 10, a linux instance connected to a pvid switch won’t get a DHCP address from the router CHR, and if I set an IP I can’t ping back to the 10.0.10.1 IP that is configured on the router.

My goal is to be able to create vlans as needed on the rb5009 and crs317 to facilitate networks within the hyper-visors, the crs305 doesn’t need to know.

Ideally if I could have wrapped ether1 from the ruckus and just sent it back to be handled by the rb5009 I’d love to do that, however they have a bunch of random tags it seems based on the vlan neighbors in swos when I tried to run that, so all my vlans would have to be 2000+ but I’m not sure how I would appropriately handle however many dozens or hundreds of tags they put on their stuff

You have to add bridge port as tagged member of VLANs with which IP layer of device has to communicate (and you create corresponding vlan interface, anchored off bridge port). In case of RB5009 that would be:

/interface vlan
add interface=rb5009br name=vlan10 vlan-id=10
add interface=rb5009br name=vlan99 vlan-id=99
/interface bridge port
add bridge=rb5009br interface=sfp-plus1
add bridge=rb5009br interface=ether1 pvid=10
/interface bridge vlan
add bridge=rb5009br tagged=sfp-plus1> ,rb5009br > vlan-ids=10
add bridge=rb5009br tagged=sfp-plus1,rb5009br vlan-ids=99

In case of switches, bridge port only has to be member of management VLAN (I guess in your case that’s VLAN 99).

Thanks for the reply, that didn’t seem to help, here is what I have in my lab.
Screenshot 2024-09-25 at 12.42.46.png
R1

/interface bridge
add name=bridge1 vlan-filtering=yes
/interface ethernet
set [ find default-name=ether1 ] disable-running-check=no
set [ find default-name=ether2 ] disable-running-check=no
set [ find default-name=ether3 ] disable-running-check=no
set [ find default-name=ether4 ] disable-running-check=no
/interface vlan
add interface=ether1 name=vlan10 vlan-id=10
add interface=ether1 name=vlan20 vlan-id=20
add interface=ether1 name=vlan99 vlan-id=99
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=vlan10-pool ranges=10.0.10.100-10.0.10.200
add name=vlan20-pool ranges=10.0.20.100-10.0.20.200
/ip dhcp-server
add address-pool=vlan10-pool disabled=no interface=vlan10 name=vlan10-dhcp
add address-pool=vlan20-pool disabled=no interface=vlan20 name=vlan20-dhcp
/interface bridge port
add bridge=bridge1 interface=ether1
add bridge=bridge1 interface=ether2 pvid=20
/interface bridge vlan
add bridge=bridge1 tagged=ether1 vlan-ids=10
add bridge=bridge1 tagged=ether1,bridge1 vlan-ids=20
add bridge=bridge1 tagged=ether1,bridge1 vlan-ids=99
/ip address
add address=10.0.10.1/24 interface=vlan10 network=10.0.10.0
add address=10.0.20.1/24 interface=vlan20 network=10.0.20.0
add address=10.0.99.1/24 interface=vlan99 network=10.0.99.0
/ip dhcp-server network
add address=10.0.10.0/24 dns-server=8.8.8.8 gateway=10.0.10.1
add address=10.0.20.0/24 dns-server=8.8.8.8 gateway=10.0.20.1
/ip route
add distance=3 gateway=10.0.99.3
/system identity
set name=R1

However ubuntu-0 does not get a DHCP address; ubuntu-1 gets an address on vlan 20 as it should, ubuntu-2 is on pvid 10 and gets a vlan10 address. Here is the other configs

SW1

/interface bridge
add name=bridge1 vlan-filtering=yes
/interface ethernet
set [ find default-name=ether1 ] disable-running-check=no
set [ find default-name=ether2 ] disable-running-check=no
set [ find default-name=ether3 ] disable-running-check=no
set [ find default-name=ether4 ] disable-running-check=no
/interface vlan
add interface=bridge1 name=vlan99 vlan-id=99
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/interface bridge port
add bridge=bridge1 interface=ether1
add bridge=bridge1 interface=ether2 pvid=20
add bridge=bridge1 interface=ether3 pvid=20
add bridge=bridge1 interface=ether4
/interface bridge vlan
add bridge=bridge1 tagged=ether1,ether3,ether4 vlan-ids=10
add bridge=bridge1 tagged=ether1,ether4 vlan-ids=20
add bridge=bridge1 tagged=ether1,ether4,bridge1 vlan-ids=99
/ip address
add address=10.0.99.2/24 interface=vlan99 network=10.0.99.0
/system identity
set name=SW1

R2

/interface bridge
add name=bridge1 vlan-filtering=yes
/interface ethernet
set [ find default-name=ether1 ] disable-running-check=no
set [ find default-name=ether2 ] disable-running-check=no
set [ find default-name=ether3 ] disable-running-check=no
set [ find default-name=ether4 ] disable-running-check=no
/interface vlan
add interface=bridge1 name=vlan99 vlan-id=99
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/interface bridge port
add bridge=bridge1 interface=ether2 pvid=20
add bridge=bridge1 interface=ether3 pvid=10
add bridge=bridge1 interface=ether4
/interface bridge vlan
add bridge=bridge1 tagged=ether4 vlan-ids=10
add bridge=bridge1 tagged=ether4 vlan-ids=20
add bridge=bridge1 tagged=ether4,bridge1 vlan-ids=99
/ip address
add address=10.0.99.3/24 interface=vlan99 network=10.0.99.0
/ip dhcp-client
add disabled=no interface=ether1
/ip route
add distance=1 dst-address=10.0.0.0/8 gateway=10.0.99.1
/system identity
set name=R2

I’m also having trouble pinging 8.8.8.8 from R1; I can on R2, I thought I might need 10.0.0.0/8 route back to R1, but I feel that this may just be some kind of misunderstanding on my part of how mikrotik handles routing compared to fgt and cisco firewalls i use at work.

The Ext-connection is a NAT in CML, so I don’t believe I would need to actually create a route on my actual gear for it to know how to get back.

I am an idiot. I had /interface vlans set to ether1 on R1 instead of the bridge, this seems to be working after that change for getting an ip on ubunut-0.

I still have no clue what i’m doing wrong with the routing though

R2

[admin@R2] > /ip route print 
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit 
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 ADS  0.0.0.0/0                          192.168.255.1             1
 1 A S  10.0.0.0/8                         10.0.99.1                 1
 2 ADC  10.0.99.0/24       10.0.99.3       vlan99                    0
 3 A S  172.16.0.0/16                      172.16.0.1                1
 4 ADC  172.16.0.0/24      172.16.0.2      ether1                    0
 5 ADC  192.168.255.0/24   192.168.255.141 ether1                    0
[admin@R2] > /ping 8.8.8.8
  SEQ HOST                                     SIZE TTL TIME  STATUS                                                                                                                         
    0 8.8.8.8                                    56 250 11ms 
    1 8.8.8.8                                    56 250 12ms 
    sent=2 received=2 packet-loss=0% min-rtt=11ms avg-rtt=11ms max-rtt=12ms

R1

[admin@R1] /interface vlan> /ip route print 
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit 
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 A S  0.0.0.0/0                          10.0.99.3                 1
 1 ADC  10.0.10.0/24       10.0.10.1       vlan10                    0
 2 ADC  10.0.20.0/24       10.0.20.1       vlan20                    0
 3 ADC  10.0.99.0/24       10.0.99.1       vlan99                    0
[admin@R1] /interface vlan> /ping 8.8.8.8
  SEQ HOST                                     SIZE TTL TIME  STATUS                                                                                                                         
    0 8.8.8.8                                                 timeout                                                                                                                        
    1 8.8.8.8                                                 timeout                                                                                                                        
    sent=2 received=0 packet-loss=100%

Yeah your default route look wrong. But it’s unclear where the internet go out, so hard to say what to fix… But the route where the internet goes out is what should the be the gateway= (likely on the 10.0.99.x), and the router with upstream internet needs a NAT masquerade rule too.

well it’s a NAT to the CML link so 192.168.255.1 but I took that out and just did a straight IP on R2 ether1 to another CHR and I couldn’t ping from R1 to that either even though the route tables are dog water simple

r1 -
    br1 - 10.0.99.1/24
    172.16.0.0/16 > 10.0.99.3

r2 -
    br1 - 10.0.99.3/24
    ether1 - 172.16.0.1/24
    172.16.0.0/16 > 172.16.0.1/24
    
r3 - 
    ether1 - 172.21.0.2/24
    10.0.0.0/8 > 172.16.0.1

Maybe it is just the masquerade but I would think that this is simple IP routing and doesn’t require firewall configuration

I guess I’m trying to say is all the routers should generally have the same routes… as the return packets need some path back to the source.

All good now?