Inter-vlan routing not working on CRS112 (The church campus saga)

Hello

I’m new to working with Mikrotik gear and I’m trying to set up a small campus network for a local church and daycare. My biggest stumbling block right now is inter-vlan routing, I think. I’m sure I’m missing something, but for the life of me I can’t figure out what. If anyone can drop me some ideas, I’d sure appreciate it. I’ve added The church campus saga to the end of the subject line. This may seem pretentious, but I think this is going to the be the first post in a series of posts as I get this entire network together. Hopefully the extra info in the subject line will help tie these posts together. If this is against any type of forum rules, let me know and I’ll remove it. I just thought it might be helpful in the long run.

Now, on to the details…

Setup
I have a Mikrotik CRS112-8G-4S-IN being used as a layer 3 routing switch, with a pfSense firewall. This is the beginning of a larger and more complex network desigh, but I’m running a minimum working example just to try and iron out all the kinks. There are two vLANs, both with an IP address on each vLAN interface, untagged ports for each vLAN, DHCP server, and default route to the pfSense LAN IP address. I’ll post the Mikrotik export below. One vLAN network (99: Mgmt) has the IP with the pfSense firewall, and pfSense does have a correct route back into the LAN.

When setting all this up, I mostly referenced this wiki page

I don’t know if my basic network diagram will embed itself, but it can be found here if not.

What’s Happening
When I set a static address on a laptop using the same network segment as pfSense but using pfSense as the gateway, everything out to the internet works fine. Local routing of course doesn’t quite work properly. The DHCP addresses are set using the local vLAN interface as their gateway. For the Mgmt network, a laptop receiving DHCP can reach pfSense but not the internet. For the other network, a laptop receiving DHCP doesn’t reach the internet or pfSense. I’ve been able to verify that pfSense is not seeing the traffic. Additionally, inter-vlan routing does not seem to work correctly between the two networks. There are no firewall rules on the Mikrotik. And the final kicker is that sometimes it all actually works as it’s supposed to. I just can’t figure out why, when I only change small things incrementally and re-testing those changes never produce the same result. I’m starting to lose my mind, I believe.

I’m sure I haven’t been as clear here as I should have been, so start firing away questions for clarification. I truly hope someone can provide me with some insight. Thanks in advance!

Mikrotik Export

/interface bridge
add admin-mac=CC:2D:E0:4B:C3:0E auto-mac=no name=bridge
/interface vlan
add interface=bridge name=Mgmt vlan-id=99
add interface=bridge name=vLAN101 vlan-id=101
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile
set [ find default=yes ] html-directory=flash/hotspot
/ip pool
add name=dhcp_pool0 ranges=10.10.1.199-10.10.1.200
add name=dhcp_pool1 ranges=10.10.101.50-10.10.101.99
/ip dhcp-server
add address-pool=dhcp_pool0 disabled=no interface=Mgmt lease-time=1d name=dhcp1
add address-pool=dhcp_pool1 disabled=no interface=vLAN101 name=dhcp2
/interface bridge port
add bridge=bridge interface=ether1
add bridge=bridge interface=ether2
add bridge=bridge interface=ether3
add bridge=bridge interface=ether4
add bridge=bridge interface=ether5
add bridge=bridge interface=ether6
add bridge=bridge interface=ether7
add bridge=bridge interface=ether8
add bridge=bridge interface=sfp9
add bridge=bridge interface=sfp10
add bridge=bridge interface=sfp11
add bridge=bridge interface=sfp12
/interface ethernet switch egress-vlan-tag
add tagged-ports=switch1-cpu vlan-id=99
add tagged-ports=switch1-cpu vlan-id=101
/interface ethernet switch ingress-vlan-translation
add customer-vid=0 new-customer-vid=99 ports=ether1
add customer-vid=0 new-customer-vid=99 ports=ether7
add customer-vid=0 new-customer-vid=101 ports=ether5
/interface ethernet switch vlan
add ports=switch1-cpu,ether1,ether7 vlan-id=99
add ports=switch1-cpu,ether5 vlan-id=101
/ip address
add address=10.10.1.254/24 interface=Mgmt network=10.10.1.0
add address=10.10.101.254/24 interface=vLAN101 network=10.10.101.0
/ip dhcp-server network
add address=10.10.1.0/24 dns-server=208.67.220.123,208.67.222.123 gateway=10.10.1.254
add address=10.10.101.0/24 dns-server=208.67.220.123,208.67.222.123 gateway=10.10.101.254
/ip route
add distance=1 gateway=10.10.1.1
/system routerboard settings
set silent-boot=no

OK, here’s how I got it to work. If anyone stumbles across this, I hope it will help. My huge thanks to pcunite for his post " Using RouterOS to vLAN your Network." It was immensely helpful in finding the correct way to do this.

The firewall has a static IP in the same network as interface ethernet1. A default route to this provides routing out to the internet. Each vLAN is established on /interface vlan, untagged ports are added to the bridge with the pvid of their vLAN, tagged ports are added to the bridge with no pvid, and finally, each vLAN is added to /interface bridge vlan specifying the bridge, tagged and untagged ports. To enable inter-vLAN routing, then establish an IP address on each vLAN interface.

Please note that this config works, but is not necessarily secure or includes all of the items that might be necessary in your own environment. You should be able, however, to dump this config onto a CRS112 and get it to function with multiple routed vLANs and pass traffic up to a firewall.

# model = CRS112-8G-4S
/interface bridge
add name=BR1 protocol-mode=none vlan-filtering=yes
/interface vlan
add interface=BR1 name=Guest vlan-id=103
add interface=BR1 name=Office vlan-id=101
add interface=BR1 name=Mgmt vlan-id=99
add interface=BR1 name=Student vlan-id=102
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile
set [ find default=yes ] html-directory=flash/hotspot
/ip pool
add name=dhcp_pool0 ranges=10.10.100.50-10.10.100.100
add name=dhcp_pool1 ranges=10.10.101.50-10.10.101.99
add name=dhcp_pool2 ranges=10.10.102.50-10.10.102.199
add name=dhcp_pool3 ranges=10.10.103.50-10.10.103.199
/ip dhcp-server
add address-pool=dhcp_pool0 disabled=no interface=Mgmt lease-time=1d name=dhcp1
add address-pool=dhcp_pool1 disabled=no interface=Office lease-time=1d name=dhcp2
add address-pool=dhcp_pool2 disabled=no interface=Student lease-time=4h name=dhcp3
add address-pool=dhcp_pool3 disabled=no interface=Guest lease-time=2h name=dhcp4
/interface bridge port
add bridge=BR1 interface=ether8 pvid=99
add bridge=BR1 interface=ether7 pvid=99
add bridge=BR1 interface=ether6 pvid=102
add bridge=BR1 interface=ether5 pvid=103
add bridge=BR1 interface=ether2
add bridge=BR1 interface=ether3
add bridge=BR1 interface=ether4
/interface bridge vlan
add bridge=BR1 tagged=BR1,ether2,ether3,ether4 untagged=ether6 vlan-ids=102
add bridge=BR1 tagged=BR1,ether2,ether3,ether4 untagged=ether5 vlan-ids=103
add bridge=BR1 tagged=BR1,ether1,ether2,ether3,ether4 untagged=ether7,ether8 vlan-ids=99
add bridge=BR1 tagged=BR1,ether2,ether3,ether4 untagged=ether7 vlan-ids=101
/ip address
add address=10.10.1.2/24 interface=ether1 network=10.10.1.0
add address=10.10.101.254/24 interface=Office network=10.10.101.0
add address=10.10.102.254/24 interface=Student network=10.10.102.0
add address=10.10.103.254/24 interface=Guest network=10.10.103.0
add address=10.10.100.254/24 interface=Mgmt network=10.10.100.0
/ip dhcp-server network
add address=10.10.100.0/24 dns-server=208.67.220.123,208.67.222.123 gateway=10.10.100.254
add address=10.10.101.0/24 dns-server=208.67.220.123,208.67.222.123 gateway=10.10.101.254
add address=10.10.102.0/24 dns-server=208.67.220.123,208.67.222.123 gateway=10.10.102.254
add address=10.10.103.0/24 dns-server=208.67.220.123,208.67.222.123 gateway=10.10.103.254
/ip route
add distance=1 gateway=10.10.1.1
/system identity
set name=core-switch-0
/system routerboard settings
set silent-boot=no