InterVLAN Routing

RouterOS Beginner Basics
1

Hi, I’ve had a MikroTik RB4011iGS+ for a while now, but I’m finally looking at putting it to good use as my main router. It’s replacing a PFSense router. I’ve learned a lot about subtle quirks of this router’s configuration, limitations of the switch chip with VLANs, etc.

I think I have everything working in the current config except one thing – for some reason I can’t get the router to route packets between VLANs. All VLANs can talk to the router, DHCP, even NAT to the wan, but they can’t talk to each other. I even see the dynamic routes added for each subnet. From everything I’ve read, this should just happen automatically. It’s quite strange.

The router can ping all hosts, but no hosts ping or make TCP connections across subnets (and no, these aren’t Windows hosts). I’ve tried ridiculously permissive firewall rules, static routes, mangle preroute rules. Nothing seems to make any difference and I’ve been beating my head against the wall for days now.

The basic setup is I have ether5 as a trunk with 4 VLANs that will have different levels of access to each other. Ether1 is the WAN connection that comes in untagged, but gets tagged as VLAN 2. pvid=4094 is just a random unused vlan tag from prior problems I was having with untagged traffic on the trunk port. I think I have Layer 2 ironed out and working now. This appears to be a layer 3 problem…

# oct/21/2020 13:13:11 by RouterOS 6.47.4
#
# model = RB4011iGS+
/interface vlan
add interface=local name=guest vlan-id=4
add interface=local name=lan vlan-id=1
add interface=local name=priv vlan-id=8
add interface=local name=wan vlan-id=2
add interface=local name=wggw vlan-id=16
/interface bridge
add fast-forward=no name=local protocol-mode=none pvid=4094 vlan-filtering=yes
/interface bridge port
add bridge=local hw=no interface=ether1 pvid=2
add bridge=local hw=no interface=ether2
add bridge=local hw=no interface=ether5 pvid=4094
/interface bridge vlan
add bridge=local tagged=ether5,local vlan-ids=1
add bridge=local tagged=local vlan-ids=2
add bridge=local tagged=ether5,local vlan-ids=4
add bridge=local tagged=ether5,local vlan-ids=8
add bridge=local tagged=ether5,local vlan-ids=16
/ip pool
add name=dhcp_pool0 ranges=10.0.1.100-10.0.1.254
add name=dhcp_pool1 ranges=10.0.2.100-10.0.2.254
add name=dhcp_pool2 ranges=10.0.3.100-10.0.3.254
add name=dhcp_pool3 ranges=10.0.4.100-10.0.4.254
/ip dhcp-server
add address-pool=dhcp_pool0 disabled=no interface=lan lease-time=2h name=dhcp1
add address-pool=dhcp_pool1 disabled=no interface=guest lease-time=2h name=dhcp2
add address-pool=dhcp_pool2 disabled=no interface=priv lease-time=2h name=dhcp3
add address-pool=dhcp_pool3 disabled=no interface=wggw lease-time=2h name=dhcp4
/ip address
add address=10.0.1.1/24 interface=lan network=10.0.1.0
add address=10.0.2.1/24 interface=guest network=10.0.2.0
add address=10.0.3.1/24 interface=priv network=10.0.3.0
add address=10.0.4.1/24 interface=wggw network=10.0.4.0
/ip dhcp-client
add disabled=no interface=wan
/ip dhcp-server network
add address=10.0.1.0/24 dns-server=10.0.1.1 gateway=10.0.1.1
add address=10.0.2.0/24 dns-server=10.0.2.1,XXX.XXX.XXX.XXX gateway=10.0.2.1
add address=10.0.3.0/24 dns-server=10.0.3.1,XXX.XXX.XXX.XXX gateway=10.0.3.1
add address=10.0.4.0/24 dns-server=10.0.4.1,XXX.XXX.XXX.XXX gateway=10.0.4.1
/ip firewall filter
add action=accept chain=input comment="accept established,related" connection-state=established,related
add action=drop chain=input connection-state=invalid
add action=accept chain=input comment="allow ICMP" in-interface=wan protocol=icmp
add action=drop chain=input comment="block everything else" in-interface=wan
add action=fasttrack-connection chain=forward comment="fast-track for established conn" connection-state=established,related
add action=accept chain=forward comment="accept established conns" connection-state=established,related
add action=drop chain=forward connection-state=invalid
add action=drop chain=forward comment="drop access to clients behind NAT from WAN" connection-nat-state=!dstnat \
    connection-state=new in-interface=wan
/ip firewall nat
add action=masquerade chain=srcnat out-interface=wan

The routing table seems to be populated with dynamic routes to cover this

/ip route print
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 ADS  0.0.0.0/0                         XXX.XXX.XXX.XXX               1
 1 ADC  10.0.1.0/24    10.0.1.1    lan                       0
 2 ADC  10.0.2.0/24    10.0.2.1    guest                     0
 3 ADC  XXX.XXX.XXX.XXX/19     XXX.XXX.XXX.XXX  wan                       0
 4 ADC  10.0.3.0/24     10.0.3.1     priv                      0
 5 ADC  10.0.4.0/24     10.0.4.1     wggw                      0

In the end, I can’t get any traffic across VLANs! They can all hit the router and get to the WAN, but not to each other. Very strange. I am fairly new to RouterOS, so hopefully I’ve just missed something obvious.

Any feedback is greatly appreciated!

2

Nevermind…I figured it out. Or, at least I figured out it’s not mikroTik…It was the client I was using for testing. iptables -S shows it’s wide open, but it seems to only be accepting connections from the local lan. It’s Ubuntu, which I don’t usually use for servers, so maybe they added something.

Anyway I still don’t know exactly what I wasted the last day on, but adding more clients to the other subnet proves the router is working just fine.

3

(1) I would change vlan1 to any other number make it 10.

(2) You could get away with this…
add bridge=local tagged=ether5,local vlan-ids=4,8,16

(3) I dont understand why you created a vlan for ether1 your WAN connection (vlan2) because you pvid it which means the router assigns the vlan tag to the traffic comeing from the WAN.
If the WAN traffic is coming in to the router without tags, why are you assigning to a vlan?

(4) Related is this line that has no bridge port assigned??? (okay I get its assumed that untagged=eth1 is there by default, however I prefer to actually type it so there is no confusion)
add bridge=local tagged=local vlan-ids=2

however the same question in 3…why??

4

Those are all good questions. I rebuilt the config quite a few times trying to solve previous problems before I got to the last one I posted here…
Basically it started full router-on-a-stick with ether1 as the main trunk. The WAN connection is a POE modem and is tagged in my external managed switch as vlan 2 and was on the primary trunk (ether1). I might switch it back at some point to free up a port or at least stop removing tags at the switch and adding them back on at the router…

I had a lot of trouble with layer 2, mostly because I was trying to make the internal switch handle the VLANs and the 4011 just can’t do that in the switch hardware. That’s fine and the performance is great with the current bridge config, but it took a bit of reading to understand all of that.

I also had issues because I hadn’t added the bridge as tagged on any of the VLANs in /interface bridge vlan. Everytime I would turn on vlan filtering it would die. I figured all of that out, but there was a lot of re-configuring and uncertainty along the way that led to this kinda messy config.

Is it bad practice to use vlan 1? I’ve just always had it set up that way. It would be easy enough to change…

Things are actually working great now. I’ll take your advice on cleaning up the config.

By the way, the root cause to this issue ended up being extremely silly. I had setup an old wireguard connection on the server I was testing with that I had forgotten about and all of the replies to my pings and connections were going out the wrong interface when they weren’t in the local subnet! I should have checked the client’s routing table sooner, I just didn’t expect it to have a second interface. (Wireguard sure is resilient).

Thanks for your analysis and response.

5

For a standard setup, there is no requirement to identify the WAN with a VLAN.
My wanisp internet is on a vlan so I have to assign a vlan to the wan interface but no dhcp or anything like that.
By identifying the wan with IP Client, the router does all the heavy lifting.

6

Right. I understand it’s not required. It’s just a leftover from my original configuration.

Thanks again.

7

For a stadard setup, there’s no requirement for any VLAN whatsoever. However, if one configures router in ROS (Router On a Stick) manner, it is vital to get WAN to router tagged.

Why would one want to do it? Well, in my home LAN, I’ve got 3 ROS devices and as WAN is put in a VLAN by a managed switch, I could run routing/firewalling on any of ROS devices regardless the exact physical location (well, ISP tech, find my router if you can) … If WAN is actually tagged (it’s not in my case), then things are even more straight-forward, simply carry on ISP’s VLAN tag.

8

Okay so what you are saying in plain english is that there may be cases where you want to route the ISP traffic coming from their ONT/MODEM to several ROS routers. In this case you specifically would require a block of IP addresses (which was never discussed if I recall correctly), and thus it made no sense to me, nor does your intejection unless you are also talking about multiple IPs.

In which case you are simply routing the ISP traffic coming in on ether1 and then somehow attaching vlan tags to it, (untagging on the way back out to the ONT/modem) and then vlaning that traffic to other ethernet ports (LAN ports) to other ROS devices that will act as routers for the other IP addresses.


What I cannot figure is the first router… Is it simply good enough to assign manuallly one of the IP addresses (Wan) from the block to the IP DHCP client and then good to in terms of interent for the first router??? Yes, the vlan traffic could go next to a smart switch and then to 3 other ROS devices/routers

9

Not route, switch. Inside a VLAN.

10

My bad switch… how bout simply move the traffic around in an L2 construct via vlan tagging and bridge and etherports to other smart devices taht will use that incoming L2 traffic in any way they see fit, for example attaching to their IP DHCP client settings for example… closer???

11

Sure. Whatever fits you. Just remember: WAN is no different than any of LANs when it comes to switching, the only entity making it fundamentally different is firewall.
Nope, not even routing is that different, default is only English word for 0.0.0.0/0 which is a really large _L_AN with really short subnet mask :wink:

In my case I’m already doing what you just described. My ISP delivers IPTV in a VLAN and I simply switch that VLAN from ISP’s gadget (an xDSL modem) to ports connecting set-top boxes. I tried to play smart with IGMP snooping and what not, but didn’t go well (Mikrotik’s implementation of IGMP snooping breaks IPv6), so my LAN is pretty transparent for that particular VLAN.