InterVLAN routing very basic concept

I will be setting up my first CRS328 switch. I am new to VLAN switch configuration, but have read lots of posts about how to set this optimally (ensuring I don’t use the CPU for inter VLAN routing). But I have very basic questions that aren’t clearly answered anywhere (or I misunderstood).

  1. Do I need to give the switch an IP address on each VLAN that it will bridge? (If so, doesn’t that force creation of CPU VLAN’s, and therefore CPU based routing)?
  2. Do I need to enable IP-forwarding, and setup routing rules to perform the interVLAN routing?
  3. If I don’t do the above, does the CRS328 just magically route between the VLANs?

I apologize if this is obvious…but I’m new to this and perhaps I’m getting stuck on a very basic concept. I don’t understand why some people say don’t create VLAN interfaces since that will force routing through the CPU (instead of the switch chip)…but how does the routing work then?

Hello there.
Best guide → http://forum.mikrotik.com/t/using-routeros-to-vlan-your-network/126489/1

example of a switchs setup after you read the article above.
https://www.youtube.com/watch?v=YLtGQAQ8iS0&t=403s

note1: only one vlan needs to be identified on the switch ( the management or trusted vlan )
note2: only the management/trusted vlan is tagged to the bridge on /interface bridge vlan settings.
note3: highly recommend the first thing you do is take one port OFF the bridge, and do all your configuring from that safe port.

Take an unused port or temporarily assign one to get a stable working config.
aka port 7
and the first step is to remove it from /interface bridge port settings:

/interface ethernet
set [ find default-name=ether7 ] name=OffBridge7
/ip address
add address=192.168.55.1/30 interface=OffBridge7 network=192.168.55.0
/interface list member
add interface=OffBridge7 list=LAN ( and to BASE or Management if they exist )

Then simply plug in your laptop to port 7, change iPV4 settings to 192.168.55.2 and you should gain access the standard way.
From here do all the configurating.

If you really want this switch to route between vlans - i.e. actually let the devices on vlan A be able to talk to devices on vlan B without a separate router handling that somehow - you have the options of routing on the CPU or via L3 Hardware Offloading.

The CRS328 - both variants - have a DX3000 series ASIC which contains one weak-ish CPU core (meant to be management and not much more) so the first option is mostly out.
And you don’t want it either.

Sadly the DX3000 isn’t /amazing/ on the L3HW side either, but it can do some, so take a look here, then scroll back to the top and read as it’s a… complex topic.
https://help.mikrotik.com/docs/spaces/ROS/pages/62390319/L3+Hardware+Offloading#L3HardwareOffloading-CRS3xx:SwitchDX3000andDX2000Series

If you will use exclusively the “switch rules”, as you most probably should on this platform, I’m 95% sure that the only “/interface/vlan” layer 3 CPU interface you need is the management vlan, and then you don’t need to enable software forwarding in /ip/settings. It will basically all happen exclusively in hardware, with the CPU being blissfully unaware.

Nevermind, of course you need the software-side setup done to specify the switch’s IP address in every VLAN (after all it has to be an available local gateway), etc.
But yes, then it should do things in hardware if you configure it all using the switchrules.
See this chapter. https://help.mikrotik.com/docs/spaces/ROS/pages/62390319/L3+Hardware+Offloading#L3HardwareOffloading-Inter-VLANRouting
And sorry - I’m clearly undercaffeinated today.

  1. NO, only the management vlan
  2. NO, done on the upstream router
  3. NO, one on the upstream router, the switch only does two things conceptually.
    a. take a bunch of vlans coming in tagged on the trunk port from the upstream smart device, usually a router, and then distributes those vlans to its ethernet ports either untagged (going to a dumb device - access port), or tagged ( going to smart device - trunk port) or both untagged for one vlan and tagged for one or more vlans ( rare - going to a device that expects the base vlan untagged and data vlans tagged ).
    b. Grabs for itself an address from the base or management vlan, so that you the admin can access and configure the device over the vlan.

+++++++++++++
The previous poster raises a good point, I assumed that you have a router and are using this switch as a switch ???

(Edited-in an important correction to my previous post above, sorry!)

I’m working my way through the responses above and have some thoughts…

  • I’m trying to understand the example from wrqk showing adding the VLAN interfaces to the bridge (tagged) and the physical port (tagged) at https://help.mikrotik.com/docs/spaces/ROS/pages/62390319/L3+Hardware+Offloading#L3HardwareOffloading-Inter-VLANRouting. If I did that for each port, would the hardware actually do intervlan routing (without adding IP’s to an interface on each VLAN, and without adding ip rules). This example does not show any routing rules, yet the section is called intervlan routing…

  • Most examples show the CRS doing switching only, using an external router. Although I have an external router (rb4011igs), I didn’t want to overload it so I thought it would be a good idea to do some routing (intervlan) in the switch to spread out the load. Does that make sense? OR is this a bad idea?

  • If I don’t do any routing on the CRS, should I just use the SwitchOS instead of RouterOS? (if I don’t do any L3 stuff, why bother with RoS)? SwitchOS seems much simpler to setup…even though i have learned RoS from other projects.

  • My rb4011 has one interface defined for each VLAN, and an IP assigned to each interface (so ip routing rules will work). I assumed that all switching was done in hardware, but after reading some posts (still confused), does that mean my rb4011 is doing all routing through the CPU? (It must be seriously congested then). Or do RB devices handle all that stuff in hardware, but SWITCH devices do it in software? (I don’t want to redesign the whole network at this time…must limit how much I take on during this switch upgrade).

  • My network (which i inherited) uses untagged for management, and vlans for all else. I thought this made sense, but all the examples use a vlan for management as well. Is this important? I didn’t really want to start making network wide changes…but I supposed if needed I can make a VLAN used only inside the switch for handling management traffic (externally untagged). I’m wondering if there is a good reason that all examples place management on a VLAN instead of untagged!

Oh dear.

Okay, to get the easy shot out of the way - SwitchOS cannot do any routing, in hardware or in software.
But if it’d be more comfortable for you - and you’ll decide you don’t need any of the other ROS features that SwOS is missing - sure, why not.
I personally run ROS on all to have the same look-and-feel on the devices, even if I don’t really need anything more, but SwOS is basically “as simple as it can be” and that can be quite convenient.

Next thing - yes, the RB4011 is a 100% software router. With strong high speed quad core CPU, it can do a lot in software.
It has hardware acceleration for IPsec encryption (at only some combinations of the myriad of IPsec settings possible), but everything else is just CPU work and optimizations in config (FastTrack, etc).
The RB4011 has two simple-minded hardware switches each responsible for 5 ports; there’s a few small simple things you can order them to do in hardware, but in “typical” operation they only CPU-free operations will be forwarding packets between two ports on the same chip in the same VLAN, and VLAN tagging/untagging.
A huge number of MT devices, large and small, are like this (although most of the time usually with just one layer2 switch chip).
It’s basically only some of the new gear with Marvell chips which have the L3HW functionality, which is… tricky to use but if it can be used for one’s needs gives quite the massive performance boost.

Yes, it is quite common on big networks to have a “core switch” which does the basic L3 but very fast for “internal” networks, with some simple filters/ACLs and so on, at hardware “wire speed” or nearly so, then a separate router or these days “next generation firewall” doing the adwanced layer 3 stuff with connection tracking, inspection, NATs, dynamic routing protocols, etc etc etc.

You didn’t say much what is the load on “your network”, but if the RB4011 handles it without bottlenecking on a single core (watch /tool/profile during various high-traffic times?), it’s… fine to just keep doing that?

To give you an example, at one of my customers, I have a CRS317 (which has quite a bit more L3HW capacity than the CRS328) doing L3HW routing for the LAN with a lot of internal traffic (servers and NAS connected with 10Gb ports) and an RB1100x4 which has the same 4-core Annapurna CPU as the RB4011 doing WAN, NAT, and a ton of VPN terminations, with complex firewall rules.
But for a few years prior the 1100 was doing the job alone paired just with “dumb” CRS326 switches, until its 1Gb ports - not the CPU - became the bottleneck for the LAN part.


Regarding the “what vlan to use for what” - it really depends if you ask from point of view of convenience, or security, for example.
“Untagged management”* makes it easy to reach something if it falls to some kind of default mode; but because it makes it easy for you, it makes it easy for everyone else just as well.
(* I kinda guess you actually mean “vlan 1 for management, and leaving that as default/untagged on trunks”…)
In any case, again, something a lot of “security” forgets is that #1 step is to decide what you want to defend from.
There’s a nice term for it, “define your threat ” but with another noun which I can’t recall.

Lemme just tell you that at my main job I still have some sites doing the same because of how much risk and man-hours it would be to re-organize the management layer just-because.
New sites, and full-teardown refreshes are set up more modern, with isolated management zones of various trust levels and stuff, but what’s old-and-running stays until top management decides it’s a good use of time and money.

Maybe you really should do the change because you’re running a powerplant.
Maybe “better is the enemy of good-enough” and it can stay until you’ll replace all hardware to lay down your future 25Gb network all over the place and configure that fresh.

yeah, I’ll leave the network redesign for another day

I found another video similar to the link you posted that offers a nice (simpler/dumbed down) explanation:
https://www.youtube.com/watch?v=c2sAA6jMjCY

This video doesn’t explain how to add static rules (he uses ospf) but I assume even if I don’t add any rules the HWl3 switch just magically takes care of directing my packets?

If that’s correct, I’ll go with RoS and try the setup from this video

I will be completely honest, I can’t quite watch a half-hour video right now.

RouterOS overall approach to things is “you program things the same way for software-processing OR for hardware-accelerated-processing, and if the right set of pre-requisite conditions is met, hardware acceleration will kick in”. (Unfortunately, there is a lot of caveats, asterisks, and small print in the list of conditions.)

So if he is guiding you to set up the L3HW switch as the gateway for these specific vlans (yes the switch needs an IP in each of them, and typically should take over the “.1” from your current router), how to set up the bridge, and how to enable L3HW at the end, then following this should be fine.

ROS of course knows it has “direct-connected” routes for each vlan on which you set up an IP address for it, and then you need one or more static routes towards your big router to access the rest of the world.
Set it up, then watch and see if you have the H-flags visible everywhere they should be. :slight_smile:

The only question mark is, will RoS automatically move frames between VLAN’s based on their destination? Your link (example CLI commands) suggest that frame just magically move between VLANs..I don’t see any CLI command to do forwarding.

And maybe the video I posted too (or he skips a steps), but no forwarding setup.

Do I need a forwarding rule somewhere? Is this automatic with l3hw offloading? Do i need to add static routes?

For “direct connect” subnets, routes will automatically appear (check /ip/route/print) once you create an /interface/vlan and give it an /ip/address.
But you probably want to route some of the traffic further upstream (to the RB4011) so you will need static routes for that.

Once you activate l3-hw-offloading=yes, these routes will be programmed into the hardware and the hardware will start directing packets as instructed.
Make sure to activate it first on all the physical ports (/interface/ethernet/switch/port menu) and then enable the master toggle on the switch chip itself (/interface/ethernet/switch).

There’s a lot more to it, but well, best way to learn is to experiment.
That documentation page is a monster but it does have everything you need to use L3HW.
It /does/ build on the assumption you already understand how to do the layer 2 switching and standard software routing in ROS.
Which I believe you do as you’re running a router already.