I have just purchased a CRS518-16XS-2XQ for my home setup. Where the setup should be fairly simple: 2 Cisco UCS c240-m4 servers with VSAN and a single ucs-c220m3 server for the witness host.
All three servers are at the moment connected with 2x 10G each and running VMware ESXi 8.0.1 and VMware NSX 4.1. But here is where the problems begins. I have pretty many years of experience with Cisco, DELL, Arista and Juniper networks switches. But it looks like Mikrotik is doing it quite differently.
What I need to configure is 6x 802.1q trunks ports for the UCS servers and a couple of 802.1q trunks ports for an uplink Mikrotik 326 running Switch OS (that one is not yet configured) and a Cisco 2960s switch as well.
I need a few VLANs with an IP address for default gateway for the physical servers and later on for BGP towards the NSX. But my problem is. I have created VLAN 3 as the first VLAN, and every time I tries to add another VLAN for bridge 1. The switch simply stops forwarding packets and all connections are lost.
I have pasted the configuration for anyone to look at, and point me in the right direction, if I have made it all wrong just let me know
TWO THINGS
One, DONT USE Quote marks in rules?? You can name things in quotes usually done if there are spaces in names but most people rightly avoid them.
Second thing, is you need to tag the bridge itself!
Mikrotik have a default native VLAN ID of 1 on bridges (for the switch-to-CPU interface) and bridge ports, as with many other defaults it doesn’t appear in /export. Attempting to add an /interface vlan with the same ID usually results in odd behaviour and/or loss of connectivity due to the mix of tagged and untagged for the same VLAN.
Most Mikrotiks only support hardware offload on a single bridge, additional bridges will pass all bridged traffic through the CPU leading to significantly less than wire-speed performance. You will also require additional configuration for L3 hardware offload.
If you are going to configure any access or hybrid ports also be aware there should only be a single VLAN ID specified for bridge vlans vlan-ids= when that ID is used untagged. You should only specify multiple VLAN IDs when all of the IDs are used tagged on trunk ports.
@anav for whatever reason /export in some versions of RouterOS on some hardware quotes the bridge vlan interface list(s) unnecessarily.
Thanks a lot, those links helped to better understand the difference between Cisco and Mikrotik
Regarding the HW bridge on the switch. Does that mean, that when creating bridge1, that is not done in HW?
Is it only the default bridge that is done in HW?
My idea was to have one bridge for the management interface and everything else on a different bridge. I had some issues when changing the default MTU on the default bridge on the management interface. That is why i created the second bridge. But if only the default bridge is done in HW, I guess I can simply switch the to, and have management on bridge1 and everything else on the default bridge.
I am trying to understand why you want a second bridge “device”.
You talk about wanting management to have a different (probably standard 1500) MTU.
A vlan interface (created with /interface vlan) is similar to a Cisco SVI vlan interface, and it is the CPU’s connection to a specific vlan on the switch part of the bridge. When you create the vlan interface, you can specify an MTU there on the vlan interface, which is also the entity that will have an ip address attached.
If you don’t have HW switching, everything will be forced through the CPU and it is meant only for management operations. Look at the block diagram for why I say this.
As I wrote in my first post. It might just be me that has misunderstood the function of the Mikrotik switch
So in reality I really do not need to create a bridge1, I can have everything on the default bridge?
When I tried to change the MTU for the default bridge it did fail, and that was the reason for creating the additional bridge. And most of the guides I initially tried to follow did that as well. So my thought was, that this was the best practice to do.
I will give a try on reconfigure using just the default bridge.
The first bridge created will use hardware offload. If you disable hardware offload (with hw=no) on all the bridge ports of the first bridge the next bridge will use hardware offload.
The management interface doesn’t have to be a member of a bridge, you can assign an IP address to it directly.
I have the switch fully working now, and with BGP from my NSX Edge nodes towards the Mikrotik switch as well.
I just tested the power consumption for the switch compared to my old Cisco Nexus 5548UP. The Cisco uses 360watt continuously and the Miktotik 20watts…!!
