Ok, so I finally sorted out the problem…
To recap, I’ve got 2 * RB450G and a couple of Sun servers in a cluster.
- The Sun boxes have limited ethernet port density and no way to add an additional card (I need the only PCIe slot for a SCSI card).
- I need a single port to deal with tagged and un-tagged VLANs
- I need to use two RB450Gs to allow one to be updated/fixed/changed while the other is still active (it’s a High Availability project)
- MikroTik RouterOS 4.6 on both RB450Gs
Both RB450Gs have the same config (other than things like IP addresses for the switch itself).
Admin bridge containing ports:
- Ether 2 (non 802.1q device)
- Ether 3 (untagged)
- Ether 4 (untagged)
- Ether 5 (Tagged VLAN 2)
Production bridge
- Ether 1 (non 802.1q device)
- Ether 3 (Tagged VLAN1)
- Ether 4 (Tagged VLAN1)
- Ether 5 (Tagged VLAN1)
Problem arises because although Ether 3 & 4 can be added to the Admin bridge and the associated VLAN interfaces can be added to the production bridge, traffic doesn’t flow properly in this config. From various reading on the 'net, it seems there are problems using raw ports and vlans on physical ports at the same time. At present, RouterOS doesn’t actually provide a mechanism for getting ‘untagged’ interfaces on the raw ports. Hence the config looks sane, but doesn’t work.
So, to work around it, I’ve done the following:
Admin:
- Ether 2
- Ether 3
- Ether 4
- Ether 5
Production:
Yup, by taking the VLAN interface off the bridge rather than the physical port, both Admin and Production networks pass traffic ok. The downside, is that the production VLAN is also present on Ether 2, where it isn’t actually needed. As the device connected to Ether 2 doesn’t talk 802.1q, it’ll never be used, so for now, I can ignore this ‘leak’.
The other ‘side effect’ is that as Ether 5 is a direct connection to Ether 5 on the second switch, this has the indirect effect of also taking across vlan1. If you create an ether5-vlan1 on both, you end up with a bridging loop, which the ‘Horizon’ feature won’t resolve, as it’s across two bridges.
So the big lesson, is that there is currently no method of using the raw port AND a VLAN off that port at the same time. Until RouterOS delivers a method of getting the untagged packets from an interface, then you have to jump through hoops to get the config to work, and there are restrictions in what you can do.
One last issue, is that you can’t run RSTP on both the Admin and Production bridges; in theory it should work, as the RSTP packets for Production should be encapsulated in vlan1 when added to the ‘admin’ bridge, but I can get one OR the other to work with RSTP, not both.
For now, again this is acceptable, as the “Admin” bridge doesn’t actually need RSTP, it’s just the ‘Production’ bridge.
Hope this helps someone else and the search engines pick it up accordingly.