Hello,
I've just noticed the following behaviour on a 6.44.3-equiped CRS328-24P:

- CRS328 is set in router mode with ether1 as a DHCP-configured WAN port
- all other etherXX, sfpplus-XX interfaces belong to bridge-named bridge
- a dhcp1 DHCP server is running on this "bridge" interface.

When I remove ether2 from bridge, dhcp1 server becomes invalid.
When I restore ether2 bridge membership, dhcp1 server becomes valid.
When I remove any other etherXX from bridge (XX being different from1 or 2), dhcp1 remains valid.

Under the hood, it seems ether2 has a specific role.

1. How can I assign this role on next available interface or any other ethernet interface (ether3, ether4) and let ether2 becomes a secondary DHCP-configured WAN port ?

2. Have you met unpleasant side effect (troubling sfpplus ports configuration, ...) when assigning WAN ports to latest etherXX ports (ether24, ether23, ...) ?

Best regards

Best regards

Bridge by default assumes MAC address of the first active member port. Highly likely that's ether2 in your particular setup. If you remove "MAC donor" port from bridge, it looses MAC address.

Solution: on bridge set auto-mac=no admin-mac=uu:vv:ww:xx:yy:zz where the set MAC address is one of port MACs, but add 2 to the most significant byte of It (the uu part). Example: if ether1 MAC is B8:69:F4:20:A5:49, then bridge MAC should become BA:69:F4:20:A5:49 ...

Bridge by default assumes MAC address of the first active member port. Highly likely that's ether2 in your particular setup. If you remove "MAC donor" port from bridge, it looses MAC address.

Solution: on bridge set auto-mac=no admin-mac=uu:vv:ww:xx:yy:zz where the set MAC address is one of port MACs, but add 2 to the most significant byte of It (the uu part). Example: if ether1 MAC is B8:69:F4:20:A5:49, then bridge MAC should become BA:69:F4:20:A5:49 ...

Yes, changing the most significant byte is a nice trick to get a unique MAC.

I noticed in my setup, one IP (192.168.88.1/24) is associated with ether2.
It is a prerequisite to associate this IP with something else first (ether3, ether4, bridge, ...) before changing bridge MAC address ?

If positive would you agree to proceed in the following order:
1. Move LAN IP to bridge instead of ether2
2. Change bridge MAC address
3. Disable ether2 bridge membership
4. Configure ether2 addressing

At the first impulse I'd change the order of execution of points 1. and 2. But I'm not sure about inner working in RB.

In theory bridge is entity separate from any member port, so if IP address is associated to ether2, it should be fine to change MAC address of the bridge and all L3 connections using IP address should continue because their L2 anchor didn't move. However, we've seen cases where IP address bound to member port (instead of bridge) was causing some weird behaviour. It could happen that those IP connections are handled by bridge after all, specially so if bridge shares MAC address with the port in question.

On the other hand the behaviour should be more deterministic if you indeed first migrate IP address from port to bridge (management connection might break at this point) and later change bridge MAC address (the connection might break at this point as well, I guess probability is higher than at point #1).

At the first impulse I'd change the order of execution of points 1. and 2. But I'm not sure about inner working in RB.

On a lab box, I tried my suggested process before posting and it worked OK apparently but I'm far from being convinced by this single test.
I've been so many times beaten that I preferred to ask.

From my very limited experience I would refrain from loading up the bridge from any extra functions. In other words dont load it down with DHCP work.
Either assign DHCPs to my favourite vehicle, VLANs, or to subnets directly. The Bridge use seems tidy at first but the traps and pitfalls mount quickly.............

Either assign DHCPs to my favourite vehicle, VLANs, or to subnets directly.

What if one doesn't use VLANs and uses bridge to join several physical ports to single L2 network, where should DHCP go then? I don't think that assigning my DHCP to your VLAN would do any good ...


BTW, my friend @anav has a point here.... CRS3xx can HW offload VLAN filtering and using VLANs to simply partition the switch into several L2 domains might be actally better than using several bridges (I'm not sure if limit of single bridge per device being HW accelerated applies to CRS3xx as well). If only single physical port on device is member of an L2 domain this doesn't matter (or, rather, better use it directly without hokus-pokus with bridges or VLANs) as any traffic on that port would have to hit CPU regardless the topology used ...

BTW, my friend @anav has a point here.... CRS3xx can HW offload VLAN filtering and using VLANs to simply partition the switch into several L2 domains might be actally better than using several bridges (I'm not sure if limit of single bridge per device being HW accelerated applies to CRS3xx as well).

I often wondered what could be the benefit of several bridges instead of using VLANs.
Is there some use or corner cases (the same network present in several bridges ?) that requires one type of setup ?
I would be very curious to read about this.

In ROS versions up to 6.40 it was not possible to use bridge vlan-filtering. So for certain use cases (e.g. when bridging other types of ports, such as wireless or PtP or ...) it was necessary to use one bridge per VLAN. Or, more accurately: that was one of ways of doing it.

Another use case is already mentioned switch segmentation ... use of multiple bridges is a most straight-forward approach. It is possible to achieve that by using VLANs internally inside device (all ports are access) on single vlan-filtered bridge.
On devices that don't support VLANs in hardware (there are some, one of most notable is hEX RB750Gr3) partitioning has to be done in software and with ROS <=6.40 it could only be done with multiple bridges. Not that that's awfully usable with hEX'es huge number of 5 ether ports plus SFP (which is connected to CPU and can't be HW offloaded anyway).


I don't fully understand what you mean by "the same network present in several bridges" ... When I think of it, it doesn't matter which layer (L2 - ethernet or L3 - IP) you're talking about. In both cases having more than one bridge inside single RB device doesn't make much sense as those "several bridges" should be somehow joined in L2 domain (L3 - routing doesn't work on L2 and doesn't help inside L3 network). Proxy-ARP isn't awfully useful if there are more than a few devices which should be (transparently) accessible between those bridges. So you'd end up having a pyramid of bridges (one bridge joining other bridges).

For sure there are some special use cases when using stack of bridges is the only way out (if one has to use some particular feature implemented by bridge and nowhere else).