I am trying to configure RB4011 with vlans in a right manner and struggeling with correct settings. I see arp-table entries which permanently switching from stale to reachable as a sign of L2-misconfiguration (I use rstp-mode=none to be able to use hw-offloading).
I found a hint to reconfigure bridge settings with respect of the 2 switch-chips of model RB4011 on following help-page (suggested config below link) but it seems to be outdated for ROS7 now. At menu switch of RB4011 running latest ROS7.20.6 I can't configure as described on referenced help-page due to lack of possibilities for port and vlan settings under /interface ethernet switch .
It would be nice to have an actual sample config example for vlans with hw-offloading enabled for systems with multiple switch chips and optional wifi.
Help-Page for L2-troubleshooting for models with multiple switch-chips like RB4011:
Suggested config at help-page which seems to be outdated for current ROS7-versions:
Maybe you need to explain that better. Also, don’t search too much behind the ARP status, there have been changes in v7 which make that status table largely useless. Before doing anything with it, delete all entries that are not permanent (“status contains not permanent” in the filter).
RB4011 supports HW offloaded vlan-filtering with RouterOS v7, see these examples:
And since 7.20, you do not need to worry about the multiple switch chips on RB4011, as RouterOS will dynamically add CPU to the VLAN table, if VLAN goes across multiple switch chips:
*) bridge - added dynamic tagged entry named "switch-cpu" in scenarios where the same VLAN spans multiple switch chips or is used on both HW and SW ports;
Edit: Or you can manually add bridge interface as tagged VLAN member (in case dynamic entry does not appear).
This new feature seems to be broken on E60iUGS w/ v7.21rc2, forcing me to add the bridge as tagged port in order to make the VLAN forwarded to non Hw. Offload ports.
Yes, only when same VLAN ID crosses two or more switch chips (RB1100AHx4, RB4011, CCR2004-16G-2S+).
But there are other situations, like crossing same vlan-id on hw and non-hw ports (ether and SFP on RB4011, CCR2004-16G+2S+, hEX S, or ether and "software-only" ports like EoIP, wifi).
Are you saying that if you configure a single bridge that "spans" both switch chips (single bridge includes switch-ports from both switch chips), that the bridge will still be hw offloaded?
What if a single bridge is configured, but no vlan spans over both switch chips. Will that be offloaded?
What if two bridges are configured, and no vlan spans over both switch chips. I think this will be offloaded, but I am not sure.
I have no routers with two switch chips, so this is primarily a curiosity question.
Assume an RB4011 and a CRS326 in the same rack next to each other (so any cabling between them is not a factor).
Goal: to have multiple (say 4) vlans and to use the RB4011 for inter-vlan routing. Would like to be able to utilize the RB4011 ether ports as access ports for the vlans, but (assume) all access ports (on the RB4011) for any vlan can be configured with all switch-ports on the same switch chip.
The easiest approach would be a single bridge including all switch ports (except for internet port), but I don't know how much could or would be offloaded.
The basic truth: VLAN HW offload is about L2 (even though VLAN is sometimes considered as L2.5 it still not L3, in OSI layers philosophy layer numbers get truncated rather than rounded).
And L2 is switching (in particular between ports members of same VLAN).
Routing is L3. In case of RB4011 it's done entirely by CPU. As all the traffic has to pass to CPU, there's not much that switch chip can do (adding/stripping a VLAN tag or two is not much).
The trick ROS uses when it comes to use switch-connected ports as individual interfaces is to either use switch chip vendor's proprietary additions (as it's in case of Qualcomm's switch chips) or to use VLAN tags (hidden from users) ... which was the reason that e.g. Realtek switch chips didn't have the /interface/ethernet VLAN setup (this is pretty directly mapped to switch chip programming interface). Since bridge HW offload (introduced in v7 for Realtek and Mediatek) L2 HW offload does work, but given the mechanizm of using individual ports as interfaces, there's no difference in performance regardless of config scenario (individual interfaces vs. VLANs with access ports ... the later being identical to the former but with VIDs explicitly set).
My question is about whether L2 between two ports in the same vlan in the same RTL8367 switch chip will be done by the RTL8367 switch chip, or if it will need to be bridged by software in the CPU.
Consider the following (hand generated config) with an RB4011 configured as a switch (no routing, assume any inter-vlan traffic will be routed by some router connected to the sfp-plus1 trunk. Also assume configured from console. And 7.19.6 where the bridge had to be explicitly added as tagged.
With this config, if two PC were connected to ether4 and ether5 and iperf was run with small packets, and compare the CPU used by RB4011 compared to when connected to ether5 and ether6 (which must be bridged by software running on the RB4011 CPU). Since all of these bridge-ports are in the same vlan (20), then this should be almost exclusively L2 traffic (once the connnection is established). But 4 & 5 are on the same switch chip, 5 & 6 are on different switch chips.
With shown configuration, traffic will be switched by switch chip if:
both ports (ingress and egress) are connected to same switch chip
traffic remains inside same VLAN. It doesn't matter if port is access for that VID (PVID set) or trunk/tagged for that VLAN
When traffic doesn't entirely fit into the criteria above, then it's somehow handled by CPU. E.g.:
If ports involved in switching/bridging are connected to different switch chips (or one is connected directly to CPU, e.g. SFP+ port or wifi interface), then traffic will pass CPU where it'll be handled by bridging code.
If traffic has to pass between different VLANs, it has to be routed. Again this is CPU task, but routing code.
"HW offload" means that traffic is handled completely by switch chip ... and it's not possible in either of above mentioned cases.
So in your scenario above, HW offload is possible when testing between ports ether4 and ether5 and it's not possible when testing between ports ether5 and ether6.
Some are suggesting using two bridges (one per switch chip) and adding external physical wired connection (e.g. between ether5 and ether6) which indeed does bypass using CPU for traffic between ports connected to different switch chips. IMO it's not necessary: CPU in RB4011 is powerful enough to bridge at full speed, one consumes 2 ports and limits switch chip interconnect to 1Gbps (it's 2.5Gbps when going via CPU) ... and doesn't help with SFP+ if it's used as another LAN pirt.
It would have been best when there was an internal ethernet connection between the switch chips. They probably opted not to do that because this chip has only 5 ports so it would limit the number of external ports to 8, so they would have to use a chip with more ports.
