There is a larger thread in the general discussion where multiple people have reported the same issue but I didn’t see a detailed post in the bug section outlining the issue so I’ll try and make this as clean as possible to get the quickest resolution (either it is a bug, or its the intended functionality). Any thoughts are greatly appreciated, also here is the link to the larger thread where some other attempts have been made and also similar examples of it not working for other people.
Version: 7.8rc2 (but tested on 7.7 stable and 7.8rc1 and it doesn’t work there either)
Model: CCR2216-1G-12XS-2XQ
Notes:
No firewall rules blocking anything, just the accept rule for fasttrack.
In a more complex setup I am able to get non-VLAN traffic from another port on the CCR (like sfp28-2) to any VLAN on the bridge to be offloaded. But the key is that inter-VLAN traffic from a VLAN on the bridge to another VLAN on the bridge is not offloaded.
Steps to Reproduce:
Setup a downstream switch with multiple VLANs that will be trunked from a single port on the switch to a single port on the CCR. For example have a switch with 8 GbE ports and 2 SFP+ ports. Access ports for individual VLANs on ports 2-5 of the switch correspond to VLANs 2-5 and have them all trunked to a single SFP+ uplink that you connect to one of the CCR’s sfp28 ports.
Setup a bridge on the CCR.
Add a bridge port with the desired sfp28 interface.
Create bridge VLAN entry with the VLAN’s 3-5 tagged and the sfp28 port as well as bridge tagged.
Attempt to pass traffic (iperf3 or any traffic SMB/FTP etc.) from a workstation on VLAN 3 to another workstation on VLAN 4.
— Expected functionality, is that within the appropriate connection limits etc, the traffic should be offloaded as per the documentation on L3HW offloading. Actual functionality is that the traffic is not offloaded.
Simple Config:
/interface bridge
add frame-types=admit-only-vlan-tagged name=bridge pvid=4094 vlan-filtering=\
yes
/interface vlan
add interface=bridge name="IoT Wifi" vlan-id=5
add interface=bridge name="Mobile Wifi" vlan-id=4
add interface=bridge name="Secure Wifi" vlan-id=3
add interface=bridge name="WAN (4000)" vlan-id=4000
/interface ethernet switch
set 0 l3-hw-offloading=yes
/interface ethernet switch port
set 0 l3-hw-offloading=no
set 1 l3-hw-offloading=no
set 2 l3-hw-offloading=no
set 3 l3-hw-offloading=no
set 4 l3-hw-offloading=no
set 5 l3-hw-offloading=no
set 6 l3-hw-offloading=no
set 7 l3-hw-offloading=no
set 8 l3-hw-offloading=no
set 9 l3-hw-offloading=no
set 10 l3-hw-offloading=no
set 11 l3-hw-offloading=no
set 12 l3-hw-offloading=no
set 13 l3-hw-offloading=no
set 14 l3-hw-offloading=no
set 15 l3-hw-offloading=no
set 16 l3-hw-offloading=no
set 17 l3-hw-offloading=no
set 18 l3-hw-offloading=no
set 19 l3-hw-offloading=no
/interface bridge port
add bridge=bridge frame-types=admit-only-vlan-tagged interface=sfp28-12 pvid=\
4094
/interface bridge settings
set use-ip-firewall=yes use-ip-firewall-for-pppoe=yes \
use-ip-firewall-for-vlan=yes
/interface ethernet switch l3hw-settings
set ipv6-hw=yes
/interface bridge vlan
add bridge=bridge tagged=bridge,sfp28-12 vlan-ids=3-5,4000
l3 hw offload - stateless offload of IPv4/IPv6 routes into hardware
l3 fw offload - stateful offload of IPv4 connections and NAT (IPv6 fastpath/fasttrack yet to be implemented)
Must have l3-hw-offloading=yes on the switch chip but disabled on all ports
Must create mangle rules to use fasttrack on connections that you want to offload to hardware
MikroTik may be able to comment if the gap on these limitations has closed between 7.6 and 7.8rc2.
Updated title to reflect the situation of the issue clearly. Specifically, the FW compatible L3HW offload is not working with inter-vlan traffic. Simply doing L3HW offloading with no firewall expectations which is basically L2 switching + VLAN (L3) making it L3HW connected routes offloaded switching is not what I’m referring to in my post. This is soley about firewall compatible L3HW offloading (stateful firewall + NAT).
I would say that’s accurate because in fw offload you have to disable hw-offload on the port level (but enabled at the switch level) so the only way traffic is getting hw-offloaded is if it hits a mangle rule that enables fasttrack. Presumably this is so that mangle can enable/disable offload per flow instead of per port.
It could be that development hasn’t reached a point where they can be used together and not a permanent limitation.
You can use them simultaneously, just you need to set the switch (port level) hw-offloading to enabled. But in doing so for that particular interface port you will not be able to use FW L3HW offload and HW offload at the same time. So in short, per documentation, at the port level you can only use one or the other but at the device level (CCR) you can have some ports that do L3HW FW offload and some that do L3HW connected routes offload.
I sent an email (as I don’t have access to this servicedesk). Unfortunately nothing as of yet. Any possibility for any Mikrotik people to try and setup two VLANs on a bridge with each respective VLAN assigned to a bridge port that is associated with a different underlying physical interface and verify that the L3HW offloading works for → inter-VLAN ← traffic? Additionally, I can also confirm that just having 2 VLANs on a bridge that is associated with the same underlying physical interface also doesn’t work for L3HW offloading. This is all tested on 7.8 stable.
Also if any forum members know of a version of RouterOS where L3HW offloading was or is working for inter-vlan traffic please let me know as I would like to downgrade and test.
Do either of you mind posting a sanitized network config from your CRS that show how you are doing your L3HW offloading I might be able to glean something from it. I suspect maybe there is an issue between how the CRS is doing it versus the CCR.
Jesus, after a ridiculous amount of testing I figured out the problem. If you have IP-Firewall set as active and then set either of the two options like “Use IP Firewall For VLAN” or “Use IP Firewall For PPPoE” set, then it doesn’t work. And I was told by a Mikrotik member to disable IP-Firewall and I did and there was no change. However, this is due to a bug, you need to first disable “Use IP Firewall For VLAN” and “Use IP Firewall For PPPoE” while the parent option “Use IP Firewall” is enabled and then disable the parent option “Use IP Firewall”. Otherwise even though IP Firewall must be enabled for either of the lower two options to be selectable, disabling the parent option doesn’t actually disable the other options so somewhere in the code the router still thinks the “Use IP Firewall For VLAN” and “Use IP Firewall For PPPoE” are still enabled even when the parent option “Use IP Firewall” is disabled.
Long story short, if you want L3HW FW Compatible Offloading, disable “Use IP Firewall” and then if you don’t see “Bridge Fast Path Active” checked (make sure Allow Fast Path is checked as well), then try the below steps and when it becomes checked then you know you’ve got it working.
Enable Use IP Firewall (if it isn’t already).
Disable Use IP Firewall For VLAN
Disable Use IP Firewall For PPPoE
Disable Use IP Firewall
Restart router.
@Mikrotik Mods, this thread can be closed since the problem is solved.
I’m glad that L3HW FW-Compatible Offloading finally works on your side. We have identified an issue where disabling “Use IP Firewall” does not entirely disable the firewall if VLAN or PPPoE options are still enabled. It will be fixed in future releases. Meanwhile, disable all three options to make FastPath work.