Community discussions

MikroTik App
 
blacksnow
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 50
Joined: Wed Feb 15, 2023 4:46 pm

L3HW Firewall Offloading - Doesn't Offload Inter-VLAN traffic

Mon Feb 20, 2023 8:34 am

There is a larger thread in the general discussion where multiple people have reported the same issue but I didn't see a detailed post in the bug section outlining the issue so I'll try and make this as clean as possible to get the quickest resolution (either it is a bug, or its the intended functionality). Any thoughts are greatly appreciated, also here is the link to the larger thread where some other attempts have been made and also similar examples of it not working for other people.

Main Thread (Start of my post): viewtopic.php?p=985420#p984701
Other Example: viewtopic.php?p=985420#p929511
Other Example: viewtopic.php?p=985420#p963157

Version: 7.8rc2 (but tested on 7.7 stable and 7.8rc1 and it doesn't work there either)
Model: CCR2216-1G-12XS-2XQ
Notes:
1) No firewall rules blocking anything, just the accept rule for fasttrack.
2) In a more complex setup I am able to get non-VLAN traffic from another port on the CCR (like sfp28-2) to any VLAN on the bridge to be offloaded. But the key is that inter-VLAN traffic from a VLAN on the bridge to another VLAN on the bridge is not offloaded.
Steps to Reproduce:
1) Setup a downstream switch with multiple VLANs that will be trunked from a single port on the switch to a single port on the CCR. For example have a switch with 8 GbE ports and 2 SFP+ ports. Access ports for individual VLANs on ports 2-5 of the switch correspond to VLANs 2-5 and have them all trunked to a single SFP+ uplink that you connect to one of the CCR's sfp28 ports.
2) Setup a bridge on the CCR.
3) Add a bridge port with the desired sfp28 interface.
3) Create bridge VLAN entry with the VLAN's 3-5 tagged and the sfp28 port as well as bridge tagged.
4) Attempt to pass traffic (iperf3 or any traffic SMB/FTP etc.) from a workstation on VLAN 3 to another workstation on VLAN 4.
--- Expected functionality, is that within the appropriate connection limits etc, the traffic should be offloaded as per the documentation on L3HW offloading. Actual functionality is that the traffic is not offloaded.

Simple Config:
/interface bridge
add frame-types=admit-only-vlan-tagged name=bridge pvid=4094 vlan-filtering=\
    yes
/interface vlan
add interface=bridge name="IoT Wifi" vlan-id=5
add interface=bridge name="Mobile Wifi" vlan-id=4
add interface=bridge name="Secure Wifi" vlan-id=3
add interface=bridge name="WAN (4000)" vlan-id=4000
/interface ethernet switch
set 0 l3-hw-offloading=yes
/interface ethernet switch port
set 0 l3-hw-offloading=no
set 1 l3-hw-offloading=no
set 2 l3-hw-offloading=no
set 3 l3-hw-offloading=no
set 4 l3-hw-offloading=no
set 5 l3-hw-offloading=no
set 6 l3-hw-offloading=no
set 7 l3-hw-offloading=no
set 8 l3-hw-offloading=no
set 9 l3-hw-offloading=no
set 10 l3-hw-offloading=no
set 11 l3-hw-offloading=no
set 12 l3-hw-offloading=no
set 13 l3-hw-offloading=no
set 14 l3-hw-offloading=no
set 15 l3-hw-offloading=no
set 16 l3-hw-offloading=no
set 17 l3-hw-offloading=no
set 18 l3-hw-offloading=no
set 19 l3-hw-offloading=no
/interface bridge port
add bridge=bridge frame-types=admit-only-vlan-tagged interface=sfp28-12 pvid=\
    4094
/interface bridge settings
set use-ip-firewall=yes use-ip-firewall-for-pppoe=yes \
    use-ip-firewall-for-vlan=yes
/interface ethernet switch l3hw-settings
set ipv6-hw=yes
/interface bridge vlan
add bridge=bridge tagged=bridge,sfp28-12 vlan-ids=3-5,4000
Last edited by blacksnow on Mon Feb 20, 2023 5:21 pm, edited 1 time in total.
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 2989
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: L3HW Offloading - Doesn't Offload Inter-VLAN traffic

Mon Feb 20, 2023 8:55 am

try removing this
/interface bridge settings
set use-ip-firewall=yes use-ip-firewall-for-pppoe=yes use-ip-firewall-for-vlan=yes
EDIT
and maybe try adding this
/interface ethernet switch
set 19 l3-hw-offloading=yes
 
blacksnow
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 50
Joined: Wed Feb 15, 2023 4:46 pm

Re: L3HW Offloading - Doesn't Offload Inter-VLAN traffic

Mon Feb 20, 2023 9:39 am

Unfortunately, nothing changed. Traffic is still not offloaded.
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 2989
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: L3HW Offloading - Doesn't Offload Inter-VLAN traffic

Mon Feb 20, 2023 9:45 am

Try rebooting

Try disabling ando enabling Global L3 hw offload on switch

Maybe reboot again
 
blacksnow
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 50
Joined: Wed Feb 15, 2023 4:46 pm

Re: L3HW Offloading - Doesn't Offload Inter-VLAN traffic

Mon Feb 20, 2023 9:58 am

Unfortunately, still no dice.
Try rebooting

Try disabling ando enabling Global L3 hw offload on switch

Maybe reboot again
 
User avatar
StubArea51
Trainer
Trainer
Posts: 1739
Joined: Fri Aug 10, 2012 6:46 am
Location: stubarea51.net
Contact:

Re: L3HW Offloading - Doesn't Offload Inter-VLAN traffic

Mon Feb 20, 2023 5:10 pm

One thing that would help to disambiguate:

l3 hw offload - stateless offload of IPv4/IPv6 routes into hardware
l3 fw offload - stateful offload of IPv4 connections and NAT (IPv6 fastpath/fasttrack yet to be implemented)

Brief list of what we discovered with fw offload in our lab and in prod for an ISP CG-NAT gw. More details in this thread: viewtopic.php?t=183142#p985541

Limitations of fw-offload (as of 7.6) -

- No LACP
- No VLANs
- No hairpinning on a single port
- must have dedicated in and out physical interfaces

Config requirements (once you've removed limitations above:

Must have l3-hw-offloading=yes on the switch chip *but* disabled on all ports
Must create mangle rules to use fasttrack on connections that you want to offload to hardware

MikroTik may be able to comment if the gap on these limitations has closed between 7.6 and 7.8rc2.
 
blacksnow
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 50
Joined: Wed Feb 15, 2023 4:46 pm

Re: L3HW Firewall Offloading - Doesn't Offload Inter-VLAN traffic

Mon Feb 20, 2023 5:25 pm

Updated title to reflect the situation of the issue clearly. Specifically, the FW compatible L3HW offload is not working with inter-vlan traffic. Simply doing L3HW offloading with no firewall expectations which is basically L2 switching + VLAN (L3) making it L3HW connected routes offloaded switching is not what I'm referring to in my post. This is soley about firewall compatible L3HW offloading (stateful firewall + NAT).
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 2989
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: L3HW Offloading - Doesn't Offload Inter-VLAN traffic

Mon Feb 20, 2023 6:08 pm

One thing that would help to disambiguate:

l3 hw offload - stateless offload of IPv4/IPv6 routes into hardware
l3 fw offload - stateful offload of IPv4 connections and NAT (IPv6 fastpath/fasttrack yet to be implemented)

Brief list of what we discovered with fw offload in our lab and in prod for an ISP CG-NAT gw. More details in this thread: viewtopic.php?t=183142#p985541

Limitations of fw-offload (as of 7.6) -

- No LACP
- No VLANs
- No hairpinning on a single port
- must have dedicated in and out physical interfaces

Config requirements (once you've removed limitations above:

Must have l3-hw-offloading=yes on the switch chip *but* disabled on all ports
Must create mangle rules to use fasttrack on connections that you want to offload to hardware

MikroTik may be able to comment if the gap on these limitations has closed between 7.6 and 7.8rc2.

This implies that if i use L3 FW offload I can't use L3 HW offload too, FW and HW simultaneously ?
 
User avatar
StubArea51
Trainer
Trainer
Posts: 1739
Joined: Fri Aug 10, 2012 6:46 am
Location: stubarea51.net
Contact:

Re: L3HW Offloading - Doesn't Offload Inter-VLAN traffic

Mon Feb 20, 2023 6:14 pm

This implies that if i use L3 FW offload I can't use L3 HW offload too, FW and HW simultaneously ?

I would say that's accurate because in fw offload you have to disable hw-offload on the port level (but enabled at the switch level) so the only way traffic is getting hw-offloaded is if it hits a mangle rule that enables fasttrack. Presumably this is so that mangle can enable/disable offload per flow instead of per port.

It could be that development hasn't reached a point where they can be used together and not a permanent limitation.
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 2989
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: L3HW Firewall Offloading - Doesn't Offload Inter-VLAN traffic

Mon Feb 20, 2023 6:23 pm

@IPANetEngineer

thank you for your answer
 
blacksnow
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 50
Joined: Wed Feb 15, 2023 4:46 pm

Re: L3HW Firewall Offloading - Doesn't Offload Inter-VLAN traffic

Mon Feb 20, 2023 6:24 pm

You can use them simultaneously, just you need to set the switch (port level) hw-offloading to enabled. But in doing so for that particular interface port you will not be able to use FW L3HW offload and HW offload at the same time. So in short, per documentation, at the port level you can only use one or the other but at the device level (CCR) you can have some ports that do L3HW FW offload and some that do L3HW connected routes offload.
 
User avatar
StubArea51
Trainer
Trainer
Posts: 1739
Joined: Fri Aug 10, 2012 6:46 am
Location: stubarea51.net
Contact:

Re: L3HW Firewall Offloading - Doesn't Offload Inter-VLAN traffic

Mon Feb 20, 2023 6:26 pm

That's prob a better way to say it.

More accurately - you can't use them together on the same set of ports.
 
blacksnow
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 50
Joined: Wed Feb 15, 2023 4:46 pm

Re: L3HW Firewall Offloading - Doesn't Offload Inter-VLAN traffic

Tue Feb 21, 2023 9:36 pm

Any thoughts on this, is it possible for a MikroTik engineer to replicate to at least confirm/deny if this is a bug or not?
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 2989
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: L3HW Firewall Offloading - Doesn't Offload Inter-VLAN traffic

Thu Feb 23, 2023 12:41 am

this is a users forum

you must open a ticket with support:


https://help.mikrotik.com/servicedesk/servicedesk
 
blacksnow
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 50
Joined: Wed Feb 15, 2023 4:46 pm

Re: L3HW Firewall Offloading - Doesn't Offload Inter-VLAN traffic

Thu Mar 02, 2023 11:29 pm

I sent an email (as I don't have access to this servicedesk). Unfortunately nothing as of yet. Any possibility for any Mikrotik people to try and setup two VLANs on a bridge with each respective VLAN assigned to a bridge port that is associated with a different underlying physical interface and verify that the L3HW offloading works for -> inter-VLAN <- traffic? Additionally, I can also confirm that just having 2 VLANs on a bridge that is associated with the same underlying physical interface also doesn't work for L3HW offloading. This is all tested on 7.8 stable.

Also if any forum members know of a version of RouterOS where L3HW offloading was or is working for inter-vlan traffic please let me know as I would like to downgrade and test.
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 2989
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: L3HW Firewall Offloading - Doesn't Offload Inter-VLAN traffic

Fri Mar 03, 2023 12:22 am

just today i deployed a CRS 317 doing inter VLAN routing with L3 HW offload 2 gbps of traffic with 1% of CPU usage, working ok with Ros 7.6

in my profile you can contact me to help you to solve the issue
 
biomesh
Long time Member
Long time Member
Posts: 561
Joined: Fri Feb 10, 2012 8:25 pm

Re: L3HW Firewall Offloading - Doesn't Offload Inter-VLAN traffic

Fri Mar 03, 2023 12:41 am

My crs317 with 7.8 works fine with inter and intra vlan traffic on the same or different interfaces.
 
blacksnow
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 50
Joined: Wed Feb 15, 2023 4:46 pm

Re: L3HW Firewall Offloading - Doesn't Offload Inter-VLAN traffic

Fri Mar 03, 2023 4:40 am

Do either of you mind posting a sanitized network config from your CRS that show how you are doing your L3HW offloading I might be able to glean something from it. I suspect maybe there is an issue between how the CRS is doing it versus the CCR.
 
blacksnow
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 50
Joined: Wed Feb 15, 2023 4:46 pm

Re: L3HW Firewall Offloading - Doesn't Offload Inter-VLAN traffic  [SOLVED]

Fri Mar 03, 2023 6:32 am

Jesus, after a ridiculous amount of testing I figured out the problem. If you have IP-Firewall set as active and then set either of the two options like "Use IP Firewall For VLAN" or "Use IP Firewall For PPPoE" set, then it doesn't work. And I was told by a Mikrotik member to disable IP-Firewall and I did and there was no change. However, this is due to a bug, you need to first disable "Use IP Firewall For VLAN" and "Use IP Firewall For PPPoE" while the parent option "Use IP Firewall" is enabled and then disable the parent option "Use IP Firewall". Otherwise even though IP Firewall must be enabled for either of the lower two options to be selectable, disabling the parent option doesn't actually disable the other options so somewhere in the code the router still thinks the "Use IP Firewall For VLAN" and "Use IP Firewall For PPPoE" are still enabled even when the parent option "Use IP Firewall" is disabled.

Long story short, if you want L3HW FW Compatible Offloading, disable "Use IP Firewall" and then if you don't see "Bridge Fast Path Active" checked (make sure Allow Fast Path is checked as well), then try the below steps and when it becomes checked then you know you've got it working.

1) Enable Use IP Firewall (if it isn't already).
2) Disable Use IP Firewall For VLAN
2) Disable Use IP Firewall For PPPoE
3) Disable Use IP Firewall
4) Restart router.

@Mikrotik Mods, this thread can be closed since the problem is solved.
 
User avatar
raimondsp
MikroTik Support
MikroTik Support
Posts: 267
Joined: Mon Apr 27, 2020 10:14 am

Re: L3HW Firewall Offloading - Doesn't Offload Inter-VLAN traffic

Mon Mar 06, 2023 11:01 am

Jesus, after a ridiculous amount of testing I figured out the problem. If you have IP-Firewall set as active and then set either of the two options like "Use IP Firewall For VLAN" or "Use IP Firewall For PPPoE" set, then it doesn't work. And I was told by a Mikrotik member to disable IP-Firewall and I did and there was no change. However, this is due to a bug, you need to first disable "Use IP Firewall For VLAN" and "Use IP Firewall For PPPoE" while the parent option "Use IP Firewall" is enabled and then disable the parent option "Use IP Firewall". Otherwise even though IP Firewall must be enabled for either of the lower two options to be selectable, disabling the parent option doesn't actually disable the other options so somewhere in the code the router still thinks the "Use IP Firewall For VLAN" and "Use IP Firewall For PPPoE" are still enabled even when the parent option "Use IP Firewall" is disabled.

Long story short, if you want L3HW FW Compatible Offloading, disable "Use IP Firewall" and then if you don't see "Bridge Fast Path Active" checked (make sure Allow Fast Path is checked as well), then try the below steps and when it becomes checked then you know you've got it working.

1) Enable Use IP Firewall (if it isn't already).
2) Disable Use IP Firewall For VLAN
2) Disable Use IP Firewall For PPPoE
3) Disable Use IP Firewall
4) Restart router.

@Mikrotik Mods, this thread can be closed since the problem is solved.

I'm glad that L3HW FW-Compatible Offloading finally works on your side. We have identified an issue where disabling "Use IP Firewall" does not entirely disable the firewall if VLAN or PPPoE options are still enabled. It will be fixed in future releases. Meanwhile, disable all three options to make FastPath work.

Thank you for the feedback!

Who is online

Users browsing this forum: No registered users and 24 guests