Community discussions

MikroTik App
 
StricUmea
just joined
Topic Author
Posts: 3
Joined: Tue Feb 07, 2023 12:42 pm

Hardware offloading FastTrack on CRS354 not happening

Tue Feb 07, 2023 1:23 pm

Intro:
We are planning to use a CCR2216 or CCR2004 as main firewall for a bunch of servers. While doing a POC with a CRS326 and a CRS354-48P-4S+2Q+, we found performance issues around HW-offloading and wonder if we're doing something wrong or if our plan will fail.

Intended goal:
Transparent firewall with FastTrack for "connection=established,related" and very few other firewall rules. No routing/NAT, only bridge/switch. Currently no VLAN, later on having a few VLANs (tagged and port-based).
Uplink through ether1 (during testing, using SFP+ in production), the rest for clients/servers/switches.
(Currently testing with CRS354 at gigabit, later on doing CCR2216/CCR2004 at 10/25/40/100Gbit depending on results here)

Expected results:
Getting high performance going from ether2->ether3 with iperf3.
Getting high performance going from ether2->ether1->outside with iperf3.
Counters being non-zero for firewall rules "special dummy rule to show fasttrack counters".
Low CPU load when doing a single TCP stream.

Actual results:
Getting about 950Mbit/s going from ether2->ether3 with iperf3
Getting slightly under 100Mbit going from ether2->ether1->outside with iperf3 (CRS326 managed ~350Mbit due to faster CPU in the same situation)
Bypassing the CRS354 gives 950Mbit.
Counters for firewall rule "special dummy rule to show fasttrack counters" stays at 0, same for those counters under mangle.
CPU maxed out according to /tool/profile (firewall+networking+bridging).

Leading notes:
We first tried with the CRS326, but later found in the docs that the CRS326 switch chip can't do FastTrack.
https://help.mikrotik.com/docs/display/ ... iceSupport
"These devices do not support Fasttrack or NAT connection offloading."

The CRS354 is listed to have 2.25k Fasttrack connections, so we switched testing to that one.

As far as we have understood, we need hw=no on the ethernet uplink for the firewall rules to be applied at all. Adding hw=yes skips all firewall rules and gives us 950Mbit/s.

Steps to reproduce:
Do empty reset (with Keep Users) of the CRS354, running RouterOS v7.7.
Do config (protocol-mode=none is to avoid this switch being blocked by the next Cisco hop):
/interface bridge
set [ find name=bridge ] protocol-mode=none

/interface ethernet switch
set 0 l3-hw-offloading=yes

/interface bridge port
set [ find interface=ether1 ] hw=no

/interface bridge settings
set use-ip-firewall=yes use-ip-firewall-for-pppoe=no use-ip-firewall-for-vlan=no

/ip firewall connection tracking
set enabled=yes

/ip firewall filter
add action=fasttrack-connection chain=forward comment=FastTrack connection-state=established,related hw-offload=yes
add action=accept chain=forward comment="Established, Related" connection-state=established,related
add action=accept chain=forward
add action=accept chain=input
add action=accept chain=output

Then doing iperf3 ether2->ether1->outside => ~100Mbit.
/interface bridge port
set [ find interface=ether1 ] hw=yes
Same iperf3 => ~950Mbit, but this skips all the firewall rules.

Are we doing something wrong? We have been reading everything we found on help.mikrotik.com regarding hardware offloading, and various forums that even for CSR354 just says "add a fasttrack forward".
https://wiki.mikrotik.com/wiki/Manual:C ... Offloading
"Only Fasttrack connections gets processed by HW, which means that CPU is processing packets until connection gets fasttracked."
"Warning: Currently user must choose whether to use hardware accelerated routing or firewall. It is not possible to use both at the same time." - we are not doing any routing that we know of. Tried removing any trace of default-setup-routing to no avail.
You do not have the required permissions to view the files attached to this post.
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 2989
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: Hardware offloading FastTrack on CRS354 not happening

Tue Feb 07, 2023 6:27 pm

check this

Offloading Fasttrack Connections
https://help.mikrotik.com/docs/display/ ... onnections

info you found on wiki.mikrotik.com is legacy documentation

L3 Hardware offloading is a new feature, so is better to stick with help.mikrotik.com as your source of information about that


related useful info:

Bridging and Switching
https://help.mikrotik.com/docs/display/ ... +Switching

CRS3xx, CRS5xx, CCR2116, CCR2216 switch chip features
https://help.mikrotik.com/docs/display/ ... p+features

L3 Hardware Offloading
https://help.mikrotik.com/docs/display/ ... Offloading
 
StricUmea
just joined
Topic Author
Posts: 3
Joined: Tue Feb 07, 2023 12:42 pm

Re: Hardware offloading FastTrack on CRS354 not happening

Wed Feb 08, 2023 4:56 pm

check this

Offloading Fasttrack Connections
https://help.mikrotik.com/docs/display/ ... onnections

info you found on wiki.mikrotik.com is legacy documentation

L3 Hardware offloading is a new feature, so is better to stick with help.mikrotik.com as your source of information about that


related useful info:

Bridging and Switching
https://help.mikrotik.com/docs/display/ ... +Switching

CRS3xx, CRS5xx, CCR2116, CCR2216 switch chip features
https://help.mikrotik.com/docs/display/ ... p+features

L3 Hardware Offloading
https://help.mikrotik.com/docs/display/ ... Offloading
Thanks for those.. (Un)fortunately, we had browsed most/all of those already and we've tried just about everything we can think about - still no fasttracked connections show up even with pretty much the simplest setup we could think of - the stuff in my first post..

/Tomas
 
EdPa
MikroTik Support
MikroTik Support
Posts: 274
Joined: Fri Sep 15, 2017 10:05 am
Location: Riga
Contact:

Re: Hardware offloading FastTrack on CRS354 not happening

Thu Feb 09, 2023 1:09 pm

Hi StricUmea,

The first thing that stands out:
Transparent firewall with FastTrack for "connection=established,related" and very few other firewall rules. No routing/NAT, only bridge/switch.

Bridged traffic by default is not passed through the IP firewall and connection tracking. Yes, you can enable the "use-ip-firewall" under the bridge settings and pass the L2 traffic additionally through the routing chains (prerouting, forward and postrouting). However, this requires that bridged packets are forwarded on the software bridge (CPU), not ideal for the CRS354 switch. Traffic that is forwarded in HW offloaded bridge will not get passed through routing chains, even with enabled "use-ip-firewall".

In addition, "use-ip-firewall" disables the bridge fast-path feature, which will affect the ability to properly FastTrack connection going over the bridge.

This setup is possible, but only running on the CPU. If you have a CRS354 switch, a much better alternative would be to use switch ACL rules. Or convert the setup from the bridged/switched to routing with connection tracking (supported by L3HW).

The last detail regarding your current configuration. To enable L3HW in FastTrack mode, you do not need to disable the bridge port HW offloading, it should be done under the switch port menu:
https://help.mikrotik.com/docs/display/ ... figuration
 
StricUmea
just joined
Topic Author
Posts: 3
Joined: Tue Feb 07, 2023 12:42 pm

Re: Hardware offloading FastTrack on CRS354 not happening

Fri Feb 10, 2023 10:21 am

However, this requires that bridged packets are forwarded on the software bridge (CPU), not ideal for the CRS354 switch
Thanks for the reponse.. The CRS354 is not the final plan, but rather a POC.. The plan is to use a CCR2216-1G-12XS-2XQ as the main thing and possibly a CCR2004-1G-12S+2XS (or CCR2116) as a fw protecting a "playground". They do have quite a bit more CPU to work with, but would they also have more hw offloading in the "transparent firewall without routing" case? These machines will not be on the edge of our organization, but rather further in (it's a university) - so no routing..

In our tests, our single-threaded iperf3 got about 1/4 of the bandwidth that mikrotik.com -> product -> Test results has.. If that scales, then it might be sufficient even in CPU only mode..

/Tomas
 
EdPa
MikroTik Support
MikroTik Support
Posts: 274
Joined: Fri Sep 15, 2017 10:05 am
Location: Riga
Contact:

Re: Hardware offloading FastTrack on CRS354 not happening

Mon Feb 13, 2023 4:57 pm

L3HW cannot be used in this setup, so those CCRs will also use the CPU.
 
User avatar
raimondsp
MikroTik Support
MikroTik Support
Posts: 267
Joined: Mon Apr 27, 2020 10:14 am

Re: Hardware offloading FastTrack on CRS354 not happening

Fri Feb 17, 2023 4:07 pm

You cannot use Layer 3 Hardware Offloading for Layer 2 operations (bridge forwarding).

Who is online

Users browsing this forum: Bing [Bot], Google [Bot], stevencameron16 and 94 guests