We are planning to use a CCR2216 or CCR2004 as main firewall for a bunch of servers. While doing a POC with a CRS326 and a CRS354-48P-4S+2Q+, we found performance issues around HW-offloading and wonder if we're doing something wrong or if our plan will fail.
Intended goal:
Transparent firewall with FastTrack for "connection=established,related" and very few other firewall rules. No routing/NAT, only bridge/switch. Currently no VLAN, later on having a few VLANs (tagged and port-based).
Uplink through ether1 (during testing, using SFP+ in production), the rest for clients/servers/switches.
(Currently testing with CRS354 at gigabit, later on doing CCR2216/CCR2004 at 10/25/40/100Gbit depending on results here)
Expected results:
Getting high performance going from ether2->ether3 with iperf3.
Getting high performance going from ether2->ether1->outside with iperf3.
Counters being non-zero for firewall rules "special dummy rule to show fasttrack counters".
Low CPU load when doing a single TCP stream.
Actual results:
Getting about 950Mbit/s going from ether2->ether3 with iperf3
Getting slightly under 100Mbit going from ether2->ether1->outside with iperf3 (CRS326 managed ~350Mbit due to faster CPU in the same situation)
Bypassing the CRS354 gives 950Mbit.
Counters for firewall rule "special dummy rule to show fasttrack counters" stays at 0, same for those counters under mangle.
CPU maxed out according to /tool/profile (firewall+networking+bridging).
Leading notes:
We first tried with the CRS326, but later found in the docs that the CRS326 switch chip can't do FastTrack.
https://help.mikrotik.com/docs/display/ ... iceSupport
"These devices do not support Fasttrack or NAT connection offloading."
The CRS354 is listed to have 2.25k Fasttrack connections, so we switched testing to that one.
As far as we have understood, we need hw=no on the ethernet uplink for the firewall rules to be applied at all. Adding hw=yes skips all firewall rules and gives us 950Mbit/s.
Steps to reproduce:
Do empty reset (with Keep Users) of the CRS354, running RouterOS v7.7.
Do config (protocol-mode=none is to avoid this switch being blocked by the next Cisco hop):
Code: Select all
/interface bridge
set [ find name=bridge ] protocol-mode=none
/interface ethernet switch
set 0 l3-hw-offloading=yes
/interface bridge port
set [ find interface=ether1 ] hw=no
/interface bridge settings
set use-ip-firewall=yes use-ip-firewall-for-pppoe=no use-ip-firewall-for-vlan=no
/ip firewall connection tracking
set enabled=yes
/ip firewall filter
add action=fasttrack-connection chain=forward comment=FastTrack connection-state=established,related hw-offload=yes
add action=accept chain=forward comment="Established, Related" connection-state=established,related
add action=accept chain=forward
add action=accept chain=input
add action=accept chain=output
Then doing iperf3 ether2->ether1->outside => ~100Mbit.
Code: Select all
/interface bridge port
set [ find interface=ether1 ] hw=yes
Are we doing something wrong? We have been reading everything we found on help.mikrotik.com regarding hardware offloading, and various forums that even for CSR354 just says "add a fasttrack forward".
https://wiki.mikrotik.com/wiki/Manual:C ... Offloading
"Only Fasttrack connections gets processed by HW, which means that CPU is processing packets until connection gets fasttracked."
"Warning: Currently user must choose whether to use hardware accelerated routing or firewall. It is not possible to use both at the same time." - we are not doing any routing that we know of. Tried removing any trace of default-setup-routing to no avail.