I think I read all of the post about hw offloading already and also went multiple times through https://help.mikrotik.com/docs/display/ROS/L3+Hardware+Offloading to have a good understanding how it should work. Also I am aware that the CRS310 is a switch with poor routing performance - but I wanted to use it as an example to test my 10Gbit fiber connection towards my ISP using l3-hw-offloading to gain near wire speed.
The issue is, I am not able to bring l3-hw-offloading to work and I am out of ideas.
The following configuration works fine (l3-hw-offloading=no) but with poor performance with 100% cpu utilization on /tools/profile and about 500mbit speed testing from sfp-sfpplus2 (RJ45 10Gbit) to sfp-sfpplus1 (10Gbit fiber - ISP):
# 2024-08-25 08:37:52 by RouterOS 7.15.3
# software id = 6QMK-65QJ
#
# model = CRS310-8G+2S+
# serial number = HG209JDP7ZV
/interface bridge
add name=bridge1 pvid=10 vlan-filtering=yes
/interface vlan
add interface=bridge1 name=vlan10 vlan-id=10
add interface=bridge1 name=vlan20 vlan-id=20
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=dhcp_pool0 ranges=192.168.22.20-192.168.22.254
/ip dhcp-server
add address-pool=dhcp_pool0 interface=vlan20 name=dhcp1
/interface bridge port
add bridge=bridge1 interface=ether2 pvid=20
add bridge=bridge1 interface=ether3 pvid=20
add bridge=bridge1 interface=ether4 pvid=20
add bridge=bridge1 interface=ether5 pvid=20
add bridge=bridge1 interface=ether6 pvid=20
add bridge=bridge1 interface=ether7 pvid=20
add bridge=bridge1 interface=ether8 pvid=20
add bridge=bridge1 interface=sfp-sfpplus1 pvid=10
add bridge=bridge1 interface=sfp-sfpplus2 pvid=20
/interface bridge settings
set use-ip-firewall=yes
/ip settings
set rp-filter=loose
/interface bridge vlan
add bridge=bridge1 tagged=bridge1 untagged=sfp-sfpplus1 vlan-ids=10
add bridge=bridge1 tagged=bridge1 untagged=ether2,ether3,ether4,ether5,ether6,ether7,ether8,sfp-sfpplus2 vlan-ids=20
# /interface ethernet switch
# set 0 l3-hw-offloading=yes
/ip address
add address=192.168.88.1/24 interface=ether1 network=192.168.88.0
add address=192.168.22.1/24 interface=vlan20 network=192.168.22.0
/ip dhcp-client
add interface=vlan10
/ip dhcp-server network
add address=192.168.22.0/24 dns-server=8.8.8.8 gateway=192.168.22.1
/ip dns
set allow-remote-requests=yes servers=1.1.1.1,8.8.8.8
/ip firewall filter
add action=accept chain=input connection-state=established,related
add action=accept chain=forward in-interface=vlan20 out-interface=vlan10
add action=drop chain=input in-interface=vlan10
/ip firewall nat
add action=masquerade chain=srcnat out-interface=vlan10
/ip service
set telnet disabled=yes
set ftp disabled=yes
set api disabled=yes
set api-ssl disabled=yes
/system clock
set time-zone-name=Europe/Zurich
/system note
set show-at-login=no
/system ntp client
set enabled=yes
/system routerboard settings
set boot-os=router-os
Now as soon I switch on l3-hw-offloading the connection to the internet drops.
/interface ethernet switch
set 0 l3-hw-offloading=yes
I can reach the public ip address assigned via dhcp client on vlan10, but not the next hop at my ISP anymore. Switching back to l3-hw-offloading=no, everything works fine,
Hi. Considering the time, I hope that you already resolve the issue. But in any case the settings above causes the slow-down. It tells the bridge to process all communication by CPU and even disables the FastPath…
But another issue is that the L3HW offloading of this switch cannot work with active firewall…. which is something I would like to deal with in my setup….
This is your problem CRS 310 just like you said it’s not a workhorse for doing NAT which will eat your CPU because it’s a switch, what i’ve learned from mkx and other people here is if it’s a switch don’t make it a router if it’s a router don’t make it a switch
l3-hw-offloading is different for accelerated NAT which your hardware doesn’t have
Actually I find the switch quite powerfull anyway. At least here, where gigabit connectivity to ISP is still not quite common. The CPU of the switch seems to be very reasonable for the 1,3 Gbps CPU bus - it is not far behind the router such like L009 (for more CPU demanding tasks - e.g. routing with 25 FW rules - is just approx 25% slower). The main drawback is that the L3HW doesn’t support fas-track offloading. This could really make the configuration much easier. Without this you must be very carefull…
But even without this feature there are some workarounds…
Typical scenario where this switch could provide reasonable routing performance is when 1 Gbps connectivity to ISP/WAN is sufficient and you will benefit from fast communication on LAN only - such as using one or more NAS (e.g. with bonded connection) so computers could work with files directly on NAS. But even in such case the configuration is not straightforward (due to missing fast.track offloading). The router actually tries to offload all routing so some communication might be tracked improperly…
What works in my case - several internal VLANs with one VLAN on trunk interface to serve as gateway:
Standard definition of ports and VLANs in bridge (hw offloaded)
static route to default gateway with supressed HW offloading (in /ip/route)
ACL rule on /etherent/switch/rule which sends traffic from GW interface (at particular VLAN as it is on trunk IF) directly to CPU (for proper connection tracking)
enabled full L3HW on switch and all physical ports
and just for sure I have also ACL rule to pass any communication intended for the router’s L3 IF to the CPU, just to be sure (as I still haven’t finished all the troubleshooting of the configuration)
With this scenario you can benefit from fully offloaded inter-VLAN routing on LAN and have CPU/firewall for processing WAN traffic (NAT etc., even including fast-track). The offloaded LAN communication cannot run via FW (it will be probably invisible for CPU), but you still can implement some FW rules using ACL to either block some communication or at least to send particular communication via CPU - the ACL matcher (stateless firewall) is still quite powerfull (switch chip supports 128 ACL rules) - see this for details: L3 Hardware Offloading - RouterOS - MikroTik Documentation
The main thing tho keep in mind when considering multiple architectures and features of various HW, which I havent’t seen explicitely described, is that even if fast-track is near-wire-speed, this wire-speed is related to CPU bus. Fast-track can be processed without consuming CPU too much, but the traffic still flows via the CPU bus. This can bee seen on fast-path performance of CRS3x switches vs routers. But the L3HW doesn’t use the CPU bus and this is why their throughput can be so high…. as long as the limitations in routing will not interfere with your use case….and in the same time L3HW feature on router could provide benefit only if there is bottleneck between switch chip and CPU. Otherwise the fast-path will serve enough and the configuration will be more elegant (there were some questions related to RB5009 and L3HW, but the bottleneck here is not that big).
On the other hand, you might be able co create appropriately scaled solution using CRS3x switches and routers (in case of ports, speed etc.) for better price than using bulky router.
I have small update to CRS310 and L3HW capabilities, probably applicable to all CRS3x without fast-track offloading support and with ROS bellow 7.21 (which is currently still no a stable release):
Simply said, if some communication shall pass the FW, it can’t be HW offloaded. And It aso didn’t work correctly was the combination of several VLAN’s in the trunk port, where some shall be offloaded and some shall not. The per-VLAN offloading capability is currently mentioned for ROS v.7.21 in the documentation.
But with 7.20 and bellow, for stable performance, it seems that the only solution is to place FW (NAT,whatever) between ports with L3HW ON and L3HW off and never mix them - never put VLAN which shall be routed via HW on ports containing VLAN where FW/NAT/etc is needed.
My previous solution seemed to work…but after time, it was not stable and some routes didn’t work… until I reduce the bonded interface and put the non-offloaded VLAN on separate ethernet port with L3HW disabled. After that (and updated FW filter), it is stable…finally
