There are two types of Hardware Routing (L3HW): Full Hardware Routing and Firewall-Compatible Hardware Routing. Full L3HW, in turn, differs between routing via an explicit nexthop gateway(-s) and routing to a connected L2 network (a.k.a. Connected Routes). As a result, there are three different cases, so let’s describe each of them.
1. Full Hardware Routing via Nexthop Gateway(-s)
Full Hardware Routing is set by enabling l3-hw-offloading both on the switch and switch ports:
/interface/ethernet/switch set 0 l3-hw-offloading=yes
/interface/ethernet/switch port set [find] l3-hw-offloading=yes
Now, let’s define a routing via a direct gateway. For instance, let’s route the 10.0/8 network via 192.168.1.1:
/ip/route add add dst-address=10.0.0.0/8 gateway=192.168.1.1
And then check the “H” flag to make sure that the route got offloaded:
/ip/route print
Flags: D - DYNAMIC; A - ACTIVE; c, s, y - COPY; H - HW-OFFLOADED; + - ECMP
Columns: DST-ADDRESS, GATEWAY, DISTANCE
# DST-ADDRESS GATEWAY DISTANCE
0 AsH 10.0.0.0/8 192.168.1.1 1
Offloading the above rule tells the switch chip to route all packets with destination IP in subnet 10.0/8 to 192.168.1.1. None of the packets to the 10.0/8 will ever enter the CPU. Hence, none of the IP Firewall rules will ever trigger. That’s why Full L3HW is incompatible with IP Firewall. On the other hand, Full L3HW is the fastest one.
However, you may offload some traffic control via switch ACL rules:
/in/eth/sw/rule
So if a limited stateless firewall is enough for the given task, you may continue with Full L3HW + ACL Rules, reaching near wire-speed performance.
2. HW-Offloading Connected Routes
That is also part of Full L3HW, but this time we specify an interface as a gateway instead of a nexthop IP. RouterOS creates Connected Routes automatically (dynamically) when assigning an IP address to an interface. In the next example, we set IP addresses to ether1, vlan32, and vlan40 interfaces, which resulted in connected routes creation in the respective networks.
/ip address
add address=192.168.1.17/24 interface=ether1 network=192.168.1.0
add address=192.168.32.17/24 interface=vlan32 network=192.168.32.0
add address=192.168.40.17/24 interface=vlan40 network=192.168.40.0
/ip/route print
Flags: D - DYNAMIC; A - ACTIVE; c, s, y - COPY; H - HW-OFFLOADED; + - ECMP
Columns: DST-ADDRESS, GATEWAY, DISTANCE
# DST-ADDRESS GATEWAY DISTANCE
DAcH 192.168.1.0/24 ether1 0
DAcH 192.168.32.0/24 vlan32 0
DAcH 192.168.40.0/24 vlan40 0
To understand how Connected Routes can be offloaded, we need to know how Connected Routes are processed. Let’s imagine an Inter-VLAN routing case where a host 192.168.32.5 (vlan32) wants to connect to 192.168.40.10 (vlan40). The former sends a packet to our router (192.168.32.17). The router checks the routing table and identified the destination is somewhere in vlan40. So it broadcasts an ARP request “Who has 192.168.40.10?” to all bridge ports that belong to vlan40 (“/in/br/vlan print where vlan-ids=40”). Let’s say the router gets the reply from sfp-sfpplus5 (which belongs to vlan40) and learns its L2 (MAC) address. Now, the router knows the physical interface and MAC address of the destination 192.168.40.10, caching the data in its Forwarding Database (FDB) for reuse in subsequent packet routing to the same IP address.
The answer “How Connected Routes can be offloaded?” is “They cannot”. The switch chip cannot send ARP requests and resolve new hosts inside the subnet. RouterOS (CPU) does that instead. Connected Routes are redirected to CPU by default, but the resolved hosts are offloaded to the hardware as /32 routes. For instance, “dst-address=192.168.40.10/32 gateway=192.168.40.10%sfp-sfpplus5 VLAN-ID=40”. That’s why there is a remark in the documentation telling:
H-flag does not indicate that route is actually HW offloaded, it indicates only that route can be selected to be HW offloaded.
A connected route having the “H” flag means that the hosts within the subnet can be hw-offloaded. The subnet itself stays on the CPU. However, once both source and destination hosts are offloaded (as /32 routes), Full Hardware Routing is established between them, offering near wire-speed performance.
Since L3HW depends on L2HW, it utilizes the full capability of the hardware L2 processing, including VLAN tagging/untagging. In other words, Full Hardware Routing provides Inter-VLAN routing between tagged and/or untagged ports out of the box. And it works in RouterOS v7.1 too. The changes in v7.2 are related to FastTrack only, which we’ll discuss in the next chapter.
3. FastTrack Connection HW Offloading
Full HW Routing is incompatible with the stateful IP Firewall. So the tradeoff must be made between speed and security. Even NAT, which requires connection tracking – a feature of stateful L4 Firewall – is incompatible with Full HW Routing. Fortunately, some device models support FastTrack Offloading, allowing to achieve near wire-speed routing performance for a limited amount of connections while keeping the Firewall running. In other words, FastTrack Offloading provides Hardware-accelerated L4 Stateful Firewall.
Firewall-Compatible HW Routing is set by disabling l3-hw-offloading on the switch ports, where Firewall rules must be applied:
# Enable full hardware routing on LAN ports
:foreach i in=[/interface/list/member/find where list=LAN] do={
/interface/ethernet/switch/port set [/interface/list/member/get $i interface] l3-hw-offloading=yes
}
# Disable full hardware routing on WAN or IoT ports
:foreach i in=[/interface/list/member/find where list=WAN or list=IoT] do={
/interface/ethernet/switch/port set [/interface/list/member/get $i interface] l3-hw-offloading=no
}
# Activate Layer 3 Hardware Offloading on the switch chip
/interface/ethernet/switch/set 0 l3-hw-offloading=yes
- Packets are processed by the CPU/Firewall by default.
- Established connections can be fast-tracked - a faster (shorter) processing path, resulting in higher speeds.
- FastTrack connections can be offloaded to the switch chip (if the latter supports that), offering near wire-speed performance.
- If a FastTrack connection requires network address translation, the NAT rule gets offloaded too.
Before v7.2, RouterOS didn’t support FastPath on vlan-filtered bridges, meaning no FastTrack for Inter-VLAN routing and no HW offloading. The feature has been introduced in RouterOS v7.2rc2, allowing Firewall-Compatible Inter-VLAN Routing.
Difference Between Connected Routes and FastTrack Connection Offloading
While it may seem that those two follow a similar pattern (redirect to CPU first, then offload to HW), there are fundamental differences.
- When a connected host gets offloaded (/32 route), all traffic to it gets routed by the hardware, bypassing the firewall. For instance, once your computer got connected to a server, the server’s IP gets offloaded to the hardware, and any device on the network can access it, including a random IoT device from an insecure VLAN.
- FastTrack Offloading applies to the offloaded connections only, and only until the respective connections are closed. If you’re connected to the server, nobody else can connect to it without going through the Firewall first.
- Amount of connected hosts that can be offloaded greatly exceeds the number of hardware FastTrack connections. At the moment of writing, the highest amount of HW FastTrack connection is 4.5k while some devices can offload up to 128k hosts (e.g. CRS317).
- When possible, try establishing Full HW Routing between trusted networks (e.g., between admin and server VLANs), leaving Firewall for crossing the unsecured zone (or NAT).
- Fine-tune FastTrack connections that can be HW-offloaded via firewall filter rules by setting hw-offload=yes|no. For instance, there is no need to HW-offload low-bandwidth connections, such as a smart power socket or a fridge controller.
