L3hw confusion

Hi to all

well, iam confused with l3hw stats

device: ccr2216 ros 7.21.4
ports: excluded from L3HW
one vlan filtered bridge
L2hw present on ports
goal: use ip firewall filter / fasttrack / l3hw

if i look at connections, there is plenty of items with "H" flags
but, if i look at l3hw stat, something is confusing

this counter: fasttrack HW unloaded
maybe i am missing something ? why the connections are unloaded ?

firewall is simple one, first rule to offload conn to fasttrack/hw
nothing complicated
bridge filter is empty
switch rules are empty

so, why these conn are unloaded from HW ?

There's a limit of how many fasttrack connections can be offloaded to HW. Have a look at L3HW Device Support tables. Numbers for CCR2216 are around 4k ... and IIRC it's been said that memory, required for holding those offloaded connections, is a pool somewhat shared between different types (so it could be that NATed connections share same pool as routes or something) ... meaning that real-life numbers might be (significantly) lower than the maximums stated ... depending on particular use case and mix of traffic.

The comments below tables say:

2 When the HW limit of Fasttrack or NAT entries is reached, other connections will fall back to the CPU. MikroTik's smart connection offload algorithm ensures that the connections with the most traffic are offloaded to the hardware.

3 Fasttrack connections share the same HW memory with ACL rules. Depending on the complexity, one ACL rule may occupy the memory of 3-6 Fasttrack connections.

4 If a Fasttrack connection requires Network Address Translation, a hardware NAT entry is created. The hardware supports both SRCNAT and DSTNAT.

which more or less explains the mix of "offloaded" and "unloaded" stats.

I'm guessing that when l3hw tables get full and further connections get "unloaded", if l3hw tables get freed up, the previously "unloaded" connections might get offloaded ... but only some of them and after a while. Which would explain the fact that even if l3hw offload tables are not full, there are still some "unloaded" connections.

thank you for response
sadly, i tried
/interface/ethernet/switch/l3hw-settings/advanced> monitor once
but there is no /advanced so could not copy&paste cli output here

about NAT .. no i have no NAT on this device
only forward/input filters & fasttrack, nothing more

what confuses me is
ft hw offloaded & ft hw unloaded are same
maybe i get this wrong, but i have feeling that everything what is offloaded is at same time unloaded ? :frowning: ?

then, connection tracking have lot of "H" connections, which suggest that l3hw is working
cpu cca 20% with l3, 35% without l3

that is my problem, i could not logicaly interpret these numbers :frowning: in L3HW stat