Hardware Offload not applying to most routes CRS354 (7.23.2)

RouterOS General
1

I have a CRS354 running 7.23.3, and im having issues getting Hardware offloading to trigger for most of my routes, everything that should have the H flag for offloading has it, ive checked that the IP firewalling on the bridge is disabled, DHCP Snooping is off, the Bond im using is on 802.3d, but the l3FW page only shows 1 route of 58. Ive also tried toggling the switch L3HW setting on and off after making changes

CRS354_conf.txt (17.1 KB)
heres my config

and the output of l3hw monitor

        ipv4-routes-total:   60
           ipv4-routes-hw:    1
          ipv4-routes-cpu:   58
  ipv4-shortest-hw-prefix:    0
               ipv4-hosts:   77
        ipv6-routes-total:    3
           ipv6-routes-hw:    0
          ipv6-routes-cpu:    3
  ipv6-shortest-hw-prefix:    0
               ipv6-hosts:    0
         route-queue-size:    0
              nexthop-cap: 8192
            nexthop-usage:   85
    vxlan-mtu-packet-drop:    0
     fasttrack-ipv4-conns:    0
   fasttrack-hw-min-speed:    0

and my route table

Flags: A - ACTIVE; c - CONNECT, s - STATIC; H - HW-OFFLOADED
Columns: DST-ADDRESS, GATEWAY, AFI, ROUTING-TABLE, DISTANCE, SCOPE, TARGET-SCOPE, IMMEDIATE-GW
    DST-ADDRESS          GATEWAY              AFI   ROUTING-TABLE  DISTANCE  SCOPE  TARGET-SCOPE  IMMEDIATE-GW       
AsH 0.0.0.0/0            10.2.0.1             ip    main                  1     30            10  10.2.0.1%uplink    
AcH 10.0.0.0/24          build-01             ip    main                  0     10             5  build-01           
AcH 10.0.50.0/24         network_analysis-01  ip    main                  0     10             5  network_analysis-01
AcH 10.0.110.0/24        prod-01              ip    main                  0     10             5  prod-01            
AcH 10.2.0.0/24          uplink               ip    main                  0     10             5  uplink             
AcH 192.168.5.0/24       bridge               ip    main                  0     10             5  bridge             
AcH 192.168.10.0/24      ilo                  ip    main                  0     10             5  ilo                
AcH 192.168.11.0/24      freenas              ip    main                  0     10             5  freenas            
AcH 192.168.12.0/24      pve                  ip    main                  0     10             5  pve                
AcH 192.168.13.0/24      power                ip    main                  0     10             5  power              
AcH 192.168.14.0/24      management           ip    main                  0     10             5  management         
AcH 192.168.15.0/24      wifi                 ip    main                  0     10             5  wifi               
AcH 192.168.16.0/24      vm                   ip    main                  0     10             5  vm                 
AcH 192.168.20.0/24      wifi-guest           ip    main                  0     10             5  wifi-guest         
AcH 192.168.88.0/24      bridge               ip    main                  0     10             5  bridge             
A   lo                                        link  main                  0                                          
A   sfp-sfpplus1                              link  main                  0                                          
A   sfp-sfpplus2                              link  main                  0                                          
A   sfp-sfpplus3                              link  main                  0                                          
A   sfp-sfpplus4                              link  main                  0                                          
A   ether1                                    link  main                  0                                          
A   ether2                                    link  main                  0                                          
A   ether4                                    link  main                  0                                          
A   ether5                                    link  main                  0                                          
A   ether6                                    link  main                  0                                          
A   ether7                                    link  main                  0                                          
A   ether8                                    link  main                  0                                          
A   ether9                                    link  main                  0                                          
A   ether11                                   link  main                  0                                          
A   ether15                                   link  main                  0                                          
A   ether16                                   link  main                  0                                          
A   ether26                                   link  main                  0                                          
A   ether31                                   link  main                  0                                          
A   ether32                                   link  main                  0                                          
A   ether33                                   link  main                  0                                          
A   ether34                                   link  main                  0                                          
A   ether36                                   link  main                  0                                          
A   ether41                                   link  main                  0                                          
A   ether42                                   link  main                  0                                          
A   ether43                                   link  main                  0                                          
A   ether44                                   link  main                  0                                          
A   ether45                                   link  main                  0                                          
A   ether46                                   link  main                  0                                          
A   bridge                                    link  main                  0                                          
A   bonding1                                  link  main                  0                                          
A   build-01                                  link  main                  0                                          
A   prod-01                                   link  main                  0                                          
A   network_analysis-01                       link  main                  0                                          
A   pve                                       link  main                  0                                          
A   freenas                                   link  main                  0                                          
A   ilo                                       link  main                  0                                          
A   management                                link  main                  0                                          
A   power                                     link  main                  0                                          
A   uplink                                    link  main                  0                                          
A   vm                                        link  main                  0                                          
A   wifi                                      link  main                  0                                          
A   wifi-guest                                link  main                  0
2

All of those routes with the "H" to the left of them (AcH) are hardware-offloaded. They're also all connected to the router itself, and therefore go straight to the CPU. The only one that isn't is your static route.

I don't see any other routes that could be offloaded. Those Port/link routes aren't IP routes that you can do anything about.

3

Ok, so the most i can really get improvement wise is making sure that Bridge fast path stays enabled?

4

That's definitely one way of looking at it. But if you look at the ipv4-hosts counter, that shows that packets to those directly connected hosts are routed in hardware. So if OP wants hw forwarding between hosts on their vlans, then these stats show that this result is successfully achieved.

5

The root issue ive been trying to fix is that the switch shuts down occasionally after a decent period of time running as expected, about a month or so, and it seems like its a RAM issue that presents itself mainly when the bond is connected

Reading another thread, tho i need to test it, this might be related to my MTU settings, i was hoping to implement jumbo frames but that might have pushed the ram usage up too high

6

Unless there's a feature of 7.23 that you need, try 7.21 first. Or even 7.19.4/6. That will surface whether or not the reboot is due to a bug.

I have dozens of switches running 7.19.4 with L3HW offload enabled and they work great (and have been since 7.19.x came out).

7

Im not sure if its a bug or not, but the CRS354's are supposed to be able to HW offload intervlan traffic so i was confused why the routes werent reporting as being HW offloaded. I can probably close this thread, I think my actual issue is just ram usage right now theres only 12 to 15 mbs free at a time and something seems to be pushing it to where it kernel panics.

8

Are you running NAT on it too?

9

No, this just handles internal routing between VLANs, i have an upstream router that handles internet traffic/NAT/etc