MikroTik

L3HW offloading only works between if all routes reside on same bridge. It seems your WAN is on off-bridge interface sfp-sfpplus1 . This requirement applies only to VLAN. If the WAN port does not need VLAN tagging, it can stay standalone - L3HW will still work. I guess the problem is with NAT. Plea...

Hi,

Please show us your "/interface/export" and "/ip/export" outputs of both VRRP master and backup devices.

Hi,

I suppose this config line causes problems:

add action=masquerade chain=srcnat

since it may masquerade VRRP packets.

Also, if you want to use a VRRP IP address for SNAT'ing, I suggest using "action=src-nat to-address=<vrrp_ip_adress>" instead of masquerade.

Hi raimondsp Hardware level traffic sampling on the Marvell ASIC's to netflow/sflow would be the optimal solution, to quote you "Hardware traffic sampling and QoS are the next major features that we consider implementing after the finalization of IPv6 L3HW. " Is this still on the high pri...

Hi, Per-VLAN L3HW counting is neither a bug nor an intended behavior. It is an unimplemented feature. Since traffic is routed via hardware, VLAN software counters are not updated (because the software does not see the traffic). We need to find a way to gather per-VLAN stats from the hardware to upda...

Hi, What's the problem with mirroring? It increases CPU usage since packets enter the CPU. However, unlike the software routing, the CPU does not throttle the performance in the case of mirroring. For instance, if the CPU can handle 10G while the actual traffic rate is 50G, all 50G will get hardware...

So with a WAN that requires VLAN that is in the bridge, once the NAT rule is in the switch chip how much additional latency are we talking about? Is the latency cost just for the first couple of packets for a new connection or is it on all packets? Are you essentially saying that L3HW regardless of...

@raimondsp, there is also I think the case where you need bridge filter rules, for instance my provider Orange in France require to set the COS to 6 for DHCP request, therefore I need to use a bridge port to set it on my rb5009 as the new-vlan-priority is not supported on this device as a switch ru...

WAN under the Bridge (or not) The debate about putting a WAN interface under the VLAN-filtered bridge or leaving it standalone is getting hot, so let's clarify the subject. Technically, RouterOS v7 allows putting a WAN interface under the VLAN-filtered bridge in any case (into a separate VLAN, of c...

Hi, what is the bridge interface's MTU (not L2MTU)? And what MTU values have bridge members? If any VLAN member (e.g., a bridge port with the respective pvid value) has a lower MTU (like 1500), the entire bridge resets MTU to the lowest value. Bridges operate on Layer 2, where packet fragmentation ...

I keep having the MTU reseting from 1700 to 1500 on a VLAN interface. 7.14 RC1. RB5009. The VLAN interface is on a VLAN aware Bridge (L2MTU = 1704 on the bridge interface). MTU = 1700 is accepted, then strangely revert to 1500 without any reported error. It's very annoying. I loose connectivity bec...

Hardware-forwarded packets are not visible by Packet Sniffer - that's correct. However, if you get 100% CPU load, then I suppose the CPU processes some packets, which are visible by Sniffer. Another option is to disable L2 HW offloading ("/interface/bridge/port set [find] hw=no") and send ...

I forgot to mention: if the traffic goes back and forth between CRS312 and Proxmox1, then it halves the bandwidth. It does not explain 100% CPU usage, though.

5) I am not sure if a standard linux bridge would forward the traffic back to CRS312 or not. Assuming it would then VLAN20 traffic would go back to CRS312 and would go back to Proxmox1 for the VM in VLAN20. If that hadn't been the case, then for CRS312 both test cases (within a single vlan and inte...

Sorry, I forgot to mention that you need to remove vlan10 from /interface/bridge/port too.

If that will not help, see if all bridge ports have H flag in /interface/bridge/port/print

Here is the problem: /interface bridge vlan add bridge=bridge tagged=bridge,vlan10,Proxmox1,Proxmox2,ccr2004,css326 vlan-ids=10 /interface vlan add interface=bridge name=vlan10 vlan-id=10 vlan10 is set on bridge (correct), but the bridge has vlan10 as a VLAN member (incorrect), resulting in a depend...

Hi, why have you disabled l3-hw-offloading on all switch ports? /interface ethernet switch port set 1 l3-hw-offloading=no set 2 l3-hw-offloading=no set 3 l3-hw-offloading=no set 4 l3-hw-offloading=no set 5 l3-hw-offloading=no set 6 l3-hw-offloading=no set 7 l3-hw-offloading=no set 8 l3-hw-offloading...

Hi, full hardware routing (a.k.a. L3 switching) does not work in this case since there is NAT in between. Disable l3-hw-offloading on the "internet" port(-s) to initially redirect the traffic to the CPU/Firewall, then offload FastTrack connections (which also support hardware NAT). Here is...

After updating to v7.14beta3, I noticed that my hAP ac3 is reverting vlan-mode settings in /interface/ethernet/switch/port to vlan-mode=disabled instead of the vlan-mode=fallback that was set before. This is happening on every reboot, even after setting to vlan-mode=fallback again. Hi and thanks fo...

Hello, i assign the ip on the bridge but no change. I just noticed that when i do a iperf between my computer (connected in 2.5G on the crs 309) and the crs309, i have 2.5Gbit in upload but only 250Mbit in download. if i do an internet speed test, i have also 250Mbit upload and 600Mbit in upload (6...

Hi,

Most likely, the traffic is processed by the CPU instead of the hardware, and that is the reason for slow networking. Post your interface and ip config, please, so we can analyze it.

/interface export
/ip export

Hi there, and sorry for the late reply, Unfortunately, the current switch chips do not support fq_codel or CAKE. However, RouterOS supports those AQM models on the software level. If the bandwidth is not too high, consider redirecting the traffic to the CPU for FQ-CoDel or CAKE usage. Most Marvell s...

Hi there,

We have identified and already fixed the issue where "suppress-hw-offload" didn't work for IPv6. The fix will be available in the next version.

Thanks for the feedback!

The multithreaded address lookup algorithm has zero overhead for accessing the ARP table, sacrificing entry insertion/deletion performance. That makes perfect sense: the router adds an ARP entry once, then may access it a million times. The garbage collector locks the entire ARP table, completely ha...

Unfortunately, I cannot give you any estimates at this moment.

The current main focus is on the core L3HW stabilization and QoS Hardware Offloading (the same team is working on both projects). After that, we will reevaluate the priorities.

Thanks. Do i need to enable on the switch ? /interface/ethernet/switch/set l3-hw-offloading=yes Does my current switch rules will work or do i need to rewrite them ? thanks Hi, Enabling l3-hw-offloading on the switch level globally turns on L3HW (i.e., starts l3hw driver). Other l3hw configuration ...

Changed the title of the topic. Please make informative titles rather than clickbait.

Technically, it is impossible to offload a route with gateway=interface (e.g., a connected route dynamically created on IP address assignment). RouterOS offloads hosts within the route instead. Hence, the route itself is redirected to the CPU (adds to ipv4-routes-cpu counter), and connected IP hosts...

For inter-VLAN routing, L3HW requires a hardware bridge for VLAN tagging. Please check out the following documentation topics:

naxus answer it the correct one. If looking at the past, you can see here how many betas has been released to the public (posted here on the forum) before RC was released: ver #beta 7.9 1 7.8 2 7.7 5 7.6 5 7.5 4 7.4 3 7.3 4 7.2 0 (no beta found) 7.1 5 7.0 3 To be precise, the above are numbers of p...

First of all thanks for the documentation @raimondsp . Upon checking it I have come across a thing that would be great if clarified: in Port settings ... By default, ports are untrusted and receive the lowest (0, best-effort) priority, where priority fields are cleared from the egress packets. Is t...

don't know if this could be supported but it would be great if we could prioritize traffic based on TCP/UDP port. We have plans to implement QoS profile assignment via Switch ACL rules, where you can match almost any L2/L3/L4 fields. The command will be something like this: # NOT IMPLEMENTED YET! I...

Using hardware QoS for bandwidth limitation is a natural next step for the project, so the feature will likely be implemented in the future. However, the current goals must be met first. The main goal of QoS HW is to provide lossless audio/video switching and (together with L3HW) routing at near-to-...

(this time, use something randomly generated). Hopefully you are not storing passwords in the clear. Not that is would matter to me, I use a password manager, and set long random passwords, different for each site. What is odd is your statement "this time, use something randomly generated"...

Why do you need to use RouterOS v6? Can you use RouterOS v7 with l3hw disabled (l3-hw-offloading=no) instead?

Hi,

We have fixed multiple l3hw-related issues in RouterOS v7.10. I suggest giving it a try (in a lab - please do not use betas in production!)

QoS-HW is compatible with L3HW. You can use both features together. Every supported device has 8 TX queues per port , and users will be able to assign QoS profiles to TX queues: either grant a QoS profile exclusive access to a queue or share a queue (or group of queues) between multiple profiles. T...

*) qos-hw - added QoS marking support for 98DXxxxx switches (CLI only);
could someone expand on this?

viewtopic.php?t=196068

Greetings, fellow community members! We are glad to announce the beginning of a new project - Quality of Service Hardware Offloading (QoS-HW) , introduced in RouterOS v7.10 . The goal of the project is to perform QoS packet marking (VLAN PCP, IP DSCP, and in the future - MPLS EXP), traffic shaping, ...

Hello support, I kindly ask for information. I'm trying to make a ccr2216 configured as a mpls P router work in hardware offload. Unfortunately, even with 7.9 I see that the traffic is only managed by the cpu. Do you have a roadmap for solving this problem? if yes for when? Hi, MPLS HW Offloading i...

Hi, and sorry for the late reply! We have made significant improvements to L3HW in RouterOS v7.10 . Give it a try once it gets released. Regarding the configuration of your CCR2216, VLAN interfaces must be set on the bridge; otherwise, L3HW cannot perform inter-VLAN hardware routing. Bypassing the b...

Jesus, after a ridiculous amount of testing I figured out the problem. If you have IP-Firewall set as active and then set either of the two options like "Use IP Firewall For VLAN" or "Use IP Firewall For PPPoE" set, then it doesn't work. And I was told by a Mikrotik member to di...

Just to make sure that I've understood this thread correctly - I have two RB5009s with VLANs in VRRP configuration. I've enabled pre-emptive mode for all the VRRP interfaces, set one of the VLAN VRRPs as the Group Master, and then enabled Sync.Connection Tracking on the Group Master interface on bo...

You need to set sync-connection-tracking=yes only on one VRRP interface (on both ends). If multiple VRRP interfaces are grouped together, enable sync on the group-master.

So only sync-connection-tracking=yes on group-master. Other VRRP in the group dont need to

exactly

Check group-master description in MikroTik VRRP documentation

Hi.
should I set the synchronization on each VRRP interface or is it enough on one?

Hi,
You need to set sync-connection-tracking=yes only on one VRRP interface (on both ends). If multiple VRRP interfaces are grouped together, enable sync on the group-master.

Hey, I tried that just now and no difference.

The rest of your posted config looks fine, so I'm unsure why it doesn't work on your side. Please create a support ticket, and we will try to reproduce your issue on our end.

Hello everybody! I hope that I can post my throuble here, if not, sorry... I'm a little bit confused with l3hw, i think the first thing that i need to know if it's possible: Where I work there was just a ccr1016, routing 400 ip cam and nating pppoe, my boss bought a crs326, to all cams route via l3...

Ok I don't think we are on the same page. Posting my export so you can get a better idea of whats going on, for brevity I have removed all other interfaces to avoid confusion. My ultimate goal is to get traffic offloaded between VLANs regardless if one VLAN is carrying WAN traffic it is still consi...

You cannot use Layer 3 Hardware Offloading for Layer 2 operations (bridge forwarding).

Currently, RouterOS supports only Active-Backup setups. Active-Active is not supported.

Has it been implemented since?

Not yet

It depends on whether you need the Firewall for inter-VLAN routing or not. For full hardware Inter-VLAN routing, leave l3hw enabled for LAN ports. All packets between VLANs will get processed by the hardware in this case. For firewall-compatible Inter-VLAN routing, disable l3hw for LAN ports too. In...

I read the thread, but still have some doubts. Let's say I have a single CCR2216-1G-12XS-2XQ unit, whereby I religiously follow the proper bridge configuration to ensure hardware offloading etc. And there is no connection_tracking/NAT. I'm assuming BGP affinity for input/output is set to “alone” pe...

So, practically, we now can define the VRRP interface used for connection-syncing as a VRRP Group master interface, assign rest VRRP interfaces as slaves. So then in the event of a reboot of the primary router, the backup will first sync back the connections to the primary, then the primary will be...

I too am wondering whether this would be implemented in ROS 7.x. A delay/hold timer is definitely needed in order not to drop connections when the Master router comes back. Please let us know if this is something in development :) Actually, the feature has already been implemented and released in R...

I will need to test this tomorrow or when I have time. You are probably quite right. When I did a quick check I indeed noticed a load on one core of upto 80%. I did not think about it more as I did not realize that the CRS317 might do 1Gbs as I only looked at the 512 bytes result for 25 rules (423....

Try disabling Bridge Firewall and see if that helps:

/interface bridge settings
set use-ip-firewall=no use-ip-firewall-for-vlan=no

The config looks ok fine, but I'd like to see the full picture. Can you provide the output of the following commands please?

/interface export
/ip export

I have one question about L3 support: is L3 offloading for the firewall supposed to work between 2 VLANs?. I have my WAN connection coming in on a VLAN. I need to firewall (NAT) it to my internal router/firewall. All VLAN interface are defined on the bridge. The fasttrack rule has hw-offload=yes. W...

"RouterOS prevents HW connections from timing out" Wait, does RouterOS inject Keepalives in the HW-Offloaded connections? what exactly does "prevents HW connections from timing out on the software side" mean? I meant that the software timer gets updated to prevent connection tim...

Hey there,

Hardware traffic sampling and QoS are the next major features that we consider implementing after the finalization of IPv6 L3HW. Unfortunately, at the moment of writing, there is no good solution to your problem.

While both CCR2216 and CCR2116 have the same CPU, their switch chips are different: 98DX8525 and 98DX3255 , respectively. Both routers provide near-to-wire-speed hardware routing. However, CCR2216 has significantly more hardware memory and, therefore, can hw-offload more routes/hosts/connections/etc.

Marvell Prestera DX switch chips provide hardware traffic counters that RouterOS utilizes for connection tracking. That's how RouterOS detects idle/slow hardware connections and unloads them to free HW space for faster connections. The algorithm is quite complex; I don't want to go deep into the det...

For me l3hw offloading doesn't seem to work at all. I read the help page multiple times, but couldn't find anything I may have misconfigured. I got it working once for a few seconds, after making changes to the bridge configuration, then there was a H entry in the connection list. But a few seconds...

@raimondsp
Okay, support ticket opened: SUP-92398

We received the support ticket and investigating the issue. Thank you!

@raimondsp: all 3 Switches became completly unresponsive, no L2 and no L3.
I only got access back after resetting the config via reset-button.

That's strange. Please create a support ticket, so we can try to reproduce your issue.

Hmmm....stange behavior on my two CRS326-24G-2S+ Switches. Winbox told me L3HW-Offloading is activated in the switch menu, CLI said it wasn´t. After activating it via CLI both switches died after a few seconds and didn´t came back, even after a cold boot. I had to netinstall both.... I´ve taken a n...

*) l3hw - fixed "H" flag presence for accelerated connection tracking entries; does this work for anyone? I have l3hw offloading enabled on my CCR2116 and fasttrack enabled for all established,related connections but I've never seen a H flag in the connection list (ipv4/ipv6). Even when I...

Just did a test on my CRS309, it worked one way for me (traffic from one port to another offloaded, the other direction not). Then I tried to disable and re-enable ip6 l3hw, and it caused temporarily loss of connectivity to the router, once recovered the ipv6 was no longer working, even after disab...

Switch chips used in CRS2x16 provide VRF support. We will implement VRF HW offloading eventually, but it is not a trivial task. Besides, other features are pending, so task prioritization is also a case. For instance, we released L3HW IPv6 functionality last week. While you cannot offload the second...

This is very exciting... *) l3hw - added support for IPv6 route offloading (disabled by default); Can anyone provide more information on this? What switch chips are supported? Does this mean that IPv6 fast-track is being worked on? Robert L3HW IPv6 is now supported by all CRS3xx, CRS5xx, and CCR2x1...

VRRP issues between a setup with an x86 & CCR2004 master/slave preemptive setup on 7.5 (both devices) - both nodes showing master, packet sniffing confirms all traffic including broadcast is seen by both nodes. I wanted to start debugging this outside production on pair lab HEXs - both on 7.5, ...

*) l3hw - fixed HW offloaded NAT; Interesting, on my CCR2116 l3hw nat was working fine for me before 7.5. Post 7.5 traffic that is natted has high % drop rate so high not even a TLS session can establish. Disabling l3hw removes the packet loss behavior. I confirmed this on my 2115 as well. L3HW swi...

Hi,

You need to create the second VRRP interface on ether1, then group both VRRP interfaces together (see group-master description in VRRP Documentation).

Please, enlighten me... Why do you care about something non-important for routing purposes like virtualization platform (containers), when there are so many unfinished things crucial for RoS 7? Multiple developers are working on multiple projects. Moving a developer to a different project is not a ...

Hi, If I understand correctly, you want to choose a gateway based on the source IP address, right? Firewall Mangle rules do not work with l3hw since hardware-routed packets do not enter the CPU, and therefore, the Firewall. The best solution would be creating a separate VRF table, but, unfortunately...

There are two lists on the official change log : What's new in 7.5beta8 (2022-Aug-09 12:36) Other changes since v7.4.1 Usually, users install a beta version for faster access to features and fixes they are interested in. Otherwise, there is no point in using beta - just play safe and install the sta...

same here. indeed weird that its fixed in stable channel while not in testing. Now it gets even more weird, the fix is not listed in the 7.5beta8 change list either... I would expect it to be at least in the next beta when it is fixed in stable already. (and of course I am disappointed once more w....

Hi, Your configuration looks fine on the first look, but it is hard to tell due to its size - maybe I have missed something. Here are some remarks: Check the Route Configuration for controlling which routes to offload. Do NOT enable use-ip-firewall or use-ip-firewall-for-vlan in bridge settings unle...

Hi,

Please type the following commands in the terminal and show us the output:

/interface export
/ip export

I have the same issue. I'm using 7.3.1 (latest stable as of now) and I had the management port with no link after enabled the HW offload. My bridge has 8 VLANs as well. I've disabled the HW offload, rebooted the router, but still, no link on the management port (eth1). I'm using a CCR2216-1G-12XS-2...

CRS3xx series support only ONE hardware bridge. Others get software-forwarded and, therefore, software-routed. So the rule of thumb is to have only one bridge, segregating networks with VLANs (vlan-filtering=yes). If you need Inter-VLAN routing, add the bridge interface itself to the tagged members ...

RB760 iGS

switch -> L3 HW Offload

it is some mistake?
or 750gr3/760igs will be "special" in small device category ?

That looks like a visual issue in WinBox. RB760 iGS does not support L3 HW Offloading.

I upgraded my CRS309 with 7.3 hoping, blindly, that this change here l3hw-offload on main table only Would fix the issue where when its enabled it breaks VRF completely. This did not fix the issue and it still remains. FYI if anyone has similar problems. Adding some outputs to reflect my not unders...

@raimondsp: Do you have some more information about your roadmap here? I had checked the L3HW documentation a couple of months back and there was no note yet that MLAG and L3HW features cannot be used together. So I assumed it would work - just learned now that it actually doesn't. So is this somet...

A great article!

FYI IPv6 hw-offloading is not completed yet. Currently, those "H" flags at IPv6 routes mean that the routes are subjects for hw-offloading, but the offloading does not work yet.

If a switch port with l3-hw-offloading=no is a VLAN member, the entire VLAN becomes l3hw-disabled (i.e., software-routed) If a switch port with l3-hw-offloading=no is a member of a VLAN-unaware bridge (vlan-filtering=no), the entire bridge becomes l3hw-disabled. I wouldn't call that a hardware or s...

Unlike CRS309, CRS305 does NOT support FastTrack hw-offloading - L3HW Device Support.

Here is a L3HW question. Is it possible to see the switch chip memory utilization? I understand that each model has different restrictions when it comes to ACLs, Routes/prefixes, Nexthops, Fasttrack connections and NAT entries. It would be nice to monitor the memory utilization for these items. Oth...

Netinstall may also work on a PC with multiple interfaces, but it is tricky. Netinstall broadcasts the initial handshake message to 255.255.255.255, so the packets are usually sent to the interface where the default gateway is located. To use Netinstall on a different interface, you need to (tempor...

L3 Hardware Offloading working with below configuration. VLAN interface doesn't show any bandwidth. Now what to do for Offloading Fasttrack Connections? [admin@2216-1] > /interface/export # apr/27/2022 19:05:55 by RouterOS 7.2.1 # software id = 13BN-J2X6 # # model = CCR2216-1G-12XS-2XQ # serial num...

Hardware Inter-VLAN Routing requires a Hardware Bridge for VLAN tagging. More info here.

I am using v7.2.1. If works please help to configure it.

Show me your setup:

/interface export
/ip export

Offloading Fasttrack Connections works for vlans ? Testing CCR2216-1G-12XS-2XQ but Offloading Fasttrack Connections for vlans not working.

Which RouterOS version are you using? VLAN FastPath/FastTrack support has been introduced in RouterOS v7.2.

L3HW IPv6 is in development. VRF and MPLS HW offloading most likely will come next. Regarding VXLAN and tunneling in general, I'm not sure if all the switch chips in the CRS3xx lineup support those features. Further investigation is needed.

Also, your setup has an issue: the IP address of a VRRP interface must have a /32 prefix, not /24.

add address=10.1.0.1/32 comment="VRRP: LAN IP" interface=vrrp_lan network=10.1.0.0

Which RouterOS version are you using? RouterOS v7 supports grouping of VRRP interfaces, where all group members are either VRRP MASTER or BACKUP. In your case, VRRP group will prevent the situation where vrrp_wifi is master but vrrp_lan is backup on the same device.

Routing metrics were done before L3HW was implemented. So those on product pages show software routing stats. Regarding hardware routing, if the traffic can be offloaded, it gets near-to-wire-speed performance on any MikroTik device that supports L3HW. Roughly, hardware routing speed = wire-speed - ...

So we have one of these(CRS328-24P-4S+RM l3hw) switches at a remote site. It is fed by an MLAG bond from a pair of cores for redundancy. BGP is standard in our network and it is only announcing a single network at the moment. Default route is received from the cores along with more specific interna...

But the problem still persist: Control-C without selection EMPTY THE CLIPBOARD

In a terminal, CTRL+C is the interrupt command and should not be used as a copy (despite it is working in some cases).

As an alternative, you can use CTRL+INS / SHIFT+INS for copy/pasting in Winbox terminal.

L2 MAC table is a separate one. RB5009 can store up to 16k MAC entries.

CCR2116 and CCR2216 already have full L3HW support. CCR2004 cannot have L3HW because its Marvell 88E6191 switch chip physically does not have L3 capabilities. RB5009 uses Marvell 88E6393 with very limited L3HW options. Maybe one day, we will implement L3HW support for RB5009, but it is not worth it...

Im trying to enable L3HW routing on some 317s. They have 900 (all incl 0.0.0.0/0 from OSPF) routes and right now I tried only with switches having one uplink. When I use /interface/ethernet/switch set 0 l3-hw-offloading=yes /interface/ethernet/switch/port set [find] l3-hw-offloading=yes The traffic...

What is the problem that you are reporting with this specific 7.2rc6 and what does it have to do with 6.4x versions? It was just to show that in 7.x series MT has change from posting many beta version of the software to posting many RC version. Since we have switched the main focus to stability, it...

Routes received via BGP were not properly installed to switch chip on CRS328-4C-20S-4S+ with 7.2rc5. Symptoms: establish a BGP session, connectivity is normal enable L3-hw-offloading, connectivity still normal restart BGP session, connectivity is lost until L3-hw-offloading is disabled. Is there a ...

Hello,

Any plans to add IPv6 support for l3hw?

IPv6 L3HW is currently in development. However, it is not going to be included in v7.2.

We have found a few semi-severe issues on 98DX2xxx/98DX3xxx switch chips (CRS305, CRS318, CRS328, etc.) when L3HW is used together with dynamic routing protocols (OSPF, BGP). Most of the issues have already been fixed in v7.2rc5. The remaining ones will be fixed in the next version. Unfortunately, f...

Stacking multiple CCR2x16 for full BGP table offloading wouldn't be cost-efficient. On the other hand, CRS317 is reasonably cheap for its switch chip capabilities. For example, get four CRS317 and bridge them together. Create routing filters to accept dynamic routes only within the 0.0/2 range on th...

Unfortunately, L3HW IPv6 implementation has been delayed due to additional work on RouterOS v7.1 and v7.2 stabilization. Currently, the development is resumed and moving towards finalization. No ETA yet, though.

*) bridge - added fast-path and inter-VLAN routing FastTrack support when vlan-filtering is enabled; *) l3hw - added HW offloaded FastTrack support for inter-VLAN routing; *) l3hw - fixed HW offloaded NAT; Wait, if this is true and it's applies to all devices this is going to be a major improvement...

So i also unterstund the limitation table wrong. For example the CRS317 can hold up to 240k routes and can route packets in hardware for all routes that a stored in the routing table of the switch chip. There is no connection limit because there is no connection tracking? The limitation for fastrac...

The Switch Chip Features document had been written before L3HW implementation, so it does not contain L3-related tables. The latter are specified on L2HW Device Support page. Marvell switch chips use classified proprietary algorithms for routing, which we cannot reveal without violating NDA. What is...

@raimondsp thx for the great effort. Before trying to describe your answers in my own words could you tell me where the L3 information is stored in the switch chip? As that causes me difficulties to imaging how the processes on the Switch Chip are happening. According to the https://help.mikrotik.c...

We were unable to reproduce your issue. On our side, initial sync gets performed as intended.

Please create a support ticket, so we can reproduce exactly the same setup as yours.

There are two types of Hardware Routing (L3HW): Full Hardware Routing and Firewall-Compatible Hardware Routing . Full L3HW, in turn, differs between routing via an explicit nexthop gateway(-s) and routing to a connected L2 network (a.k.a. Connected Routes ). As a result, there are three different ca...

IP address on VRRP interface must have /32 netmask if address configured on VRRP is from the same subnet as on router's any other interface. In your case, it should be: /ip/address add address=10.1.160.1/32 interface=vrrp.voip-router Regarding the NETLINK socket error, did you force connection trac...

So I understand you tested with tagged ports. I'm scared untagged ports would prevent fasttrack to work for them... So, faced same issue, FastTrack does not work for untagged ports in a VLAN-filtered bridge. Works as soon as ports are tagged. SUP-73092 opened accordingly. We have reproduced the iss...

Don't you think it would be a good idea to work towards a 7.2 release that is feature complete relative to 6.49.2 and has most of the visible bugs fixed that were introduced during the v6 to v7 transition? Then the next step could be a 7.3 which adds all kinds of nice new features. But at least the...

Weird, checked it again and now l3-hw-offloading is no and I can't set it to yes neither No idea how that worked before, I fiddled around a lot.. However, with v7.2rc3 I have fasttrack forward rules with hw offload working on RB5009. Did not find time for propper testing yet, but a quick ipperf run...

Fast Path for vlan-filtered traffic has been introduced in RouterOS7.2rc2. Now FastTrack should work fine, including offloading.

If sync-connection-tracking is set and running, you should be able to see the connections on the backup router (no counters, though). Make sure you have the same Firewall rules set on both ends. Also, forcing connection tracking might help:

/ip/firewall/connection/tracking/set enabled=yes

I was always wondering where exactly fastpath and fasttrack touch each other? And why do I see fasttrack counters increase on my hAP ac2 running 6.49.1 with vlan-filtering enabled bridge ... I have single bridge with a few VLANs, one of VLANs carries PPPoE (used as WAN) and another VLAN is used as ...

FastPath / FastTrack / L2HW / L3HW Clarification

FastPath and FastTrack are two different things. Similar naming may sound confusing, and some people think both are synonyms, but that's not true. I will oversimplify the clarification to keep it short. You can find more info on help.mikrotik.com or other internet resources. FastPath is an ability ...

Unfortunately, FastTracking of VLAN-filtered bridged traffic is not finished yet, and I cannot tell the exact completion date or version number. The development is almost completed, though. However, a long testing phase is pending since this feature affects all MikroTik routers. The good news is tha...

If a host offload fails, then the traffic to that host (IP) goes via CPU, and, therefore, causes a performance drop. Are those hosts in the log real, or are you running a network test by sending packets to random destinations in the subnet? Hosts can be offloaded only after resolving ARP (IP-MAC). H...

Currently, a bridge with vlan-filtering=yes does not support FastTrack (both in v6, v7). The feature is in development.

"-14" is an internal error code, meaning that the L3HW driver is turning off or restarting. Were you rebooting the router or setting "l3-hw-offloading=no" when those log messages appeared?

Please read this.

There are plans on implementing MPLS hardware offloading at least for Marvell 98DX8xxx switch chip series (CRS317, CRS309, etc.) and CCR2116. However, please don't expect that soon - we have to finish IPv6 offloading first.

Hardware IPv6 routing on Marvell switch chips (CRS3xx, CCR2116) is in development.

IPv6 Fasttrack implementation is on our TODO list, but at the moment, I cannot tell when it will happen.

About the useless switch of the RB4011, here are the tests showing that the issue isn't solved. With bridge vlan filtering on: Screenshot 2021-12-02 133319.png With bridge vlan filtering off: Screenshot 2021-12-02 133101.png https://forum.mikrotik.com/viewtopic.php?f=1&t=177092#p878135 Long sto...

A feature can be offloaded to hardware only if the hardware (switch chip) supports the feature. Switch chips that provide a broad L3 feature set (routing, connection tracking, NAT) are not cheap. I wouldn't expect a three-digit-priced switch chip in a two-digit-priced router. Meanwhile, a brand new ...

CCR2116 supports L3 Hardware Offloading. In some cases, it can route packets at close to 10Gbps speed while keeping the CPU idle.

Please upgrade to RouterOS v7.1rc7. The OSPF authentication issue should be fixed there.

Meanwhile, the developers have been identified the SNMP increasing memory issue and are working on the fix.

Hi, and sorry for a late reply, I'm glad that the problem got resolved by upgrading to v7.1rc6. Rc4 had an issue with MTU offloading so that most likely was the case. Regarding: It is recommended to turn off L3HW offloading during L2 configuration. Configuring L2 while L3HW is enabled does not cause...

Hi,

An issue with OSPF authentication will be fixed in the next version.
Also, we were able to reproduce an increasing memory usage by SNMP, and developers are looking for a solution.

Thanks for the feedback!

Please don't use that version. It is in no way stable or suggested. It is just a number. If you care for an actually stable version, use 7.1rc6 +1 "Stable" means that v7.0.3 has passed QA/QC tests for Chateau. Unfortunately, it does not mean "bug-free". Since there is no reporte...

Hello there,

I'm glad that everything besides SNMP works fine on your end. Our support team should contact you shortly, asking for details of SNMP setup and use-cases. If they will forget, ping them via support email

Have a good day!

Hey there,

Unfortunately, the compatibility layer between MLAG and L3HW is not implemented yet and postponed after RouterOS 7.1 stable release (due to feature freeze). Until then, it is not recommended to use both features together.

Then I suppose that our ARP fixed won't help. Well, thanks for indirectly helping us to identify and solve those issues anyway. Now back to your problem. We compared side-by-side your provided configuration of DistributionSW4. Since we are using the same hardware and software but cannot reproduce th...

Then looks like a different problem

And how many ARP entries are in total?

b) just remove the backhaul's port from the bridge and set IP etc on the 'natural' port. This scheme won't allow for HW offload of any kind (neither L2 nor L3). L3 HW routing only works if ports in question are all logically handled by switch chip directly. And that is achieved by configuring ports...

Great News! Actually in our environment we have completely disabled IPv6 furthermore we continuously check the ARP cache and it seems pretty much static and empty. Or are you referring to something that is not normally visible via interface? You may check: /ip/arp/print count-only where !complete W...

Hi, we have identified another issue. Actually, it is not a memory leak - it is just an increased memory consumption due to unresolved ARP entries (or IPv6 neighbors). We will test the solution and send you a new beta soon.

Try disabling connection tracking and see if that prevents RAM consumption:

/ip/firewall/connection/tracking/set enabled=no

Requires router reboot after disabling conntrack.

Hey there,

We are sorry that the fix didn't fully address your issues, and we're continuing to investigate the possible problems.

Which is sw4 on the diagram above? I see there CoreSwitch1&2, and DistributionSwitch1,2,3.

After many attempts, we were finally able to reproduce the issue. By the looks of it, the memory leak is unrelated to L3HW/OSPF/ECMP but MIPS-specific (CRS326-24S+2Q+RM uses MIPS CPU). The case has received top priority and is under investigation to identify the root cause. Thanks for the feedback, ...

Hi Raimond and thanks for your feedback. we did as instructed and so far it seems ECMP is the main culprit, as disabling it in the lab resulted in major improvement. However exactly the same setup in production (not really production - no active users, but few L2 access switches connected) is still...

Thank you for the detailed feedback! We had put CRS326-24S+2Q+RM under a heavy stress testing load in our lab for the weekend. Unfortunately (or fortunately?), we were unable to reproduce your issue and didn't detect memory leaks (RAM usage kept stable during the entire session). However, we didn't ...

I'm almost certain that GRE tunnel offloading will be implemented. However, I wouldn't expect it soon. First, we need to stabilize RouterOS v7.1. And the next big feature is IPv6 hardware routing support. Only then can we evaluate GRE L3HW support and put it on the roadmap.

We are investigating the issue. Supout data does not show anything unusual. We will put CRS326-24S+2Q+RM under heavy load for the weekend to see if the issue is reproducible in our lab. What is the common route rate received via OSPF on your side (i.e., an average number of routes per second or minu...

Hi there,

Are you the one who created the SUP-64006 ticket?

I'm so sorry. The MAC src fix in the rc5 is for the DX2000/DX3000 switch chip series. However, CRS317 has a DX8216 chip. Currently, we are testing the fix for the DX8000 series. If you wish, we may send you a private firmware build with the fix included after the testing is done. So you won't have t...

The issue has been fixed. Now, L3HW uses bridge/vlan's MAC for the source address of routed packets.

The fix will be included in the upcoming 7.1rc5.

According to the Marvell Prestera documentation, the switch chip of CRS317-1G-16S+ supports GRE tunneling. Unfortunately, the feature is not implemented in RouterOS 7 yet, meaning that tunneled traffic gets processed by the CPU (slowly). I have created a ticket to investigate and evaluate L3HW offlo...

Hi @raimondsp; I just reported the issue as SUP-63543. You can find the configuration and a supout.rif there in case you need it to reproduce the issue.

Got it, thanks!

Hi,

If a port is a bridge member, it should set the bridge's MAC as the source address in the routed packets.

We will investigate the issue.

Thanks for the feedback!

Is it possible to sync connection tracking state in an active/active setup? I like to peer via BGP with my upstream provider, so i like to have two active bgp sessions and so on to route the traffic from WAN to LAN where the traffic arrives. So there is a possibility of asymmetric routing. Because ...

L2 traffic should get on fastpath. Moreover, in most cases, L2 traffic gets offloaded to the hardware (switch chip), which is capable of forwarding L2 traffic at wire-speed. Inter-VLAN routing (L3) is the one that is not implemented on the fastpath yet. Please, export the interface and IP configurat...

... Currently, RouterOS cannot perform VLAN filtering on the L3 fast path. ... Does this mean that Router OS never, not at any device, performs any fasttracking, if all IPs of a device are on an `/interface/vlan` where the `interface` of the vlan is a ` /interface/bridge`? e.g.: /interface bridge a...

FastTrack does not support Inter-VLAN routing on a bridge with VLAN filtering enabled. It never did. It is not a bug but an unimplemented feature, which is currently in development.

Edit: Updated to mention L3.

There is now further information in the latest newsletter about the performance of the CCR1009 model in RouterOS v7, which we can compare to RouterOS v6: RouterOS v6 - 25 ip filter rules, 512 bytes, Mbps: 3251.8 Mbps RouterOS v7 - 25 ip filter rules, 512 bytes, Mbps: 2618 Mbps This is about a 20% d...

Good morning! Nice to hear that hardware routing works and ipfix got fixed! Connection tracking does not work with hardware routing since the packets never enter the CPU, and the switch chip does not provide the routing performance data. Your random VLAN inaccessibility sounds weird. Do VLAN members...

RoutersOS v6 uses routing cache while ROS v7 doesn't. While ROS6 performs routing faster in the happy-path scenario (cache hit), its performance is significantly slower in the rest of the cases (cache miss). Usually, synthetic speed tests utilize a small number of routes and, therefore, always hit t...

The issue has been fixed and will be available in the next rc.

Hi there, We reproduced the issue and are working on the fix. Thank you for the feedback! P.S. Worth mentioning that MTU change works fine on forwarded traffic. Only the INPUT chain is affected by the bug. For example, in "dev1 <=> CRS312 <=> dev2" setup, you can send jumbo packets from de...

I have found an issue in your setup: /interface bridge vlan add bridge=bridge1 tagged=bridge1 untagged=sfp-sfpplus1,sfp-sfpplus14,vlan1 vlan-ids=1 /interface vlan add interface=bridge1 name=vlan1 vlan-id=1 Circular reference: vlan1 marked as a tagged interface of bridge1, but bridge1 is the interfac...

I was unable to reproduce your issue. I did a similar setup with 802.3ad bonding and VLAN bridge with both tagged and untagged interfaces. And L3HW offloading clearly worked as intended. Here is my setup: /interface bridge add name=bridge vlan-filtering=yes /interface vlan add interface=bridge name=...

Hey again, and thanks for the feedback!

I reported the import/export issue and waiting for the fix.
Meanwhile, we are trying to reproduce your case. I'll keep you in touch.

INPUT and OUTPUT chains work fine with l3hw offloading. The reason why the FORWARD chain does not work in L3HW Full Routing mode (l3-hw-offloading=yes on both the switch and ports) is that the forwarded packets never enter the CPU, and, therefore, do not trigger the Firewall. I'm not sure about L2TP...

Running Doom on CCR1072's LCD is on the roadmap /s

Just to clearify all the doubts. CRS317 with routeros 7 latest sfpplus1 + 2 is a 802.3ad BOND (wan) sfplus 15 +16 is a 802.3ad BOND (to backbone, towards users) sfpplus 10 is a remote network sfpplus 11 is a remote network on all ports we have a /29 and we do plain BGP v4+v6, no filters on FORWARDE...

300-400 mbit/s is the maximum routing speed that CRS317's CPU is capable of. That means the routing is performed by the CPU, not the hardware.
Please provide the output of the following commands:

/interface export
/ip export

Must be mentioned: Do not use the same passwords from 2018 ever again! Even on different routers. The hackers who obtained system user database files via CVE-2018-14847 may apply brute force to try every stolen password on every MikroTik (and maybe even non-MikroTik) device. For example, you had the...

Thank you for the feedback! We have reproduced your issue in the lab. It turns out that this is not a bug but rather an unimplemented (yet) feature. Currently, RouterOS cannot perform VLAN filtering on the L3 fast path. Therefore, all packets that are routed through a bridge with vlan-filtering=yes ...

@aglabs Can you do us a favor and try moving the ports outside of the bridge? /interface/bridge/port disable [find where interface=sfp-sfpplus7 or interface=sfp-sfpplus8] /interface/vlan/ disable vlan11,vlan16 /ip/address/set [find interface=vlan11] interface=sfp-sfpplus7 /ip/address/set [find inter...

You can report the issue to our support. Also, check if RouterOS has generated autosupout.rif (via /file/print).

Hi,

The entire team is focused 100% on stabilizing v7.1. Let's get back to this topic after v7.1 stable release.

Thank you for the answers! We will try to reproduce your issue. When changing hardware routing settings, the existing connections might be unaffected. For example, if you have an active FastTrack connection while enabling l3-hw-offloading=yes on the respective ports, the traffic may continue through...

I've got it working after I enabled 'use ip firewall' in the bridge settings, and now I'm using the raw ip firewall table. However, I have to wonder which is the better way of getting it to work. The problem with use-ip-firewall is that it affects only the traffic that goes through the CPU and does...

A quote from the wiki:

It is recommended to turn off L3HW offloading during L2 configuration.

L3 HW Offloading: Layer 2 Dependency

Hi there,

What do you mean by "port removal"? Unplugging cable?

Please post your config and a step-by-step guide on how to reproduce the issue.

When you start replacing console hotkeys that are plain ASCII characters with "function keys" it may be time to finally replace the Ctrl-V hotkey with something else! People think (and rightly so!) that Ctrl-V means "paste" and they are quite surprised when they see the effect i...

"?" has been replaced by F1 Hi Raimonds, Why was this changed? Hi, There are two reasons for this change: It confused new users who wanted to type actually "?" (required "\?") Copy-pasting data containing "?" produced side effects. Imagine you want to post a ...

And typing “?” on CLI just gives red warning, instead of list of possible commands

"?" has been replaced by F1

All L3HW fixes are present in rc1+. At the moment of writing, there are no known issues regarding L3HW in terms of inter-VLAN routing or FastTrack offloading. We will try to reproduce your case. Please verify if our assumptions are correct: You want to set up firewall-controlled inter-VLAN routing b...

The firewall affects only the packets processed by the CPU. If the forwarding/routing is performed by the hardware, packets do not enter the CPU at all, and therefore do not trigger the firewall (even if you have FORWARD rules). In addition, you can fine-tune L3 HW offloading by disabling HW routing...

Do you mean something like this?

Is this Unreal Engine's Blueprint? I imagine user reaction if we'd announce "WinBox3D, min. requirements. GeForce RTX3080"

I see you are testing IPv6 on dead beef networks as I do. Nice!

Which exactly CRS326 model do you have? If it is CRS326-24G-2S+, then its 98DX3236 switch chip does not support Fasttrack and NAT offloading.
L3HW Device Support

CRS328 does not support FastTrack offloading ( L3HWDeviceSupport ). As for CRS317, I believe that you have encountered an issue that prevents FastTrack offloading. The issue has been already fixed and waiting for the 7.1beta7 release. Here is a quote from the upcoming changelog: - L3HW: Fixed an iss...

Not directly related to L3 routing (hence off topic in this thread), but anyway: I always thought that setting port tagged member of VLAN (/in/br/vlan entry for VLAN ID) and setting PVID on bridge port (/in/br/port) makes possible mismatch: if frame-types is not set, then untagged frames are allowe...

By the looks of it, L2 segregation for the mentioned above cases is an illusion of safety. I understand the reason why you would want to put IoT devices under a separate VLAN. For instance, you bought a no-name smart light bulb on eBay and you don't want it to access your NAS and upload its content ...

IMO there's an error in the "VLAN configuration example": /interface/bridge/port add bridge=bridge interface=ether2 pvid=20 /interface/bridge/vlan add bridge=bridge tagged=bridge,ether2 vlan-ids=20 Doesn't first line of this example set ether2 as access port for VID 20 and should thus be ...

How is there a 7.1beta7 listed if it hasn't been released yet, or are you just keeping it as up-to-date as possible?

Oops, we accidentally posted implemented but yet unreleased features. Well, I guess that now you have an official sneak peek of the upcoming changes.

Good day, ladies and gentlemen!

We have updated Layer 3 Hardware Offloading User Manual to reflect the latest features, changes, and configuration samples.

Feel free to use and do not hesitate to provide feedback!

I can put it here too. This is 7.0.3 for Chateau only: https://box.mikrotik.com/f/7e3cad5779804d0b878d/?dl=1

We should put a big disclaimer next to it:
DO NOT INSTALL v7.0.3 ON ANYTHING BUT CHATEAU!
But I doubt it will solve people's inability to read.

Search found 270 matches