V7.20beta [testing] is released!

Sometime ago I suggested a more “event-based” triggering feature for RouterOS scripting.

Suggestion: Hooks to Scripts on /routing/filter/rule actions

It colud cover all the already existing script events, like:

  • /ppp/profile/ “on-up” and “on-down”
  • /ip/dhcp-server/ “lease-script”
  • /ipv6/dhcp-server/ “binding-script”
  • /tool/netwatch/ “up-script” and “down-script”

And also would allow almos everything like those suggestions that were already done at the thread:

Also would avoid the need of some stupid pooling log that are triggered by time-scheduled script.
An example? Do something whne some user logs-in via any protocol.
I use that for some “fail2ban” tool.

Based on your description of what you need in container scripting… To me seems like a hook-based / event-based scripts trigger (in this case related to container) wolud solve your problem.

I agree with “events everywhere” – but more pointing out that unlike most of RouterOS, the /container operatations are already asynchronous. i.e. If you “interface”, the next script line can use it. But in container you issue a “start”, which many take time, but the CLI returns immediately… so if you want to next say use the new 7.20 /container/shell cmd= “after container started”, you a :for/etc or :delay. And with the new “shared devices”, the likelyhood of failure you’d want to handle goes up – but stuck with polling.

But even some alternative to “full” event support, be a wait=yes that at least make the CLI/script wait until any state change, that way you can avoid the :for/:delay when trying to script /container.

‡ which could be just a listen along with print on command in CLI, which API already has (and presumable webfig/winbox), but not exposed to scripting engine.

I just hope this will be fixed until “stable” release. No any response from support, that I wrote 1 month ago.

I have to agree 100%. Management of network gear should be in dedicated management VRFs. It’s just a good security practice and industry standard. The lack of HW Offload support with VRFs is incredibly disappointing especially when MikroTik has acknowledged that its available in hardware they just need to put in the work on their software. I noticed that with VRFs enabled I cannot even use my gear as a proper P router let alone a PE. At this point of relegated my MikroTik routers to simple switches. Lots of great features and capability which mean nothing if I can’t properly secure the device.

I’m running into a similar issue on beta4. My setup looks like this:

[CRS309: Tagged VLAN 10] <— sfp-sfpplus1> [CRS354 (Leaf): VNI 100010] <qsfpplus1-1 — qsfpplus16-1> [CRS320 (Spine): VNI 100010]

I’ve got a vlan interface on the CRS320 bridge with IP 192.168.10.38/24, and the rest of the network resides behind the tagged VLAN on the CRS309. Communication is only able to pass over the VXLAN interfaces if either:

  1. VXLAN hardware offloading is disabled on CRS354, OR
  2. Hardware offloading is disabled for the sfp-sfpplus1 bridge port on the CRS354, OR
  3. Both are disabled

Enabling or disabling hardware offloading on the CRS320 makes no difference. I’ve also tried changing the link between the CRS309 and and 354 to untagged vlans, which did not work either. The 309 is running SwitchOs, so I’ve omitted the config for that. Is hardware offloading both of these functions supported? I don’t see anything in the docs to indicate that it wouldn’t be…

Relevant configs:

# model = CRS520-4XS-16X
/interface bridge
add frame-types=admit-only-vlan-tagged name=bridge vlan-filtering=yes

/interface bridge vlan
add bridge=bridge tagged=bridge vlan-ids=10

/interface vlan
add interface=bridge name=MGT vlan-id=10

/interface vxlan
add bridge=bridge bridge-pvid=10 dont-fragment=disabled learning=no local-address=10.8.1.1 name=vxlan100010 vni=100010

/ip address
add address=192.168.10.38/24 interface=MGT network=192.168.10.0

# model = CRS354-48P-4S+2Q+
/interface bridge
add name=bridge vlan-filtering=yes

/interface bridge port
add bridge=bridge frame-types=admit-only-vlan-tagged hw=no interface=sfp-sfpplus1

/interface bridge vlan
add bridge=bridge tagged=sfp-sfpplus1 untagged=vxlan100010 vlan-ids=10

/interface vxlan
add bridge=bridge bridge-pvid=10 dont-fragment=disabled hw=no learning=no local-address=10.8.2.1 mac-address=AE:F2:FC:13:21:0F name=vxlan100010 vni=100010

Remote VTEPs and MACs are learned using EVPN.

Did you test VXLAN fabric in real Spine&Leaf topology? Where Spine only interconnect Leaves and VTEPs only configured on Leaves?


I’m only curious if it works.

This topology works in my testing but not in the real hardware for now, as soon as v7.20 become stable i’m going to try this in real world with real hardware, this is working fine in GNS3

1 Like

So it’s expected that VXLAN HW offloading doesn’t work properly yet with real hardware? If so, that’s disappointing, since the docs (seem to, by my reading, correct me if I’m wrong) indicate that it’s been supported on the hardware I’m trying to use for two major versions. I haven’t noticed anything terribly wrong with the BGP/EVPN side of things in the beta on real hardware, other than lack of support for some of the L3 features which I was aware of before starting, and sometimes having to disable and re-enable a “/routing/bgp/evpn” entry to get it to pick up VTEPs after a reboot. If someone could clarify exactly what is supported currently for VXLAN hardware offload, that would be very helpful, hopefully I’m just configuring things wrong.

If you have issues with a beta feature, then you should open a support ticket with the needed informations to reproduce issues. Then engineers might fix them.

In beta4 this has ben fixed again on my limited testing with GNS3, we are early on the beta stage things might change quite fast or slow who knows all of us don’t have visibility on MT is doing, the hardware offload not to mentioned (VRF + hardware offload) sure these things will follow because there’ s no point of buying/using all this high performance gear without this important features.

Imho we should wait at least 3 to 4 stable version to fully realized all of this, just my 0.002$

high CPU is related to container, but contaneir doesn’t change from april/may

Just out of curiosity… Is the switch-marvel package available in the installation packages on these CRS with ARM? Have you tried installing it to see what happens?

The swtich-marvell package is available on the CRS520 (but not installed), but not on the CRS354. Is this package required for hardware offload? I can give installing it on the CRS520 a try, but as far as I can tell the issues is on the CRS354.

EDIT:

I’m unable to install switch-marvell on the CRS520. After enabling, applying, and rebooting, it remains stuck in the disabled state, with no messages in the logs other than “enabled switch-marvell-7.20beta4”.

This is still stuck in MikroTik’s obscurity.
I believe this package may have remarkably close relationships with Marvel’s hardware off-load chips…

And I also guess that, at least for now, this feature is tied to the arm64 architecture.

But this is just a guessing game on my part.
It’s what’s left when there’s no clarity in the relationship.

But nothing has been published about it.
Nothing other than the very little explanatory text below:

And there are still some fanboys who said words of displeasure to me because I asked for more details about this.

Yes it would be helpful when they add a list of chip types supported by these packages (instead of only a manufacturer name) so people can lookup the chip type in their device and not be tempted to install a package that is not for them.

I’ve tried removing all BGP/EVPN configuration, setting it up as a simple VXLAN tunnel with manually defined VTEPs, and downgraded both the software and routerBOARD firmware on the CRS354 to 7.19.2. The issue with hardware offloading persists. At least it’s not a beta issue. Given that the CRS354 is listed as supporting VXLAN hardware offloading (and my configuration is pretty much identical to the example), I’ll open up a support ticket, unless anyone here has any other advice.

1 Like

https://dev.to/ivan_kuten/linux-and-cpss-assembly-on-marvell-board-with-prestera-dx-switch-1agg

They mention 98DX3257, but maybe there are other chips supported by CPSS.

Edit: based on what contains this package, it supports AlleyCat5P and AC5X chips.

Solution found, in case someone else runs into this:

  1. Disable hardware offload on physical bridge port, enable on vxlan interface, enable “dont fragment“ on the vxlan interface
  2. Send traffic down the interface
  3. While traffic is flowing, enable hardware offload on bridge port
  4. VXLAN will be properly offloaded.

Perhaps a reboot alone would be enough to start offloading the VXLAN, without going through the hassle of ensuring traffic is flowing while the changes are made. This seems to persist after reboot, so maybe that would have been enough in the first place. Enabling “don’t fragment“ without either following the above steps or rebooting didn’t get it to offload, which is why I previously dismissed it as a solution.

Could you please try enabling back the evpn + vxlan and see how far you can go?

That was with EVPN/VXLAN re-enabled. Aside from occasionally having to disable and re-enable “/routing/bgp/evpn” instances to get the routes to propagate properly after reboots, it looks like everything is working fine, at least with only one spine (I have 5 leaves, only one of which is a mikrotik device, yay interoperability!). The line in the VXLAN hardware offloading docs about not working over ECMP gives me concern that it won’t work with two spines, but I haven’t tried that yet. I did purchase two spines, but I’ve left one in the box for now to make returns easier if necessary. I don’t plan to unbox that one until I have the rest of the network working on the current single-spine setup, so it might be a few weeks. Currently I’ve only migrated one VLAN for testing.