Hardware VXLAN Broken recently?

RouterOS General
1

I have opened a support ticket (SUP-196645), but something really strange has happened.

After doing a proof-of-concept between a couple of CRS317's, I deployed a few and upgraded them all to 7.19.4, and suddenly everything stopped working.

After doing a bunch of packet captures it appears that the hardware vxlan is unable to get data from the bridge. Enabling the vxlan sees low level traffic (SPAN, MDP, that sort of stuff), but no actual traffic from the vlan it's meant to be linked to.

Does anyone actually have this working, or was I just massively lucky to get it working at the start?

The only way it seems to work NOW is to turn off hardware accel on the port that brings the vlan into the bridge, AND hardware accel on the vxlan itself.

Oddly, the 'l3-hw-offloading=yes' setting doesn't really seem to do anything. Even with it off, enabling HW Accel on the vxlan still sends small amounts of (real) traffic.

2

I'm now starting to think that no-one is actually using l3-hardware because I've tried this on a completely new 317 and it still doesn't work. I copied and pasted the example config from VXLAN - RouterOS - MikroTik Documentation and nothing. sadface

3

For reference to anyone else looking at this thread:

When L3 hardware acceleration is enabled, packets do not show in the packet sniffer. (Since they are not processed by the CPU.

4

I was sniffing on the transit links and at the other end. We actually ended up giving up on using Mikrotik for this, and switched to CIsco, sadly.

5

Enabling Hardware Offload for VXLAN takes just two clicks, as long as you have the proper device. To be safe, it's a good idea to export your config here

6

I did get a response in the ticket (4 months later) saying ‘we can’t duplicate this problem’, but there’s been nothing in the release notes about it being fixed.

7

I’ve found HW VXLAN rather finicky to get working but I have it working fine in production once I figured out its quirks.

One example that comes to mind: Using dynamic bridge attachment + VLAN ID instead of manually adding to a bridge

8

xrobau - just wondering how does it look on your end.

9

Would you share an example of a working setup - i only have it partly working. Between directly connected switches it just works, but if there is a routed hop between mac addresses are learned correct, arp resolved correct but not able to ping the other side. I would really apriciate a hint

10

After all this time, I managed to get it working.

I've documented it here: L3HW VXLAN finally works on the CRS317!