CRS v7.18.1 VXLAN hw-offload

RouterOS General
1

Hello,

in 7.18 changelog I’ve noticed that

*) l3hw - added initial HW offloading for VXLAN on compatible switches;

I’m trying to setup a 3 node proxmox cluster using VXLAN, where nodes talk to each other using VXLAN + to the switch, which “unfolds” VNIs into VLANs for the rest of the network.
However, I’m failing to get any hw-offloading running (cpu tops out at 100% ~1Gbps traffic on VXLAN). in the docs I’ve spotted that:

Underlay (routing encapsulated VXLAN packets):
[…]
2. VTEPs are not supported over bond, VLAN interfaces,

It’s a bit confusing, as all the other bullet points on this list mention only 1 thing, and this one mentions 2 with a comma (is it an OR or an AND).
Does that mean, that my CRS310-8G+2S+IN can’t communicate over VLAN interfaces to the VTEPs?
I tried to add address directly on bridge and pvid 1 (to try to avoid vlans) but to no avail.

current config:

/interface bridge
add name=bridge port-cost-mode=short vlan-filtering=yes
/interface vxlan
add bridge=bridge bridge-pvid=4 dont-fragment=disabled local-address=192.168.0.5 mac-address=... name=vx-dmz vni=40
/interface list
add name=pve-mgmt
add name=pve-vxlan
/interface bridge port
add bridge=bridge edge=yes interface=pve-mgmt pvid=10
add bridge=bridge ingress-filtering=no interface=pve-vxlan pvid=100
/interface bridge vlan
add bridge=bridge comment=mgmt untagged=pve-mgmt vlan-ids=10
add bridge=bridge comment=vxlan-underlay tagged=bridge untagged=pve-vxlan vlan-ids=100
/interface ethernet switch
set 0 l3-hw-offloading=yes
/interface vxlan vteps
add interface=vx-dmz remote-ip=192.168.0.11
add interface=vx-dmz remote-ip=192.168.0.12
add interface=vx-dmz remote-ip=192.168.0.13
/ip address
add address=192.168.0.5/24 interface=vxlan-underlay network=192.168.0.0

Current documentation has only 1 example with dedicated direct port to the VTEP (which is kinda lame).
Is there a way to bridge VTEPs as well as communicate with them on the same device?

2

keep in mind that CRS310-8G+2S+IN do not has Support for hardware-offloaded VXLAN

https://help.mikrotik.com/docs/spaces/ROS/pages/62390319/L3+Hardware+Offloading#L3HardwareOffloading-CRS3xx%3ASwitchDX3000andDX2000Series

3

Oh, so that’s what I missed. Thanks for the help.

That does explain why I’m having performance issue (reading is hard), but doesn’t really fully answer the question.
If I take existing config and pass the vlan 100 to a CRS309-1G-8S+ (which does support hw-offload VLAN), would that work?
Can i pass the vlan100(vxlan(packet)), or do I need to pass raw vxlan(packet)?

4

Underlay (routing encapsulated VXLAN packets):
[…]
2. VTEPs are not supported over bond, VLAN interfaces,

As I understand it, these restriction only apply for hw-offloaded VXLAN, which is not supported on CRS310.
Passing vlan100(vxlan(packet)) on CRS310 should work.

And as you have experienced, doing VXLAN on CPU severely limits bandwidth.

5

I’m rephrasing the question again with regard to CRS309-1G-8S+, which does support L3HW VXLAN.

Can I have (multiple) VTEPs on bridge (preferably on tagged vlan) AND communicate with them while having L3HW VXLAN working, or do I have to provide CRS309 with dedicated port that carries raw VXLAN packets? If the question is still unclear - can communication with a VTEP be performed over tagged VLAN?

6

I see. The described limitation seems to be about VTEPs directly on top of a VLAN interface.
Your question is about forwarding VTEP packets to tagged ports on the same bridge.
Good question and I don’t know the answer. Depending on the switch ASIC, this technically should work if done right.

The best way is probably to just give it a try. Or ask MT support.

7

Hi everyone, I’ve been testing VXLAN in hardware offload through two switches. HW3 active, management VLAN, all configurations on the bridge, VXLAN done, added to the bridge, VXLAN hardware offload is active. But I noticed, I believe it’s what the documentation says, there’s no way to pass VLAN tag inside the tunnel, I don’t know if anyone has managed to do this.

8

I have tested it in the same manner on v7.18.2

intarface (tagged 887) → bridge(tagged 887, vlan filtering on) → vxlan (untagged 887) , traffic is not going to vxlan at all.

9

Hi! Same here with 2 CRS326 2Q 24S+. Idk how to proceed