EVPN + VXLAN + Route Reflector

Hi All,

What works OOB

Spine + Route Reflector
S1-RR acts as a route reflector (RR) with loopback 10.10.10.1/32, ASN 6500.

L1, L2, and L3 are leaf routers peering with S1-RR using /31 transit subnets:

L1 ↔ S1-RR: 172.29.0.0/31

L2 ↔ S1-RR: 172.29.0.2/31

L3 ↔ S1-RR: 172.29.0.4/31

BFD is working well too

Partial Working

Currently, only one VNI is functional at a time. Additionally, configuring EVPN via /routing/bgp/evpn occasionally cause hang that leads to router crash when it reboot it will work eventually

I’m not sure if this is a GNS3-related bug. At this point, migrating the lab to real hardware feels excessive due to the instability and effort involved.


| VNI  | Purpose        | Route Distinguisher (RD) | Route Targets             |
| ---- | -------------- | ------------------------ | ------------------------- |
| 1010 | DATA (V100)    | 6500:1010                | 6500:1010 (import/export) |
| 1011 | STORAGE (V101) | 6500:1011                | 6500:1011 (import/export) |

Need Clarification

For further testing beyond this lab setup, VLAN trunk should work between L1,L2 and L3 with hypervisor host this is an absolute minimum :slight_smile: (as shown in the diagram).

At the moment, I’m unclear on the correct configuration steps to enable this, can someone shed some light on this please?

2of3.png
1of3.png
3of3.png
config.png

https://www.youtube.com/watch?v=dpukLeiRlV0

Wow we almost have the same topology :slight_smile: great thanks priceless but i’m still thinking there should be a way to pass multiple VLAN within the same VNI this is wasteful of interface :slight_smile: again thanks great find

[admin@L1] > /interface/vxlan/print detail
Flags: X - disabled, R - running; H - hw-offloaded
 0 R  name="VXLAN-DATA" mtu=1500 l2mtu=65535 mac-address=BA:99:5F:A6:6E:16 arp=enabled arp-timeout=auto
      loop-protect=default loop-protect-status=off loop-protect-send-interval=5s loop-protect-disable-time=5m vni=1010
      port=4789 local-address=172.29.0.1 dont-fragment=auto vrf=main vteps-ip-version=ipv4 allow-fast-path=yes
      max-fdb-size=4096 ttl=auto learning=no checksum=no hw=yes bridge=LAN bridge-pvid=100

 1 R  name="VXLAN-STORAGE" mtu=1500 l2mtu=65535 mac-address=BA:99:5F:A6:6E:16 arp=enabled arp-timeout=auto
      loop-protect=default loop-protect-status=off loop-protect-send-interval=5s loop-protect-disable-time=5m vni=1011
      port=4789 local-address=172.29.0.1 dont-fragment=auto vrf=main vteps-ip-version=ipv4 allow-fast-path=yes
      max-fdb-size=4096 ttl=auto learning=no checksum=no hw=yes bridge=LAN bridge-pvid=101
[admin@L1] > /interface/vxlan/print

the key is learning=no in the vxlan part, the router crash i’ve mentioned above could no longer reproduce

I found a bug, when you do have multiple vxlan and VNI only one vtep has been dynamically created, only the last entry on the /interface/vxlan menu. creating a manual vtep is the workaround for the setup to work upon next reboot

You can trigger the creation of dynamic VTEP for all VNI by using this /interface/vxlan/set dont-fragment=disable [index] to make this work but it won’t survived upon next reboot therefore creating a manual VTEP is necessary.

Toggling Enable/Disable on the vxlan interface is not enough to create a Dynamic VTEP this command /interface/vxlan/set dont-fragment=disable [index] is my only reliable way to trigger the creation of VTEP

Disclaimer: this happen on GNS3 i haven’t tried this yet on a real hardware

https://help.mikrotik.com/servicedesk/servicedesk/customer/portal/1/SUP-189706
bug.png

Where is support to MAC VRF in RouterOS?

We are not there yet I think VRF Mac support will be in a distant future :slight_smile: I hope someone from MT can confirm this.

I’m going to recall this bug report, the soon as I put my loopback address as local-address in VXLAN interface both VTEPS has been dynamically created, case close user error !

https://help.mikrotik.com/servicedesk/servicedesk/customer/portal/1/SUP-189706

I spoke to soon, there still something odd with this dynamic vtep creation since i was able to define properly my local-address in vxlan interface toggling on and off the vxlan interface create the desired VTEP.

/routing/bgp/evpn menu triggers the creation of VTEP for sure, but there’s a bug on that code that’s very fragile

let us know the outcome of the servicedesk

will surely do, I hope they were able to reproduce it and fix it

EDIT: as per mrz they are going to fix this together with some issues with evpn and vxlan in next beta :slight_smile:

Stopped for now because of the bug. I’m waiting for the next beta or rc!!!

Yeah, actually what we have right now is just the tip of the iceberg even though they land some fixes to make this very useful in DC/SP setting, some other folks here mentioned already some of the needed and must have feature to make this even worthwhile like mac-vrf, anycast gateway etc… but MT is in the right track on this one I hope they can blow the competition and bring this stuff to finish line this will be a game changer if they do

has anyone to share performance-data with real hardware for a evpn-vxlan setup ?

… what I read here L3HWFeatureSupport

… and from my testing … it’s a “for-future-use-only-exercise” in the moment

while subsecond-redundancy works fine, the rest is just ‘basic connectivity’ !?

maybe others achieved more ?