I’m trying to apply L3 HW Offload for VXLAN on a CCR2216. The problem is, I’ve done everything the documentation recommends: creating the VXLAN interface, giving it a local address, a PVID, and a bridge, so it appears as dynamic on the bridge interfaces and as untagged on the VLAN ID correctly. But when sending traffic from the Node01 router to the Edge Router, I see that the entire load is going to the CPU and not the hardware. Here’s a diagram of how everything is connected, although it seems a little strange to me that the load is still on the CPU.
As far as i’m aware, the hardware offload that’s been enabled will only work if you’re bridging the traffic to another device to handle Layer 3. In your scenario, VxLAN encap/decap and L3 forwarding are all happening in the same device which will happen in the CPU currently.
Most VxLAN implementations use something called IRB or Integrated Routing/Bridging to enable routing between VNIs and between a VNI and another L3 hop. It has to be supported in the hardware and in the network OS.
Overlay (forwarding between Ethernet and VXLAN):
1. VLAN tagging over VXLAN is not supported,
2. Routing between different VXLAN VNIs is not supported,
3. VTEPs are isolated, and there is no mechanism to control "horizon" between them.
4. Bridged VXLAN interfaces do not support IGMP snooping. When snooping is enabled, MDB entries on VXLAN are not offloaded, and multicast traffic gets restricted between Ethernet and VXLAN.
5. Bridged VXLAN interfaces are not supported by MLAG.
Hi dude! Thanks for your response. Now I’m testing vxlan betweentwo CRS326-24S+2Q but it’s impossible to make it work. Tested in RouterOS 7.17.2 and 7.18.2 the result is the same, can’t pass traffic with two routers under each CRS. Here is a topology diagram and the config of the CRS’s. Also tested with sfpplus1 in untagged mode.
Switch 1
/interface bridge
add name=bridge-vlans protocol-mode=none pvid=3 vlan-filtering=yes
/interface ethernet
set [ find default-name=ether1 ] mac-address=DC:2C:6E:90:21:1E
set [ find default-name=qsfpplus1-1 ] comment="to S2" l2mtu=1580
set [ find default-name=sfp-sfpplus1 ] comment="ro R1" l2mtu=1580
/interface vxlan
add bridge=bridge-vlans bridge-pvid=657 dont-fragment=disabled local-address=172.20.67.1 \
mac-address=12:60:B1:AB:69:F1 name=vxlan1 vni=20170
/interface lte apn
set [ find default=yes ] ip-type=ipv4 use-network-apn=no
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge-vlans interface=sfp-sfpplus1 pvid=45
add bridge=bridge-vlans interface=vxlan1 pvid=657
/ip firewall connection tracking
set udp-timeout=10s
/ip neighbor discovery-settings
set discover-interface-list=!dynamic
/ip settings
set max-neighbor-entries=8192
/ipv6 settings
set disable-ipv6=yes max-neighbor-entries=8192
/interface bridge vlan
add bridge=bridge-vlans comment="added by pvid" tagged=sfp-sfpplus1 vlan-ids=657
/interface ethernet switch
set 0 l3-hw-offloading=yes qos-hw-offloading=yes
/interface ovpn-server server
add auth=sha1,md5 mac-address=FE:28:DC:80:53:58 name=ovpn-server1
/interface vxlan vteps
add interface=vxlan1 remote-ip=172.20.67.2
/ip address
add address=172.20.66.1/30 interface=qsfpplus1-1 network=172.20.66.0
add address=172.20.67.1 interface=lo network=172.20.67.1
/ip dhcp-client
add interface=ether1
/ip hotspot profile
set [ find default=yes ] html-directory=hotspot
/ip ipsec profile
set [ find default=yes ] dpd-interval=2m dpd-maximum-failures=5
/routing bfd configuration
add disabled=no interfaces=all min-rx=200ms min-tx=200ms multiplier=5
/routing bgp connection
add address-families=ip as=65530 connect=yes disabled=no listen=yes local.address=172.20.66.1 \
.role=ibgp name=bgp1 nexthop-choice=force-self output.filter-chain=out .redistribute=\
connected remote.address=172.20.66.2/32 .as=65530 router-id=172.20.66.1 routing-table=\
main templates=default
/routing filter rule
add chain=out disabled=no rule="if(dst in 172.20.67.0/24 && dst-len in 32) {accept;}"
add chain=out disabled=no rule="if(dst in 0.0.0.0/0 && dst-len in 0-32) {reject;}"
/system clock
set time-zone-name=America/Argentina/Cordoba
/system identity
set name=S1
/system note
set show-at-login=no
/system package update
set channel=development
/system routerboard settings
set enter-setup-on=delete-key
Switch 2
/interface bridge
add name=bridge-vlans protocol-mode=none pvid=5 vlan-filtering=yes
/interface ethernet
set [ find default-name=qsfpplus1-1 ] comment="to S1" l2mtu=1580
set [ find default-name=sfp-sfpplus1 ] comment="to R2" l2mtu=1580
/interface vxlan
add bridge=bridge-vlans bridge-pvid=657 dont-fragment=disabled local-address=172.20.67.2 \
mac-address=D2:E2:29:87:97:3B name=vxlan1 vni=20170
/ip smb users
set [ find default=yes ] disabled=yes
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge-vlans interface=sfp-sfpplus1 pvid=345
add bridge=bridge-vlans interface=vxlan1 pvid=657
/ip firewall connection tracking
set udp-timeout=10s
/interface bridge vlan
add bridge=bridge-vlans comment="added by pvid" tagged=sfp-sfpplus1 vlan-ids=657
/interface ethernet switch
set 0 l3-hw-offloading=yes qos-hw-offloading=yes
/interface ovpn-server server
add auth=sha1,md5 mac-address=FE:20:2C:80:FC:A1 name=ovpn-server1
/interface vxlan vteps
add interface=vxlan1 remote-ip=172.20.67.1
/ip address
add address=172.20.66.2/30 interface=qsfpplus1-1 network=172.20.66.0
add address=172.20.67.2 interface=lo network=172.20.67.2
/ip dhcp-client
add interface=ether1
/ip ipsec profile
set [ find default=yes ] dpd-interval=2m dpd-maximum-failures=5
/ip smb shares
set [ find default=yes ] directory=/flash/pub
/routing bgp connection
add address-families=ip as=65530 connect=yes disabled=no listen=yes local.address=172.20.66.2 \
.role=ibgp name=bgp1 nexthop-choice=force-self output.filter-chain=out .redistribute=\
connected remote.address=172.20.66.1/32 .as=65530 router-id=172.20.66.2 routing-table=\
main templates=default
/routing filter rule
add chain=out disabled=no rule="if(dst in 172.20.67.0/24 && dst-len in 32) {accept;}"
add chain=out disabled=no rule="if(dst in 0.0.0.0/0 && dst-len in 0-32) {reject;}"
/system clock
set time-zone-name=America/Argentina/Cordoba
/system identity
set name=S2
/system note
set show-at-login=no
/system package update
set channel=development
Problem: disabled - the DF flag is not set on the outer IPv4 header, which means that packets can be fragmented if they are too large to be sent over the outgoing interface. This also allows packet fragmentation when VXLAN uses IPv6 underlay. Disables hardware offloading on compatible devices
Rare case now. After changing the dont-fragment option everything works fine but at the moment to install a new switch between the two CRS326 the traffic stops. Something weird is if vlan filtering is disabled for the “middle” switch, the traffic is passed again. So, this tells that the vxlan tunnel is not working cause the middle switch change it’s config to full L2. This was tested over bond and clean interfaces.
