Unable to get basic VXLAN tunnel to work over Wireguard

RouterOS General
1

Hi all,

I’m trying to extend a layer 2 domain from my local network (RB4011) to a remote host I have. Latest RouterOS 7.15.3.

The local network and the remote host are successfully connected with Wireguard (configured on the router), and I can log into the remote host via the Wireguard tunnel.

The local Wireguard address is 192.168.254.254, the remote Wireguard address is 192.168.254.253

Eventually I’m looking for the VXLAN to be bridged to the existing L2 network, but in troubleshooting I’ve gotten things down to what I believe is the most basic case, still with no luck.

Local (Mikrotik) config:

/interface vxlan
add local-address=192.168.254.254 mac-address=E2:8E:4D:CD:DE:AF name=vxlan227 port=4789 vni=227 vrf=main vteps-ip-version=ipv4
/interface vxlan vteps
add interface=vxlan227 port=4789 remote-ip=192.168.254.253
/ip address
add address=10.227.176.254/24 interface=vxlan227 network=10.227.176.0

Remote (Linux host) config:

# ip link add vxlan0 type vxlan id 227 remote 192.168.254.254 local 192.168.254.253 dev wg0 dstport 4789
# ip addr add 10.227.176.239/24 dev vxlan0
# ip link set up dev vxlan0

Now, when I try to ping 10.227.176.254 from the remote host, I get no response. I set up packet sniffers on both ends filtering for the VXLAN interfaces, and found that the remote host is sending ARP requests for 10.227.176.254, but is not receiving replies. The local RB4011 is getting the requests, and sending the replies. So, the breakdown at this point seems to be the ARP replies being dropped on the way out to the host.

Mikrotik FDB:

/interface/vxlan/fdb> print 
 0 remote-ip=192.168.254.253 mac-address=86:AF:B2:2A:5D:6E interface=vxlan227

Host FDB:

# bridge fdb show dev vxlan0
00:00:00:00:00:00 dst 192.168.254.254 via wg0 self permanent

Appreciate any insight here.

2

Just an observation:
if your aim is to extend L2, why not use EOIP across wireguard ?
Connect both ends of the EOIP interface to their respective bridges and that should be more or less it (be careful with possible loops).

3

Since the VxLAN transport is not “symmetric” in terms that the packets from node A to node B use the port specified in the /interface vxlan vteps table as destination but a randomly chosen one as source, and so do the packets from node B to node A, could it be that a firewall on the Linux side does not explicitly accept the VxLAN transport packets towards port 4789 coming in via Wireguard?

@holvoetn, the OP mentions Linux as one endpoint, and EoIP is Mikrotik’s proprietary twist of GRE that serves the same purpose as Linux gretap but the headers are encoded differently. Regardless that, EoIP and gretap are point-to-point tunnels (so you have to bridge them together if you want to interconnect LANs on more than two endpoints) whereas VxLAN can work as a multipoint overlay L2 network on its own.

4

Oops, missed that part :open_mouth:

5

Thanks for your response. The firewall on the Linux node is wide open at the moment, so it’s not that.

I did a packet trace on the VXLAN interfaces themselves, but I did not run one on the wireguard interfaces, going to do that when I have a few minutes and will report back.

And yes, you hit the nail on the head re: EoIP. For now the VXLAN setup is point-to-point, that isn’t the long-term goal.

Thanks!

6

OK, I did a capture on the wg0 interface on the remote Linux host and that does indeed show the ARP reply frames being received over Wireguard, so they’re being dropped between wg0 and vxlan0.

Wondering if there are sysctls I need to set on the Linux side.

The remote host is CentOS Stream 9, pretty much default configuration, I’m tempted to spin up another box with something Debian based and see if I get the same result.