Proxy ARP on VLAN/Bridge - was working, now isn't

RouterOS Beginner Basics
1

A quick question to start off with: Exactly where should proxy-arp be enabled for a routable bridge interface with one ethernet port and a VLAN/trunked port? So that the devices on the public network can get out through the router.

In my config (ROS 6.35.4 on an RB850Gx2) I have the following, which worked fine since setting it up yesterday (it downloaded 45GB overnight) until I moved the router to my rack from my test area and plugged it back in:

  • PPPoE dialer through Eth1, which gets the IP address x.y.z.17 assigned by ISP. I have a subnet x.y.z.16/29.
  • I set this same .17 address on a bridge interface which has port Eth2 and VLAN-FRED (that is also on Eth5 for a trunk, in case I need it). This is to bridge the public subnet onto my public-facing servers. FRED=Firewalled RED, by the way.
  • Either VLAN-FRED or the bridge was set to proxy-arp. I forget which one I put it on (the VLAN I think) because I have changed it so much while troubleshooting. Remember though, everything worked before moving the box.
  • On Port Eth2 is my ESXi server, with a Windows 2008 guest with IP address x.y.z.19. It spent the night connected to the internet (through the .17 default route) with no trouble.

I moved the router and connected into the same switch ports as it was during testing. After moving, the server-to-router connection only works briefly, and then stops. I’m testing my running a continual ping from server (.19) to router (.17). Now…

  • If I change the VLAN-FRED arp setting to disabled on one of the mentioned interfaces, I immediately get response from the ping, for exactly 24 seconds every time. During that time the arp entry for .17 exists on the server. Then the pings fail and the arp entry is no longer there.
  • If I then set the interface back to proxy-arp, the above repeats, for 24 seconds.

I also have the server, router and my LAN connected on a 192.168.1.0/24 network, and all of that works perfectly which is how I can get on the server with vSphere. The server has no gateway on that network; only on the public subnet. It’s worked that way for years with a Cisco router, which has now been replaced.
I’m completely puzzled as to why everything worked perfectly until I moved the router into the loft (Well of course, it’s hard to get to now… so it’s bound to fail).

I’ll put the relevant parts of my config below. If someone could shed any light on this, I would be grateful.

/interface bridge
add name=Bridge-VLAN-Cisco
add arp=proxy-arp name=Bridge-VLAN-FRED
add name=Bridge-VLAN-LAN

/interface ethernet
set [ find default-name=ether1 ] mtu=1492 name=Eth1-WAN
set [ find default-name=ether2 ] name=Eth2-FRED
set [ find default-name=ether3 ] name=Eth3-LAN
set [ find default-name=ether4 ] name=Eth4-Cisco
set [ find default-name=ether5 ] name=Eth5-Trunk

/interface pppoe-client
add add-default-route=yes disabled=no interface=Eth1-WAN keepalive-timeout=disabled \
    name=PPPoE password=xxx use-peer-dns=yes user=xxx

/interface vlan
add interface=Eth5-Trunk name=VLAN-Cisco vlan-id=999
add arp=proxy-arp interface=Eth5-Trunk name=VLAN-FRED vlan-id=990
add interface=Eth5-Trunk name=VLAN-LAN vlan-id=1

/interface bridge port
add bridge=Bridge-VLAN-Cisco interface=VLAN-Cisco
add bridge=Bridge-VLAN-Cisco interface=Eth4-Cisco
add bridge=Bridge-VLAN-FRED interface=Eth2-FRED
add bridge=Bridge-VLAN-FRED interface=VLAN-FRED
add bridge=Bridge-VLAN-LAN interface=VLAN-LAN
add bridge=Bridge-VLAN-LAN interface=Eth3-LAN

/interface bridge settings
set use-ip-firewall=yes

/ip address
add address=192.168.1.254/24 interface=Bridge-VLAN-LAN network=192.168.1.0
add address=x.y.z.17/29 interface=Bridge-VLAN-FRED network=x.y.z.16
add address=192.168.0.254/24 interface=Bridge-VLAN-Cisco network=192.168.0.0

Edit to add:
I’ve just realised that I can’t get a remote desktop to the server either. It’s as if it isn’t there. That is odd as that goes over the 192.168.1.0 network, and that was working yesterday too. However, I can ping it from my local PC and even file sharing (both ways) works. In fact, I’m connected now on vSphere and that is over 192.168 as well. But not RDP… hmmm…

2

The behavior you describe sounds like STP disabling an interface on your bridge. Check for loops.

Sent from my LG-D850 using Tapatalk

3

It does indeed, and at one point I did make a loop momentarily by connecting the ‘just in case’ trunk port to my switch, along with the non-trunked ports. I shut that port on the switch down in the time it took to get down from the loft (I meant to do it before, but I forgot). But that was a while before the failure started, and it definitely was working in between.

I wonder if something got ‘stuck’ somehow because of that, and didn’t show up for a while. It seems that way now, after everything I’ve been through.

Anyway, to add to the strangeness…

I decided to cut back to simple and I removed the bridge and VLAN from FRED from my config, just leaving the Eth2 port with the .17 assigned directly and with proxy arp. I also clicked the ‘safe’ button before doing it, just in case I messed up.

That config worked. The continual ping that I had never stopped immediately got a response as soon as I set the proxy arp.

However, I forgot to click the ‘safe’ button again to fix the changes, and a little while later my config got reverted. Guess what? It still worked, just as it did before it failed. With the very same config and exactly the same ports plugged in. It’s still working now, half a day later.

That’s despite having to pull down my ESXi server and reinstall it, because I decided at one point to just reboot everything and see if it cleared the issue (it didn’t) and the USB flash drive that the server booted ESXi from decided to die. The fun never ends… what’s going to die next???

4

After working all this time, this afternoon I noticed the server had no connection again. Exactly the same issue as before, with toggling the arp giving a 24-25 second connection. I guess that ‘resets’ the port in some way. I’m plugged into a different port on my main (not Mikrotik) switch this time, to eliminate a bad port there. I also eliminated a bad port or misconfiguration on the server by connecting a laptop in its place - same thing still happened.

I can’t see it being a loop (and the switch doesn’t show any RSTP issues) because the trunk port is shut down on the switch, and it worked for so long.

I’ve gone back to not having the bridge/vlan/trunk port, with just Eth2 itself having the IP address for the public-facing server. Wish I could figure this out… I’m running short on switch ports (hence the trunk).