I am trying to setup an EoIP Tunnel with OSPF, IPSEC, MPLS and VPLS. But I’m having an issue with OSPF flapping. I create the EoIP Tunnel with no issues. But as soon as I add a OSPF Network it starts flapping. Both ends are running v6.38.5 One end is a CCR1016-12G and the other is a RB3011.
So when I say flapping, if I look at the routing table, I can see routes added and then immediately removed because they are unreachable.
Quick Description of What I’ve Done:
Both Sides, Created Bridge called “Bridge Loopback” and Assigned an IP 192.168.100.x/32 to the Bridge
Both Sides, Created EoIP Tunnel with 1500 MTU
Both Sides, Add IP Address to EoIP Tunnel 192.168.110.12/30 using .13 and .14
Both Sides, Edited the default OSPF Instance adding Unique Router ID and Setting Route Options to Never / As Type 1 / As Type 1 / No / No / No
Both Sides, Created OSPF Network 192.168.110.12/30
At this point I’ve allowed the OSPF Interface to be dynamically created, although I have statically created it with no change to the Flapping Issue.
From the 192.168.110.13 side I can ping .14 except for 1-2 failures when it’s flapping.
I added OSPF to the Log and have looked at it, but can’t figure out any particular issue that’s going on. There are no “red” log entries.
New to MikroTik, so please let me know what information you want and how I might quickly get it for you.
If you’re trying to ping .13 from .14 (or vice-versa) and are seeing packet loss, then that is your issue, and OSPF is the symptom, not the cause.
When you put the x.x.x.12/30 subnet onto the GRE tunnel interfaces, then those two routers will consider that subnet as a directly-connected route, which has a distance of zero - the highest possible priority. This will not be affected by OSPF flapping, as far as the directly-connected routers are concerned.
If you’re pinging either of the endpoints from somewhere else, then the OSPF flap could be responsible for this, of course, since the /30 needs to be in the routing table of whatever router you’re sourcing your pings from/behind.
So why can I ping .13 from .14 (or vice versa) without any issues until OSPF is enabled?
I’ve fully configured EoIP, IPSEC, MPLS and VPLS, then manually added the routes to the table and everything works as expected. Soon as I add the Network to OSPF and remove the static routes everything fails. The images below show the “basic” routing table before OSPF kicks in and then the very temporary table while OSPF is working.
Well, one thing that comes to mind is to wonder what routes come through the tunnels…
For instance if router A uses IP 1.2.3.4 as the remote address for the tunnel, if 1.2.3.4 comes in the tunnel via OSPF, then the route to the other end of the tunnel is going to be through the tunnel, causing it to collapse…
I believe the issue is caused by overlapping routes some how, even though Site A and Site B share no identical networks. When I go into OSPF default instance and disable redistribute connected Route and Static Routes and add the local LAN IP Network to the OSPF Network every things works correctly.
Yup it’s likely routes are being installed that break the EoIP tunnel. When it’s broken it gets rebuilt after the OSPF routes clear. Then OSPF neighbors up, installs routes and breaks the tunnel.
I’d need to see the configs to identify where you’ve overlapped. We could start with a dump of interface and IP address:
Here are the exports, please note that this configuration is functioning; as I have disable the redistribution of connected and static routes; and added network to the OSPF networks area.
[SITE A] > /interface export
# apr/18/2017 08:08:37 by RouterOS 6.38.5
# software id = P2HD-QSSV
#
/interface bridge
add name="Bridge Loopback"
add name="Bridge vpls53"
add name="Bridge vpls100"
/interface ethernet
set [ find default-name=ether1 ] name=Trunk
/interface eoip
add !keepalive mac-address=02:D7:95:B6:CC:0A mtu=1500 name="eoip to SITE C" \
remote-address=xx.xx.xx.112 tunnel-id=10005
add !keepalive mac-address=02:5B:A7:BA:6D:A3 mtu=1500 name="eoip to SITE B" \
remote-address=xx.xx.xx.10 tunnel-id=10004
add mac-address=FE:F2:A5:EE:94:83 mtu=1500 name="eoip to SITE D" \
remote-address=xx.xx.xx.90 tunnel-id=10008
add !keepalive mac-address=02:8D:76:F3:D6:93 mtu=1500 name=\
"eoip to SITE E" remote-address=xx.xx.xx.18 tunnel-id=10006
/interface vpls
add disabled=no l2mtu=1500 mac-address=02:99:22:00:72:21 name=vpls004_53 \
remote-peer=192.168.250.4 vpls-id=53:4
add disabled=no l2mtu=1500 mac-address=02:84:37:F2:E9:35 name=vpls006_100 \
remote-peer=192.168.250.6 vpls-id=100:6
/interface vlan
add interface=Trunk name="VLAN 10" vlan-id=10
add interface=Trunk name="VLAN 20" vlan-id=20
add interface=Trunk name="VLAN 50" vlan-id=50
add interface=Trunk name="VLAN 53" vlan-id=53
add interface=Trunk name="VLAN 90" vlan-id=90
add interface=Trunk name="VLAN 100" vlan-id=100
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/interface bridge port
add bridge="Bridge vpls53" interface="VLAN 53"
add bridge="Bridge vpls53" interface=vpls004_53
add bridge="Bridge vpls100" interface="VLAN 100"
add bridge="Bridge vpls100" interface=vpls006_100
[SITE A] > /ip address export
# apr/18/2017 08:16:37 by RouterOS 6.38.5
# software id = P2HD-QSSV
#
/ip address
add address=192.168.10.10/23 interface="VLAN 10" network=192.168.10.0
add address=192.168.250.1 interface="Bridge Loopback" network=192.168.250.1
add address=192.168.240.13/30 interface="eoip to SITE C" network=192.168.240.12
add address=192.168.240.9/30 interface="eoip to SITE B" network=192.168.240.8
add address=192.168.240.17/30 interface="eoip to SITE D" network=192.168.240.16
add address=192.168.240.21/30 interface="eoip to SITE E" network=192.168.240.20
What is the use for EoIP in this case over GRE w/30’s? EoIP is more commonly used to bridge 2 broadcast domains over IP. It shouldn’t impact functionality but I would use GRE if it was me.
The real issue here is connected route redistribution. You should avoid it if at all possible. Look at Site B’s routing table. It has a connected route for xx.xx.xx.8/30. When that is redistributed into OSPF to neighbors over the tunnel then Site A goes “Oh awesome! I can get to xx.xx.xx.8/30 via 192.168.240.10” and it injects that into it’s routing table. This causes the EoIP packets to try to be sent over the virtual tunnel interface which immediately goes down. When the tunnel goes down the OSPF route is removed and the EoIP tunnel rebuilds off of the default route and the process starts all over again.
So how do we fix it.
Don’t redistribute connected if possible. Define networks and areas as needed instead. Sure it seems like it makes your life easier but it also changes how the routes are presented within and to other areas.
If redistribute connected is absolutely required then you can look at filtering out the xx.xx.xx.8/30 advertisement to the neighboring routers.
As for EoIP vs GRE, I can not answer. This is what someone who was helping me was doing with his design. As for my own network layout, all the sites are separate Networks or VLANs with no overlapping with no need to connect broadcasting.
Switching to GRE will still allow me to do the MPLS/VPLS that I’m doing? Basically I want to take a VLAN from SITE A and define a port at SITE B with that VLAN. Such that the device at Site B can connect, via broadcast, with other devices on that VLAN no matter where they are connect.
Look at Site B’s routing table. It has a connected route for xx.xx.xx.8/30. When that is redistributed into OSPF to neighbors over the tunnel then Site A goes “Oh awesome! I can get to xx.xx.xx.8/30 via 192.168.240.10” and it injects that into it’s routing table.
That would explain why I have had issues winbox’ing into the external IP of a remote site at times. Definitely not what I want to do.
Thank you for your insight, it’s been very helpful!
That’s actually what you’d need EoIP for. I didn’t see you injecting any ports into the EoIP setup and by placing the IP address on EoIP tunnels you are in essence making them behave like a point to point GRE tunnel. To bridge a VLAN with an EoIP tunnel you add it as a bridge port to a bridge. The IP would be on the bridge not the EoIP tunnel.
Say you have VLAN11 in Site A with an IP range of 10.1.11.0/24 that you want in both locations (can be VLAN11 in both locations of 11 in A and 322 in B, for simplicity we’ll keep it 11 in each).
You can then add any ports into the br11 you want to and you will see them on either side. You could run VRRP between the 2 sites if needed. I adjust the MTU to 1458 to accommodate IPv4 + GRE headers that are added on.
