ibgp inside ipsec problem

RouterOS Beginner Basics
1

Hi,

I’m fairly new to Mikrotik and started to play with the software on VMs only recently.

I would like to have this network structure:


     +-- r1 ----- leased-line ----- r3 --+
LAN1 -+   |                          |    +- LAN2
      +-- r2 ----- Internet -------- r4 --+

(r1, r2) and (r3, r4) are in a VRRP failover configuration, distributed across two sites. The routers are are also cross-connected via a bonding interface. But there should be more sites forthcoming, if things work out well enough.

The traffic between LAN1 and LAN2 should usually run over the leased line and only fall back to the Internet, if the leased line breaks. The site-to-site traffic needs to be secured with IPSEC, regardless of whether it’s on the leased line or not. To that effect, I configured two IPSEC tunnels between r1 and r3, and r2 and r4, respectively, according to this tutorial:

http://mikrotikroutersetup.blogspot.co.uk/2012/01/mikrotik-routers-site-to-site-gre-over-ipsec-tunnel-configuration.html

The tunnel stuff all works fine. Now I wanted to establish BGP sessions between r1 and r2 (works), r3 and r4 (works), r2 and r4 (also works), and r1 and r3, which is where things break down horribly. The reason for that is that the moment I enable the BGP instance, the IPSEC tunnel between r1 and r3 goes down, and the BGP session also does not work. I have bgp multihop enabled for the session between r2 and r4, but not between r1 and r3 - it’s a direct link, isn’t it? To make the BGP sessions run inside the tunnels, I have configured them to use the LAN side IP addresses for update-source and remote-address.

Btw, using the sniffer tool on the GRE interface yields nothing useful, esp. no routing traffic, and on the physical interface, I can only see ARP and IPSEC traffic.

I’m using RouterOS 6.29.1 with a bunch of VMs to simulate this situation.

What gives?

2

In order to simply the IPSEC, use EoIP over IPSEC on both links and then you can put a subnet across the EOiP interfaces as if they were connected via an ethernet cable. We have deployed OSPF, BGP and MPLS in this way with successful results. This will give you a good idea of what it needs to look like:

http://forum.mikrotik.com/t/eoip-over-ipsec-two-rb750/47596/1

Starting in RouterOS 6.30 (still a release candidate) EoIP includes a simplified IPSEC config on the EOIP interface that only requires a key on both sides. You can download the 6.30RC and test if it isn’t a production environment.

3

Hi,
thanks for the ideas. I’ll certainly look into that, but have a few comments up front:

This approach unfortunately should make broadcast traffic travel (clog) the WAN line(s). I’m unsure as to what I can do without doing EoIP, but other people seem to have been doing that for years, so I’m probably doing something stupid that it doesn’t work.

We have deployed OSPF, BGP and MPLS in this way with successful results. This will give you a good idea of what it needs to look like:

http://forum.mikrotik.com/t/eoip-over-ipsec-two-rb750/47596/1

The idea does certainly look like a good fallback option, which I’m going to try.

You can download the 6.30RC and test if it isn’t a production environment.

I am actually trying the RC for other reasons, and hope it stabilizes enough until I need to be ready. Reading this thread here raises some doubts, though (as a newbie, I’m unable to test everything they deem important): http://www.reddit.com/r/networking/comments/28ra40/whats_your_general_experience_with_mikrotik/

4

Good luck…routers don’t generate enough broadcast traffic to really be concerned about at all. When WAN link speeds were measured in Kbps, this was more of a concern as there just wasn’t much bandwidth available, but on modern WAN circuits, and even on a T-1, the broadcast traffic generated by a router is very negligible.

Even hosts, which were traditionally the biggest offenders on Broadcast traffic are much cleaner these days which is why subnets can be used larger than the traditional /24 boundary although we don’t normally exceed /22 in the Enterprise or Data Center (Service Provider networks are an exception to this).

Take a look at this for some info on broadcast concerns.

http://etherealmind.com/many-hosts-vlan-ip-subnet/

5

Hi,

thanks for clearing this up. Your link about how many hosts to have on a VLAN was interesting to read!

You earlier suggested that I try the new RouterOS release candidates because they support dynamically generating IPSEC policies. Unfortunately, they do not enable tunnel mode, only transport mode, whereas all other tutorials suggest using tunnel mode. As these policies are dynamically generated, I have no idea how I can not properly alter them to use tunnel mode in every case. So far, I have never saw a use for transport mode anywhere…

6

No problem…Tunnel mode isn’t needed when building a Layer2 EoIP tunnel, because the only IPs needed for the SA relationship are the tunnel source and destination - in other words a single set of IP pairings for transport.

Transport mode has less overhead than Tunnel and performs a little better. If you need the header encrypted as well, then you can build IPSEC underneath the EoIP and use tunnel mode - just depends on your security requirements.