Second opinion two routers BGP/OSPF

RouterOS Forwarding Protocols
1

Hello,

I am currently working with two Mikrotik CCR1036 routers
to announce a /24 subnet.
The goal is to make the network as redundant as possible, we have
the following setup:
bgmkt.png
R1:
/interface bridge
add name=loopback

/ip address
add address=1.1.1.2/30 interface=BGP1 network=1.1.1.0
add address=1.1.2.2/30 interface=BGP2 network=1.1.2.0
add address=1.1.6.1 interface=loopback network=1.1.6.1
add address=1.1.5.1/30 interface=Routers network=1.1.5.0

/routing bgp instance
set default as=123456 redistribute-connected=yes redistribute-ospf=yes redistribute-static=yes router-id=1.1.6.1
/routing bgp network
add network=3.3.3.3/24
/routing bgp peer
add in-filter=IPV4In name=BGP1 out-filter=IPV4Out remote-address=1.1.1.1 remote-as=1234567
add in-filter=IPV4In name=BGP2 out-filter=IPV4Out remote-address=1.1.2.1 remote-as=1234567
add name=R2 remote-address=1.1.5.2 remote-as=123456

/routing ospf instance
set [ find default=yes ] name=OSPF redistribute-connected=as-type-2 redistribute-static=as-type-2 router-id=1.1.6.1
/routing ospf interface
add interface=loopback network-type=point-to-point passive=yes
/routing ospf network
add area=backbone network=3.3.3.3/24


R2:
/interface bridge
add name=loopback

/ip address
add address=1.1.3.2/30 interface=BGP3 network=1.1.3.0
add address=1.1.4.2/30 interface=BGP4 network=1.1.4.0
add address=1.1.6.2 interface=loopback network=1.1.6.2
add address=1.1.5.2/30 interface=Routers network=1.1.5.0

/routing bgp instance
set default as=123456 redistribute-connected=yes redistribute-ospf=yes redistribute-static=yes router-id=1.1.6.2
/routing bgp network
add network=3.3.3.3/24
/routing bgp peer
add in-filter=IPV4In name=BGP3 out-filter=IPV4Out remote-address=1.1.3.1 remote-as=1234567
add in-filter=IPV4In name=BGP4 out-filter=IPV4Out remote-address=1.1.4.1 remote-as=1234567
add name=R1 remote-address=1.1.5.1 remote-as=123456

/routing ospf instance
set [ find default=yes ] name=OSPF redistribute-connected=as-type-2 redistribute-static=as-type-2 router-id=1.1.6.2
/routing ospf interface
add interface=loopback network-type=point-to-point passive=yes
/routing ospf network
add area=backbone network=3.3.3.3/24

The problem is that on both routers one CPU always has a 100% load, is this normal?
Does the setup look correct? Anybody has any improvement maybe?

Thanks

2

As for the 100% CPU, this seems to be the norm on CCR platforms, as the BGP process is not multi-threaded. A full table of BGP routes is pretty large, and the BGP process is constantly combing through it to make sure everything is up-to-date, etc. I think this is supposed to be fixed in ROSv7 where they’re re-writing the routing code.

As for tips about your BGP configuration:

On your iBGP peers, be sure to set “update-source=1.1.6.x” and use the remote router’s loopback IP - NOT the /30 link between them.

Get rid of “redistribute static, connected, and OSPF”
This is a very bad habit to get into.

You should use network statements with synchronize=yes on each one, and be sure that there’s a route - either static type=blackhole (or an OSPF route to some internal device) for each prefix that the router should advertise. I prefer to have the “nail-up” routes come from an internal OSPF source, and here’s why:

Suppose R1 were to have a failure of the inward-facing interface, but the router itself remains operational. If you nail up the route with a blackhole route locally in R1 (or use a network without synchronize=yes being set) then R1 is going to continue to advertise your prefixes to ISP1 and ISP2… even though it cannot reach your network! If you rely on OSPF-learned routes, then if R1 loses connectivity to the ultimate source of your IP addresses, then R1 will withdraw its advertisements of your prefixes from ISP1 and ISP2, allowing the world to converge on ISP3 and ISP4 to reach you exclusively via R2 until such time as R1’s connectivity to your core can be restored. Likewise, R1 will not be able to advertise any outbound routes to the rest of your network, so everything else will converge on R1 for outbound traffic forwarding via ISP3 or ISP4.

If you have customers directly attached to R1 and R2, then I strongly recommend that you use a separate /24 (at least /24 if not shorter prefix like /23 or /22) for each router - so that if you get a split internally, then R1 can still advertise the /24 of its local customers, and R2 can still advertise the /24 of its local customers as well.

Tips about OSPF:

I’d say that you should probably set the default interface to be passive=yes
Then specify exactly which interfaces you want adjacencies to form on, and add those manually with passive=no
Your network statements can be pretty simple:
3.3.3.0/24 area=backbone
(and if using private IPs for some link interfaces, add those networks too)
172.16.0.0/12 area=backbone
10.0.0.0/8 area=backbone
192.168.0.0/16 area=backbone

There are some drawbacks to doing it this way - mainly the fact that pretty much ANY interface is guaranteed to be in OSPF and if a time comes later that you want to remove an interface entirely from OSPF, then you’ll have to break up the network that covers it. (e.g. if you wanted an interface of 192.168.45.1/29 to be not in OSPF, you’d have to break up the 192.168.0.0/16 network in order to get rid of that prefix) However, if you’re already doing “redistribute connected” then there’s no difference in that regard. (but redistributed routes in OSPF are less-desirable, too, so try to properly originate everything that you can as an interior route.

— Finally - make sure that your ISP /30 links 1.1.1.0/30, 1.1.2.0/30, etc… are also announced in your OSPF - don’t forget to add those /30 networks to your OSPF if you go to clean up the redistributed connected routes in your OSPF table. iBGP will use the ISP’s /30 IP as the next-hop target, so those need to be available in your IGP.

3

@ZeroByte

Thanks very much for all your tips, they have been a great help. I know this is a little late from when you originally posted this but with a little luck this will still get to you.

After reading and re-reading what you have posted theres still a small thing I don’t understand and I wondered if you could fix that missing hole I have. Its all to do with, what IP address should you assign to the loopback. I understand from what you said that it should be the router ID however this makes me a little confused as if I put the router ID on the loopback, I then wouldn’t be able to connect to it. To try and explain what I mean is, im going to list below of exactly how I set my tunnels/OSPF sessons up with the hopes that you or someone else could point me to the correct way as I think theres something I am not doing quite right.

Currently the way I do my ospf links are like this. I first create a GRE tunnel using the public IP’s of the two connections. I then run IPSEC also using the same public IP’s to create the encrypted tunnel. I then add a private /30 and attach it to each side of the tunnel using the gre interface. After this is done, I then add the OSPF network (most cases a /24) as well as the /30. I don’t do anything with the loopback right now because I am unsure of what IP I should add there.

Below is the exact list of commands I would use to connect a tunnel/site up. Could you or someone point out what address I should add to the loopback and where that comes in?

Site A:
WAN Public IP: 1.1.1.1
LAN GW: 172.30.1.1
LAN SUBNET: 172.30.1.0/24
LINK addr: 172.30.0.221/30
ROUTER ID: 172.30.1.1

Site B:
WAN Public IP: 2.2.2.2
LAN GW: 172.30.2.1
LAN SUBNET: 172.30.2.0/24
LINK addr: 172.30.0.222/30
ROUTER ID: 172.30.2.1

Site A ↔ Site B

Net: CIDR Notation: 172.30.0.220/30
CIDR Address Range: 172.30.0.220 - 172.30.0.223
Usable: 172.30.0.221 - 172.30.0.222
Subnet: 255.255.255.252

# Site A
Mikrotik 1 Router IP Address Configuration:

[admin@MikroTik1] > /interface gre add name=gre-tunnel10 local-address=1.1.1.1 remote-address=2.2.2.2 comment="to Site B" allow-fast-path=yes mtu=1492
[admin@MikroTik1] > /ip address add address=172.30.0.221/30 interface=gre-tunnel10

Mikrotik1 Router IPSec VPN Configuration: 
[admin@MikroTik1] /ip ipsec peer add address=2.2.2.2 auth-method=pre-shared-key secret="wo42p7Gs7HPJzbZ6XCBdWAjNUjJl7S" generate-policy=no exchange-mode=main send-initial-contact=yes nat-traversal=no proposal-check=obey hash-algorithm=sha256 enc-algorithm=aes-256 dh-group=modp1024 lifetime=1d lifebytes=0 dpd-interval=disable-dpd dpd-maximum-failures=1
[admin@MikroTik1] /ip ipsec policy add src-address=1.1.1.1 dst-address=2.2.2.2 protocol=all action=encrypt level=require ipsec-protocols=esp tunnel=no sa-src-address=1.1.1.1 sa-dst-address=2.2.2.2 proposal=default priority=0

[admin@MikroTik1] > /ip firewall raw add chain=prerouting src-address=172.30.1.0/24 dst-address=172.30.2.0/24 action=notrack comment="Site A -> Site B"
[admin@MikroTik1] > /ip firewall raw add chain=prerouting src-address=172.30.2.0/24 dst-address=172.30.1.0/24 action=notrack comment="Site B -> Site A"

Mikroitk 1 Router OSPF Configuration:
[admin@MikroTik1] /routing ospf network add network=172.30.0.220/30 area=backbone
[admin@MikroTik1] /routing ospf network add network=172.30.1.0/24 area=backbone




# Site B
Mikrotik 2 Router IP Address Configuration:

[admin@MikroTik2] > /interface gre add name=gre-tunnel3 local-address=2.2.2.2 remote-address=1.1.1.1 comment="to Site A" allow-fast-path=yes mtu=1492
[admin@MikroTik2] > /ip address add address=172.30.0.222/30 interface=gre-tunnel3

Mikrotik1 Router IPSec VPN Configuration: 
[admin@MikroTik2] /ip ipsec peer add address=1.1.1.1 auth-method=pre-shared-key secret="wo42p7Gs7HPJzbZ6XCBdWAjNUjJl7S" generate-policy=no exchange-mode=main send-initial-contact=yes nat-traversal=no proposal-check=obey hash-algorithm=sha256 enc-algorithm=aes-256 dh-group=modp1024 lifetime=1d lifebytes=0 dpd-interval=disable-dpd dpd-maximum-failures=1
[admin@MikroTik2] /ip ipsec policy add src-address=2.2.2.2 dst-address=1.1.1.1 protocol=all action=encrypt level=require ipsec-protocols=esp tunnel=no sa-src-address=2.2.2.2 sa-dst-address=1.1.1.1 proposal=default priority=0

[admin@MikroTik2] > /ip firewall raw add chain=prerouting src-address=172.30.2.0/24 dst-address=172.30.1.0/24 action=notrack comment="Site B -> Site A"
[admin@MikroTik2] > /ip firewall raw add chain=prerouting src-address=172.30.1.0/24 dst-address=172.30.2.0/24 action=notrack comment="Site A -> Site B"

Mikroitk 2 Router OSPF Configuration:
[admin@MikroTik2] /routing ospf network add network=172.30.0.220/30 area=backbone
[admin@MikroTik2] /routing ospf network add network=172.30.2.0/24 area=backbone

Now this all works but I do keep getting random breaks in the network, I mean switches over to another link when they are still all up. I was thinking maybe thats to do with not using loopback addresses and or i’m not doing it correctly to begin with which I the likely cause.

Could someone be so kind and point me into the right direction. What I would love to know is, what address do you put on the router ID and loopback?

Also if change OSPF for BGP, I find the links just go up and down, up and down and they wont stay connected. From what I understand this is why you need to stick it on the loopback but no mater what IP I try and assign to it, it just doesn’t work. So there must be something I am miss understanding. Can anyone point me in the right direction?

Thanks in advance and sorry for bringing alive an old post.