Community discussions

MikroTik App
 
voll
just joined
Topic Author
Posts: 9
Joined: Mon Sep 03, 2018 5:04 pm

BGP Load balance over two routers

Fri Apr 09, 2021 11:51 am

Hi guys!

We're a small ISP owning two /22 networks. Our upstream ISP provides us with two separate peering endpoints located in the same datacenter.

We've been running so far with one Edge CCR router which terminated both those peers via eBGP in a failover scenario (not loadbalance). From that edge, we have iBGP to all our secondary downstream routers advertising the default route.

==================

We recently moved to a 2 x CCR2004-1G-12S+2XS routers acting as edges and we wanted to terminate each peer in a different router possibly loadbalancing them as well. Similarly, each edge has iBGP peering with each of our secondary downstream routers.

We attempted to try this setup, but only works reliably if only one edge is up and the other down. Having them both up at the same time results in loss of some hosted services, and we suspect that's because inbound & outbound traffic might get mixed up inbetween peers. Our downstream routers receive two default routes via iBGP as expected with equal distance, hence only one active at a time.

We also attempted an iBGP link between the two Edges with default originate ON, but that did not seem to work as expected and made no difference. We've read a few similar posts on the forums suggesting AS prepends, OSPF and local prefs, but we're not 100% sure how to proceed given the context.

We don't advertise anything else other than those two /22 upstream via "Networks" (no redistribute connected or any filters).

Any suggestions would be most welcome!
 
User avatar
CZFan
Forum Guru
Forum Guru
Posts: 2098
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa, Krugersdorp (Home town of Brad Binder)
Contact:

Re: BGP Load balance over two routers

Fri Apr 09, 2021 12:37 pm

I think the best in a case for above is to contact a certified Mikrotik Consultant in your area.

These guys pay big money to ensure they have knowledge and skills and are there for things like this to assist

https://mikrotik.com/consultants
 
User avatar
StubArea51
Trainer
Trainer
Posts: 1739
Joined: Fri Aug 10, 2012 6:46 am
Location: stubarea51.net
Contact:

Re: BGP Load balance over two routers  [SOLVED]

Fri Apr 09, 2021 8:04 pm

It sounds like you need verify that your border routers are peered with each other and that both can route into the interior of the network.

Here is a basic overview of a BGP border design that allows for load balancing over multiple peerings.

Image
 
jw1
just joined
Posts: 12
Joined: Thu Jul 14, 2016 8:49 pm

Re: BGP Load balance over two routers

Fri Apr 09, 2021 9:03 pm

We're in a similar situation with 2 of our connections going to the same ISP/ASN. It somewhat limits what BGP metrics you can use to select the incoming path. First get your connections between your routers squared away, probably with OSPF, etc like IPANetEngineer shows. From there, in terms of your network, what we're doing is advertising both /22s to both eBGP peers. This is for backup, in case one peering goes down. Then we break it up into /24s so that we advertise half to one peer and half to another. This is how the incoming path selection is made since more specific routes are preferred. We've got all the /24s in filter lists in both routers, so if we need to move one subnet to the other router, it's just enable or disable a filter rule. You could do /23s instead of /24s if things are relatively balanced and don't need as much granularity, but /25s and up will probably be ignored by your ISP. Verify with your ISP that the additional number of routes you're advertising doesn't exceed their limit - it's less than 20 routes, so shouldn't be a big deal. Somewhere on your network you'll need the BGP network origination for both the /22s and the /24s.

We aren't doing anything to force any particular stream to go out on the same path it came in and vice versa. So traffic goes out the first eBGP router it gets to. Hasn't been an issue for us.
 
voll
just joined
Topic Author
Posts: 9
Joined: Mon Sep 03, 2018 5:04 pm

Re: BGP Load balance over two routers

Mon Apr 12, 2021 12:08 pm

Thank you both for your valuable input!

We're going to implement some changes tonight and will let you guys how we get on.

IPANetEngineer - we also realised we did not have "force self" for next hop on the edges so that might have confused the RRs. Also, we plan to enable "if-installed" instead of "always" for "default originate". That way if a peer goes down it will not keep the advertisment for default route towards the RRs.

jw1 - we already had the /22s advertised from both as you stated. Splitting the /24 is a brilliant idea for loadbalance and we might be looking at doing just that.

Thanks again!
 
User avatar
StubArea51
Trainer
Trainer
Posts: 1739
Joined: Fri Aug 10, 2012 6:46 am
Location: stubarea51.net
Contact:

Re: BGP Load balance over two routers

Mon Apr 12, 2021 4:18 pm

IPANetEngineer - we also realised we did not have "force self" for next hop on the edges so that might have confused the RRs. Also, we plan to enable "if-installed" instead of "always" for "default originate". That way if a peer goes down it will not keep the advertisment for default route towards the RRs.

Awesome...let us know how it goes! If you want some more examples of iBGP config, take a look at this preso from the 2013 MUM

https://mum.mikrotik.com/presentations/US13/kevin.pdf
 
voll
just joined
Topic Author
Posts: 9
Joined: Mon Sep 03, 2018 5:04 pm

Re: BGP Load balance over two routers

Tue Apr 13, 2021 11:30 am

Hi guys!

We had success over the modifications done last night, with some minor hiccups due to unrelated elements.

I wanted to correct my previous posts where I was referring to our downstream routers as route reflectors... none of them are that. So both our edges just mesh up to all other routers basically. Route reflectors might be a concept we'll get into once our router fleet grows beyond maintaining a reasonable iBGP mesh.

We're aware that OSPF is the right way to go forward and we will slowly start to plan and transition towards that now that we have edge redundancy.

Thanks again guys for all your helpful comments!

Who is online

Users browsing this forum: No registered users and 17 guests