Community discussions

 
plankanater
Member Candidate
Member Candidate
Topic Author
Posts: 166
Joined: Wed Mar 14, 2012 3:56 am

BGP Advice

Thu Jul 20, 2017 10:19 pm

I have 3 cores built in a triangle. They all iBGP with each other. I have a /20 that we announce out the cores to our up stream carriers. Pretty standard.

My issues arises that if I loose connectivity to a core or if it locks up and needs rebooted, my /20 is still being announced out the carriers on the router that is down causing traffic to come into a router that cant reach the rest of my network.

I dont quite need a route server because they are fully meshed.

I was thinking have two routers sitting in the middle that have iBGP and announce the /20 to the servers, then turn on "synchronize" on my block.

Any thoughts?
 
JimmyNyholm
Member Candidate
Member Candidate
Posts: 249
Joined: Mon Apr 25, 2016 2:16 am
Location: Sweden

Re: BGP Advice

Fri Jul 21, 2017 1:07 pm

With out all the questions about: Have you checked this is your peer that....

This is what BGP is. BGP is not a millisecond failover protocol. BGP is a self repairing system for inter as communication.
Probably it is the HOLD timer on your peers that making them beleve that you are still there.

In any event there are convergence times. Witch today are very slow on mikrotik.

The hold timer could be optimized with BFD put that must be negotiated with the peers that support it. And earlier versions of routeros had issues with BFD.

Despite All this only one thing are your announcements synchronized. One can live with all the quirks in MT but you have to know the beast.
A effect of restart of a core router is minimized taking the announcements off first, second making you not good for output packets from the internal net.
disable external peer waiting for incoming traffic to stop then restart waiting for ibgp to converge before enabling external peer again and your announcement.

Don't be afraid that lesser new traffic is incoming ingress. This is also in then nature of path choosing of bgp. It takes the older paths first (if they are equal in all other aspects off course)

Hope this gives you the bits and pieces to read, evolve and live....
 
plankanater
Member Candidate
Member Candidate
Topic Author
Posts: 166
Joined: Wed Mar 14, 2012 3:56 am

Re: BGP Advice

Fri Jul 21, 2017 6:58 pm

My bgp is working correctly it works perfect.

My issue is that if Core 1, 2, and 3, are all online but I loose connection to core 3 from my other two cores. Core 3 continues to announcing its /20 network out to the internet, this it the same /20 that the other cores announce, so inbound traffic hits core 3 from our carriers and stops because it cant get anywhere.

I need a way for core 3 to stop announcing its network if it looses contact to cores 1 and 2. I am thinking the way to do this is with the "Synchronize" option but I am not 100%. Or I could have a route server that only announces our /20 to the 3 cores.
 
pe1chl
Forum Guru
Forum Guru
Posts: 5927
Joined: Mon Jun 08, 2015 12:09 pm

Re: BGP Advice

Fri Jul 21, 2017 8:35 pm

How are your local nets connected now? Are they all directly connected to one or more of the cores? In that case a synchronize would work,
however you probably have further subnetted the /20 in that case (with one or more subnets connected to each of the cores) and it gets
tricky to announce the /20 when everything is functional, and a smaller net when the link between the cores is broken.
 
JimmyNyholm
Member Candidate
Member Candidate
Posts: 249
Joined: Mon Apr 25, 2016 2:16 am
Location: Sweden

Re: BGP Advice

Fri Jul 21, 2017 10:05 pm

My bgp is working correctly it works perfect.

My issue is that if Core 1, 2, and 3, are all online but I loose connection to core 3 from my other two cores. Core 3 continues to announcing its /20 network out to the internet, this it the same /20 that the other cores announce, so inbound traffic hits core 3 from our carriers and stops because it cant get anywhere.
I need a way for core 3 to stop announcing its network if it looses contact to cores 1 and 2. I am thinking the way to do this is with the "Synchronize" option but I am not 100%. Or I could have a route server that only announces our /20 to the 3 cores.
syncronize=yes. that is your deal. Only announce if you actually reach the net. Then it comes down to that probably should split your /20 announcement down to /21's and /22,s and /23's and /24's just to be prepaired for avoiding ddos attacks and such. good ip planning should allready be in blace splitting the network regionally and thus syncronics on smaller annoncments is what your igp gives you.
 
JimmyNyholm
Member Candidate
Member Candidate
Posts: 249
Joined: Mon Apr 25, 2016 2:16 am
Location: Sweden

Re: BGP Advice

Fri Jul 21, 2017 10:08 pm

My bgp is working correctly it works perfect.
Then it comes down to that probably should split your /20 announcement down to /21's and /22,s and /23's and /24's just to be prepaired for avoiding ddos attacks and such..
That being said create the RR if your now a lir ask your lir to do it. ask your peer to accept your prefix /20 down to /24 making you able to mitigate and traffic engineer more efficiently.
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4051
Joined: Wed May 11, 2011 6:08 pm

Re: BGP Advice

Fri Jul 21, 2017 10:35 pm

My network is designed such that I originate our master prefix at several of our access-layer routers into OSPF. This way, unused space gets sunk immediately. Then the border routers only have the network statements to match the /20 block prefix, but no "nailed up static null route" on the border routers themselves.

This way, if the border router is active, and has an active link to its eBGP peer(s), but is isolated from the rest of the ASN for some reason (failed optic on the inward-facing links, for instance), then it will withdraw its advertisements of our IP blocks (as well as any longer-prefix sub-sets of these being announced for traffic engineering purposes).

Our DFZ is not large enough to completely mandate that we deploy RR hosts yet - but when I do, it's going to be the RR where I advertise the nail-up prefixes, and that will be directly into iBGP. The next hop of these nail-up prefixes will be to some /32 that I designate as a recursive route pointing at Null0 (we use Cisco for our core - sorry, Mikrotik) so that every router will know to black hole the master prefixes locally. If some border router cannot reach the RR, then that means it (the border router) is isolated from our core, so it will drop the prefixes by virtue of losing the iBGP session. (I'll experiment with this to decide whether I would rather continue using OSPF to propagate the route from the RR to the border routers for convergence time reasons)
When given a spoon,
you should not cling to your fork.
The soup will get cold.
 
plankanater
Member Candidate
Member Candidate
Topic Author
Posts: 166
Joined: Wed Mar 14, 2012 3:56 am

Re: BGP Advice

Mon Aug 07, 2017 8:59 pm

Thanks for the replies, mikrotik didnt notify me about your posts.

The network is set up as a triangle, with each router having 10 gig to the other two routers. With this we run OSPF. Then we BGP peer between the loop-backs and put on multihop. That way it can failover if a link were to go offline.

My issue is that my CCR1072-1G-8S+ are locking up. I have had each core lock up over the last 2 months and I need to log in with mac telnet and reboot the router.

I was thinking of setting a new router or two in the middle have it hold the Networks and then bgp peer them to each of the cores. Then turn on synchronize on the cores. This should hopefully keep the routers from announcing out when the loose connection to the rest of the network.
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4051
Joined: Wed May 11, 2011 6:08 pm

Re: BGP Advice

Mon Aug 07, 2017 9:15 pm

I was thinking of setting a new router or two in the middle have it hold the Networks and then bgp peer them to each of the cores. Then turn on synchronize on the cores. This should hopefully keep the routers from announcing out when the loose connection to the rest of the network.
That will work perfectly. You can even configure this/these router(s) as RR hosts. Peer the cores with each RR and not with the other cores. This also gives you the advantage of having centralized "route servers" where you can dictate centralized policy quite easily - for instance you could inject a route at the RR which has a blackhole community on it, and this will propagate out to your core routers which will adopt the blackhole route. If each core's connected ISP supports a blackhole community, then you can use out-filter on each core router to translate your internal blackhole community into whatever community the ISP uses for blackhole routing, and you can quickly mitigate a DDoS by blackholing the target if that becomes necessary.
When given a spoon,
you should not cling to your fork.
The soup will get cold.
 
chubbs596
Frequent Visitor
Frequent Visitor
Posts: 51
Joined: Fri Dec 06, 2013 6:07 pm

Re: BGP Advice

Tue Aug 22, 2017 8:44 pm

HI
Maybe look at using netwatch and some scripts to make changes to routing filters or drop eBGP sessions.
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4051
Joined: Wed May 11, 2011 6:08 pm

Re: BGP Advice

Tue Aug 22, 2017 11:14 pm

HI
Maybe look at using netwatch and some scripts to make changes to routing filters or drop eBGP sessions.
That can work, but it's definitely not best practice, especially since BGP can already accomplish the goal naturally, so long as it's properly configured.
Basically, the /20 needs to originate somewhere other than on the three core routers, which learn the path via some dynamic means - it can be iBGP or via OSPF. If using iBGP, then there's no need to configure a network statement to originate the prefix (it's already IN the BGP table if it arrives via iBGP). If using OSPF, then you need a network statement to originate the prefix if reachable. (be sure the network statement has synchronize=yes, otherwise, the prefix will just be originated always, regardless of reachability)
When given a spoon,
you should not cling to your fork.
The soup will get cold.

Who is online

Users browsing this forum: MSN [Bot] and 2 guests