Community discussions

 
airbanduk
newbie
Topic Author
Posts: 45
Joined: Mon Jun 12, 2017 2:30 pm

CCR1072 as a route server?

Wed Jun 14, 2017 4:26 pm

I'm currently looking at a new design that will see chassis based routers take over our current dual-CCR1072 core. I want to put these to good use as they've been rock solid since deployment, and I was thinking of using them as dedicated route servers. I've read a few threads on here already and the wiki pages, and there seems to be some disagreement on whether they make good route servers or not, with some posts stating CCR can't do it at all.

I'm looking at two full tables (both v4 and v6) plus feeds from multiple peers, and thinking the CPU and memory on the CCR would make a good route server and unburden the core routers from BGP processing. There would eventually be two datacentres with 2 CCRs each as a route reflector cluster.

I've labbed this up with CHR and only a handful of prefixes, but the theory seems sound. Does anyone have any thoughts to this design or real world experience of deploying this kind of topology?
 
savage
Forum Guru
Forum Guru
Posts: 1211
Joined: Mon Oct 18, 2004 12:07 am
Location: Cape Town, South Africa
Contact:

Re: CCR1072 as a route server?

Thu Jun 15, 2017 1:15 pm

The CCR's very slow with BGP, you're going to end up waiting a very, very long time for the RS to push announcements/withdraws.

Because BGP only hammers on one CPU, the rest of your CCR will basically be wasted completely. We actually have instances where our CCR (+- 90 peers) actually get's SO busy with BGP, that even OSPF times out because it can't reliably send hellos.

We're moving all our infrastructure away from CCRs currently and for now, opted for the CHR rather. Still only bound by 1 CPU in terms of BGP and what not, but at least you can give it much faster Xeon CPUs, which does seem to help, a lot.
Regards,
Chris
 
airbanduk
newbie
Topic Author
Posts: 45
Joined: Mon Jun 12, 2017 2:30 pm

Re: CCR1072 as a route server?

Thu Jun 15, 2017 3:29 pm

Thanks for the thoughts.

Does a single prefix being withdrawn require a full recompute of BGP then? I know that the CCR1072 is slow on a full table - the mistake I made with a filter ended up pushing all prefixes from our ASR1k to the CCR, and even after I fixed it in less than a minute, it still spent a good 3-5 minutes adding and finally withdrawing the routes (about 250k at peak). The idea of using them as a reflector cluster was that once they'd loaded the table, they'd only be pushing the best routes to the core, and if one changed it wouldn't take it 5 minutes to send the update. I guess this then is not the case?

Could this be something to wait for ROS7, when multithread means more BGP power?

I might try the CHR on the core network as a route collector, and see how quickly it can build the full table.
 
savage
Forum Guru
Forum Guru
Posts: 1211
Joined: Mon Oct 18, 2004 12:07 am
Location: Cape Town, South Africa
Contact:

Re: CCR1072 as a route server?

Thu Jun 15, 2017 3:43 pm

Thanks for the thoughts.

Does a single prefix being withdrawn require a full recompute of BGP then? I know that the CCR1072 is slow on a full table - the mistake I made with a filter ended up pushing all prefixes from our ASR1k to the CCR, and even after I fixed it in less than a minute, it still spent a good 3-5 minutes adding and finally withdrawing the routes (about 250k at peak). The idea of using them as a reflector cluster was that once they'd loaded the table, they'd only be pushing the best routes to the core, and if one changed it wouldn't take it 5 minutes to send the update. I guess this then is not the case?

Could this be something to wait for ROS7, when multithread means more BGP power?

I might try the CHR on the core network as a route collector, and see how quickly it can build the full table.
CCR just from when it receives the announcement, until the route has been updated in the routing table can take as you say, 3 to 5 minutes - sometimes longer. Not only for the initial load, but for a path change too - especially when the BGP process (running on a single core) is busy due to a lot of processing (a lot of peers, full tables, etc.)

CHR does it within seconds (given a fast enough XEON or similar is available). Path changes happens pretty much on par to what you can except from a decent ASR or similar device. Initial full tables loads VERY fast (fastest I've ever seen to be honest).

Honestly, I wouldn't recommend that you waste your time with this on a CCR. Rather go CHR. I was also very hesitant initially with CHRs but they are proving to be rather stable, and much, much faster than the CCRs - in every aspect (not only BGP). If I could return my CCRs and get a refund for them, I would do so in a heartbeat. Even our cores and borders are being replaced with CHRs and we're taking CCRs out of the network, completely.

Also keep in mind, it's becoming rather common knowledge that the CCRs suffer from PSU failures too. Despite what MT claims, more and more users are coming forward with reports of PSU failures. The CCR IMHO, isn't ready for a critical service, such as what BGP is on the Enterprise network. It's definitely NOT something that you'd want to go down...
Regards,
Chris
 
airbanduk
newbie
Topic Author
Posts: 45
Joined: Mon Jun 12, 2017 2:30 pm

Re: CCR1072 as a route server?

Thu Jun 15, 2017 4:09 pm

I may just go back to the original design, which had the core routers do all the main BGP processing. I figured by offloading the BGP compute, the core could get back to doing its primary job of high speed packet switching.

Shame really because they've been really stable, but if they can't function properly as a BGP router, I don't actually have a need for them anymore. I could keep hold of them until ROS7 is out, then wait until there have been more tests on before circling back. By then though I doubt I'll want to be making major changes to the core topology.
 
User avatar
IPANetEngineer
Trainer
Trainer
Posts: 1053
Joined: Fri Aug 10, 2012 6:46 am
Location: Jackson, MS, USA
Contact:

Re: CCR1072 as a route server?

Thu Jun 15, 2017 4:09 pm

I'm inclined to agree with Savage. As a route server, you'll probably want to use BIRD or Free Range Routing.

The CCR1072 is best utilized for large volumes of IP transit - it does that job very well.
Global - MikroTik Support & Consulting - English | Francais | Español | Portuguese +1 855-645-7684
https://iparchitechs.com/services/mikro ... l-support/ mikrotiksupport@iparchitechs.com
 
airbanduk
newbie
Topic Author
Posts: 45
Joined: Mon Jun 12, 2017 2:30 pm

Re: CCR1072 as a route server?

Thu Jun 15, 2017 4:12 pm

The CCR1072 is best utilized for large volumes of IP transit
72 cores and 16GB RAM, just screams BGP router rather than packet forwarder. Shame it doesn't live up to that.

I'll look in to BIRD as well, I was thinking of creating a looking glass anyway.
 
User avatar
IPANetEngineer
Trainer
Trainer
Posts: 1053
Joined: Fri Aug 10, 2012 6:46 am
Location: Jackson, MS, USA
Contact:

Re: CCR1072 as a route server?

Thu Jun 15, 2017 4:14 pm

The CCR1072 is best utilized for large volumes of IP transit
72 cores and 16GB RAM, just screams BGP router rather than packet forwarder. Shame it doesn't live up to that.

I'll look in to BIRD as well, I was thinking of creating a looking glass anyway.
It actually handles BGP very well in network designs that don't require full tables. But if you need full tables, CHR is def the way to go.
Global - MikroTik Support & Consulting - English | Francais | Español | Portuguese +1 855-645-7684
https://iparchitechs.com/services/mikro ... l-support/ mikrotiksupport@iparchitechs.com

Who is online

Users browsing this forum: MSN [Bot] and 127 guests