Community discussions

MUM Europe 2020
 
randvegeta
just joined
Topic Author
Posts: 8
Joined: Tue Jul 22, 2014 6:24 am

BGP Router for Hosting Company

Tue Jul 22, 2014 6:35 am

Hello All,

We are looking at replacing our Juniper J series router and were considering Mikrotik Cloud Core Router. I was hoping to get some feed back and real world experience.

- We are looking at using the router almost exclusively for BGP.
- We have 2 ISPs now, but we would like to have 3 - 4. Each will be 100M or less and have full routing tables.
- Reliability and Uptime is our greatest concern.
- We need multiple routers in a High Availability / Automatic Failover configuration

Our plan is to have 1 router for each ISP and working on a high availability basis so that a failed/crashed router can be pick up by another.

As I understand the BGP implementation has been a little buggy and I was wondering how this affects performance? Any real life examples? Since I don't mind buying/setting up multiple routers, any way to improve performance if it is an issue?

What's reliability like on the CCRs or a x86 RouterOS box? Anyone use it in a 'data center' deployment? Are these suitable/capable for DC deployments in order to replace J series Junipers?

Thanks in advance guys.
 
raz
Member Candidate
Member Candidate
Posts: 102
Joined: Wed Dec 19, 2012 3:26 pm
Location: Austria

Re: BGP Router for Hosting Company

Tue Jul 22, 2014 2:42 pm

Hey randvegeta,

i would recommend to you CCR1016 or better, it does what you need.

Read here more:

http://wiki.mikrotik.com/wiki/Manual:Routing/BGP

http://robert.penz.name/779/howto-setup ... k-routers/

Easy.
 
joegoldman
Long time Member
Long time Member
Posts: 501
Joined: Mon May 27, 2013 2:05 am

Re: BGP Router for Hosting Company

Tue Jul 22, 2014 2:46 pm

Hi,

We used to use Cisco 7200/7300 devices at the core, and recently moved over to Mikrotik CCR at the core. We have had good success I have about 10 BGP sessions with about ~30k routes in my table.

I'm taking on my first full table tonight, coincidentally. Through a single CCR (have a hot-standby for failover) We currently run an aggregate of about 600mbit at only about 10% CPU.

From testing and others reports, BGP is still single core so processing like bringing in the full table locks up a single core and makes it slow(ish), I will be doing mine in safe mode via SSH to make sure it doesn't lock up the whole router but others have said they take on full tables without issue.

See this thread here:

http://forum.mikrotik.com/viewtopic.php?f=14&t=85249

For a lot more user experience of CCR in real world BGP deployments.
 
FutileNetworks
newbie
Posts: 37
Joined: Tue Jan 15, 2013 9:14 pm

Re: BGP Router for Hosting Company

Tue Jul 22, 2014 4:01 pm

We switched from cisco to ccr for our border edge BGP router, CCR1036-12g-4s with 6 BGP peers, ipv4 and ipv6, about 500 mbps traffic, never really goes above 10% except when taking in full routing tables, we turn off connection tracking on ours as well.

[admin@CCR1036] > /system resource print
uptime: 17w4d19h52m46s (163 days) on v6.7

We also have 2 1016-12g's in our core doing ospf and some BGP, these are great but I would have really liked at least one sfp port, we also have a ccr1009-8g-1s-1s+ for testing,

I hope MikroTik are working on the BGP single thread/core issue for v7, or maybe in v6 to give lower priority to the BGP process so it doesn't slow the router so much when taking in 450k+ routes.

So I can recommend the CCR for what you want, but I would test them to breaking point before you drop them into production as getting the setup/config right is key.
 
randvegeta
just joined
Topic Author
Posts: 8
Joined: Tue Jul 22, 2014 6:24 am

Re: BGP Router for Hosting Company

Tue Jul 22, 2014 7:37 pm

Thanks for the feedback.

What about the CCR1009-8G-1S-1S+? I know it has only 9 cores compared to the CCR1016, but do I really benefit from that when I'm only going to be able to use 1 core any way? I assume each core on these babies is the same on all models with the same clock speed?

The CCR1009-8G-1S-1S+ has availability in my area, and the price is reasonable. The 1016 has no stock which means I would need to go for the more expensive 1036 and I'm not sure I need all those cores.

I assume then I should have no issue with 2x ISP with full routing tables on a single CCR1009? Actually I will probably do more static routing any way as our routes have very different costs. Static routes will help us manage our costs better. Mix of BGP full routing tables with some static in there.

As I am looking for about 4 - 6 units, I was wondering about the RB1100 models too. Are they any good for BGP? What do I lose from these in terms of routing performance?

Thanks again.
 
randvegeta
just joined
Topic Author
Posts: 8
Joined: Tue Jul 22, 2014 6:24 am

Re: BGP Router for Hosting Company

Thu Jul 24, 2014 8:58 am

Thanks for the feedback.

What about the CCR1009-8G-1S-1S+? I know it has only 9 cores compared to the CCR1016, but do I really benefit from that when I'm only going to be able to use 1 core any way? I assume each core on these babies is the same on all models with the same clock speed?

The CCR1009-8G-1S-1S+ has availability in my area, and the price is reasonable. The 1016 has no stock which means I would need to go for the more expensive 1036 and I'm not sure I need all those cores.

I assume then I should have no issue with 2x ISP with full routing tables on a single CCR1009? Actually I will probably do more static routing any way as our routes have very different costs. Static routes will help us manage our costs better. Mix of BGP full routing tables with some static in there.

As I am looking for about 4 - 6 units, I was wondering about the RB1100 models too. Are they any good for BGP? What do I lose from these in terms of routing performance?

Thanks again.
 
AlexS
Member Candidate
Member Candidate
Posts: 259
Joined: Thu Oct 10, 2013 7:21 am

Re: BGP Router for Hosting Company

Thu Jul 24, 2014 10:19 am

Hi
I have 6 x CCR1036-8G-2S+ I run them at 3 site, in vrrp mode.. But no BGP, only OSPF.. like them except I had reproblems pushing a single tcp stream past 1G, some limit in the cpu / nic setup. I could easily push 9.8Gb's on UDP.

I have 4 x routeros x86, running as VM's, 2 with BGP taking full feeds from 5 different peers (I am not transit, end user). The load up took time as noted, but they have been running fine, I actually have them doing BFP -> OSPF as my asa5520 doesn't do BGP :( that works well as well.

My other 2 vm's do BGP but much smaller tables.

The throughput is very minimal 10-20Mbs. I have spiked to 100Mbs. I had tested up to 1Gb/s

Overall Nice device, cheap, i like the BGP filter setup. I haven't run into any long standing bugs. I had issue with LACP but fixed with 6.12.

Currently running 6.15 on 80% of routers.
 
randvegeta
just joined
Topic Author
Posts: 8
Joined: Tue Jul 22, 2014 6:24 am

Re: BGP Router for Hosting Company

Fri Jul 25, 2014 4:36 am

I think I we will settle on the following.

Just bought 4x Edgerouter Pro 8s instead of the CCR1009. Slightly cheaper but plenty powerful enough and supposed to be less buggy than RouterOS. So I won't be getting either CCR or RB1100 models. But I will be buying a couple of licenses for RouterOS and installing them on a couple of old x86 (Core2Duo) machines, loaded with extra network cards. Since my traffic requirements are not too high (few hundred meg at most), the x86 boxes will hopefully be able to handle the traffic, and with the 2GB RAM and 2.3Ghz Dual Core CPU, I expect the BGP table population will be able to perform faster.

Hopefully that will work. If not, will just buy some CCRs later.
 
raz
Member Candidate
Member Candidate
Posts: 102
Joined: Wed Dec 19, 2012 3:26 pm
Location: Austria

Re: BGP Router for Hosting Company

Fri Jul 25, 2014 1:38 pm

If your Edgerouter will hit by TCP SYN Flood, you can dig a hole und put it in. UBNT never responded to my Tickets in this Case. So im sold my Testunit.
 
randvegeta
just joined
Topic Author
Posts: 8
Joined: Tue Jul 22, 2014 6:24 am

Re: BGP Router for Hosting Company

Fri Jul 25, 2014 11:14 pm

The router it is replacing is a Juniper J2320. I suspect this performs no better than an Edgerouter, even under DDOS. So probably not too much to worry about. I imagine the pipe would be full before the router starts to keel over. Would it not?
 
joegoldman
Long time Member
Long time Member
Posts: 501
Joined: Mon May 27, 2013 2:05 am

Re: BGP Router for Hosting Company

Sat Jul 26, 2014 4:22 am

The router it is replacing is a Juniper J2320. I suspect this performs no better than an Edgerouter, even under DDOS. So probably not too much to worry about. I imagine the pipe would be full before the router starts to keel over. Would it not?
Thats the thing with DDoS its PPS that kills not bandwidth. 1gbit of 64byte packets is very stressful
 
randvegeta
just joined
Topic Author
Posts: 8
Joined: Tue Jul 22, 2014 6:24 am

Re: BGP Router for Hosting Company

Sat Jul 26, 2014 6:46 am

The J2320 can handle 150k pps or so. The ERPro8 can do over 2m PPS at 64bytes. My lines are all 100M or less. Am I mistaken in thinking that a 100M port would almost certainly never hit 2m pps at 64 bytes?


I did buy myself a CloudCoreSwitch too just to see how it feels. I assume the interface and usability is pretty much identical?
 
User avatar
wulfgard
Frequent Visitor
Frequent Visitor
Posts: 86
Joined: Wed Oct 17, 2012 1:06 pm
Location: France
Contact:

Re: BGP Router for Hosting Company

Tue Jul 29, 2014 6:51 pm

Hello All,

We are looking at replacing our Juniper J series router and were considering Mikrotik Cloud Core Router. I was hoping to get some feed back and real world experience.

- We are looking at using the router almost exclusively for BGP.
- We have 2 ISPs now, but we would like to have 3 - 4. Each will be 100M or less and have full routing tables.
- Reliability and Uptime is our greatest concern.
- We need multiple routers in a High Availability / Automatic Failover configuration

Our plan is to have 1 router for each ISP and working on a high availability basis so that a failed/crashed router can be pick up by another.

As I understand the BGP implementation has been a little buggy and I was wondering how this affects performance? Any real life examples? Since I don't mind buying/setting up multiple routers, any way to improve performance if it is an issue?

What's reliability like on the CCRs or a x86 RouterOS box? Anyone use it in a 'data center' deployment? Are these suitable/capable for DC deployments in order to replace J series Junipers?

Thanks in advance guys.
Hello

we used to have J6350 in our backbone
it was a pain because of the central processor that is too slow to handle correctly convergence for 4 full BGP tables

we have moved to ROS x86 with i7 processor and this makes the trick
on a i7 with 4Go memory it takes less than 1 minute to get 4 full table
on a i7 with 189 bgp peers ( peering point ) it takes less than 2 minutes to setup all

if you plan to setup 4 routers i would recommend to use 2 route reflectors ( can be x86 with quagga or bird )
and so lower the number of session per router

a+
Thierry
System and Network Engineer
Mikrotik Trainer - MTCNA MTCRE
Official French Mikrotik Distributor

Who is online

Users browsing this forum: No registered users and 8 guests