Community discussions

MikroTik App
 
User avatar
apitsos
newbie
Topic Author
Posts: 35
Joined: Tue Feb 22, 2022 9:36 pm
Location: Bielefeld, Germany
Contact:

Which MikroTik product is better for BGP with two redundant 1-gig fiber optic feeds

Tue Dec 06, 2022 1:48 am

Hi there,

I currently have a MikroTik CCR1009-7G-1C-1S+ supporting our services and our equipment in Hetzner's datacenter, where we have a 1/3 rack, 14U. We are currently having a 1 gigabit connection from Hetzner (copper/ethernet) and our IPv4 and IPv6 subnets (we are RIPE members) are announced by Hetzner. Because of that, the IPs are shown with their AS number and they are getting blacklisted every time Hetzner's ASN is blacklisted by "UCE Protect". This has happened already 3-4 times in less than a year and that shitty situation affects our services, even if our IPs are crystal clear and our customers are not spamming.

In order to avoid that, Hetzner has suggested us to upgrade our connection with a redundant Internet upstream with two 1Gbit/s fiber optic connection and personal BGP Session. In order to do that, we will have to upgrade our rack, because it's full already and there is no space for the fiber optics patch panel, but we will also most probably need a second router as well. I am not sure if the current router (MikroTik CCR1009-7G-1C-1S+) is good enough for that and if it would be okay to buy a second one same model or should I upgrade to something better, like MikroTik CCR2116-12G-4S+.

The subnets we own and which we would like to announce ourselves are a /24 IPv4 and a /29 IPv6. The real traffic is usually no more than 50-100 Mbps. There may be some picks with higher bandwidth, but these are just for a few moments of a few seconds.

In truth I have several questions about all this. Here are the most important that comes in my mind:
  • Is the old router (CCR1009-7G-1C-1S+) good enough for my needs?
  • Should the two routers be the same model or could I keep the old one as secondary and just buy a new one as primary selecting a better and newer model, like CCR2116-12G-4S+?
  • Will they operate on a failover scenario or will they operate like load balancing?
  • How could I configure something like that (redundancy) for this purpose of use? Are there instructions, tutorial or videos that you would suggest me?
  • How difficult is to configure that BGP and make the announcement yourself? Are there instructions, tutorial or videos that you would suggest me?
I would like have the experts' opinions here...

Thank you so much in advance for your support and your answers, which are always very much appreciated.


With kind regards,
Angelos Pitsos
 
eduplant
Member Candidate
Member Candidate
Posts: 139
Joined: Tue Dec 19, 2017 9:45 am

Re: Which MikroTik product is better for BGP with two redundant 1-gig fiber optic feeds

Mon Dec 12, 2022 5:27 am

Rather than reinvent the wheel, I usually point folks to the NSRC's intro primers because BGP is a very big topic:

https://learn.nsrc.org/bgp/intro_bgp_best_practices

A lot of these examples tend to be written in Cisco-ese, but you can always cross reference with the Mikrotik wiki [1][2] for the similarities. It's also worth making sure you review the differences between ROS6 and 7, since BGP got a major overhaul in 7.

Second question: since you're trying to make sure your address space shows up independently from Hetzner's AS, you do have a public AS number assigned to you, correct? It's possible to do BGP with private AS numbers but they will be stripped from the path and it will still look like Hetzner's AS is the originator.

Regarding your questions, I'll give each a shot:
1. Your CCR1009 should be plenty for this use case so long as it meets all of your other needs (total bandwidth, hardware redundancy, etc.).
2. If your business needs are being met by a single router for this deployment, there's no real need to change this just because you're gaining an extra link and configuring BGP. Obviously if you are using this as a reason to improve your hardware redundancy, there's no reason you couldn't move to two routers, but it will change the design a little bit. There is no strict requirement that the routers are the same model or same capabilities.
3, 4. From what you've explained (specifically that you will have two uplinks and they are both going to the same upstream provider), the simplest situation is to accept default from your upstream on both links and announce your prefixes on both links. BGP's best-path selection [1][2] in practice will have an effect similar to a failover (in that if one peer link goes away, you will still use the others) but it is not really the same. The difference is that unlike OSPF or other IGPs in which they will select a bidirectional best path through the specific link topology, in BGP you really only have total control over the direction your outbound traffic goes. This means that your outbound best-path may be one link and your upstream ISP may choose a different link back to you.

Asymmetric routing like this is usually acceptable, especially if the two paths into your network are basically the same bandwidth and latency like you're likely to have here. If you really need inbound and outbound to match, then you can have a conversation with your upstream ISP about their policies regarding AS-path prepending, MED, and communities that trigger localpref on their side. These tools allow you to influence the choice the upstream network makes about which peer to send traffic to. The most likely circumstance I can see where you would need to do that is if you were running a stateful firewall that would get confused by the asymmetric flows.

5. The configuration won't be super complicated to do the basics like this, but I do recommend leafing through the NSRC's videos/slides to get a better sense of how BGP operates before trying a configuration. I'd also recommend lab-ing out the topology with some Mikrotik CHRs or spare hardware so that you can be confident of how it's going to behave before changing your uplinks.

When you have a skeleton config worked out from your testing, glad to take a look at things if they're not working and try and help.

[1] https://wiki.mikrotik.com/wiki/Manual:Routing/BGP
[2] https://help.mikrotik.com/docs/display/ROS/BGP
 
User avatar
apitsos
newbie
Topic Author
Posts: 35
Joined: Tue Feb 22, 2022 9:36 pm
Location: Bielefeld, Germany
Contact:

Re: Which MikroTik product is better for BGP with two redundant 1-gig fiber optic feeds

Mon Dec 12, 2022 4:03 pm

Hi eduplant!

Thanks a lot for all these information. I will study them soon and I will come back.

Wishing you a great day!


With regards,
Angelos Pitsos
 
User avatar
apitsos
newbie
Topic Author
Posts: 35
Joined: Tue Feb 22, 2022 9:36 pm
Location: Bielefeld, Germany
Contact:

Re: Which MikroTik product is better for BGP with two redundant 1-gig fiber optic feeds

Wed Dec 14, 2022 3:49 pm

...When you have a skeleton config worked out from your testing, glad to take a look at things if they're not working and try and help.
Hi Eduplant,

How could I contact you? I would be interested on having a business-level collaboration with you, in order to make this project a reality. Would you undertake it? Please send me a pm or contact me directly.

Thanks a lot in advance for your prompt attention!


With kind regards,
Angelos Pitsos
 
User avatar
StubArea51
Trainer
Trainer
Posts: 1739
Joined: Fri Aug 10, 2012 6:46 am
Location: stubarea51.net
Contact:

Re: Which MikroTik product is better for BGP with two redundant 1-gig fiber optic feeds

Wed Dec 14, 2022 6:40 pm

CCR1009 is fine if you're not doing full tables.

This really comes down to stability vs. features. ROSv7 has come a long way and we have clients using it in prod, but there is still work being done on BGP, MPLS, hw offload, etc. Unless you keep up with the changes in ROSv7 beta and understand what the new CCR2K platforms do well and where they still need work, then you'll have a better experience on CCR1K routers and ROSv6 for the moment.

Given the rapid pace MikroTik is developing at, I expect we'll see a long-term version (most stable) in Q1/Q2 of 2023.

If your business requirements are simply failover and stability, i'd stay with the CCR1009 for now and build it in a topology like this:

Image
 
eduplant
Member Candidate
Member Candidate
Posts: 139
Joined: Tue Dec 19, 2017 9:45 am

Re: Which MikroTik product is better for BGP with two redundant 1-gig fiber optic feeds

Wed Dec 14, 2022 6:52 pm

@Angelos, thanks for the consideration but unfortunately I’m not looking to take on any contract work at the moment. My involvement in the Mikrotik forums and use of the platform is mostly for fun and to enable my side projects. My day job is on networks with bigger scope and bigger boxes.

Fortunately, IPANetEngineer had a design on hand that describes a very sane version of what the canonical design for this would look like with two routers. I’m sure there are also a number of Mikrotik certified folks around here who would be willing to help you get this built in a billable capacity.

The good news is that myself and others still are likely to pop in and help if you get stuck and post about it. The market value of those opinions is about what you pay for them ($0) but the practical value is a pretty good deal. :D
 
User avatar
StubArea51
Trainer
Trainer
Posts: 1739
Joined: Fri Aug 10, 2012 6:46 am
Location: stubarea51.net
Contact:

Re: Which MikroTik product is better for BGP with two redundant 1-gig fiber optic feeds

Wed Dec 14, 2022 7:33 pm

I would be interested on having a business-level collaboration with you, in order to make this project a reality. Would you undertake it?
Since Eduplant mentioned not being interested in contract work, feel free to reach out to IP ArchiTechs if you want professional help. We have engineers in Europe and the US

MikroTik consulting
https://iparchitechs.com/ecosystem/mikr ... onsulting/

Contact info
https://iparchitechs.com/contact-ip-architechs/

Who is online

Users browsing this forum: evellin and 19 guests