Hi All
We have installed MT as our main router for using on multihomed connections with handoff from 2 providers. Both providers provide us with multiple PUBLIC class Cs.
We now want to do BGP failover so all our ips work if one provider goes down. We are in the US and I understand we need to get an AS # from Arin but the documentation MT is very little for BGP .
Is it as simple as configuring as per the document with multiple Class Cs from both providers.
Mikrotik is an excellent piece of software and works well for us far with 50mbps throughput on each provider (100mbps total)
I used routeros as a bgp border router 2 years ago. It served only on e prived, but serverd good. The config was as simple as its shown in the docs. It was 2.6.x, P3 1GHz, and 256MB ram.
Never dropped, it worked for 1 year, than I left that company, but as I know its still working with no errors.
Did you have 2 providers and IPs from both (multihomed)? How did they configure on their end. The documentation makes us put only 1 bgp peer how do you define multiple (because i am assuming each provider will be a peer)
anybody worked with mikrotik and multihoming bgp through 2 providers for seamless failover ?
I am sure people have used it for keeping it up and running.
Maybe I’m missing something, but we’re using it with dual providers and I simply created two peers on the Routing->BGP selection via winbox. We’re taking full tables from each peer, which does cause a bit of a problem when we loose one because removing the routes through the lost peer appears to take awhile, but it does eventually figure things out.
Make sure neither peer is sending you a default route unless you like seeing all your outbound traffic go through only one of them instead of through the interface that is the shortest path.
We have to MTs and want to do this eventually but to keep things simple its better to do it on one and then implement VRRP and check how things are working
Great Thanks for the info . I am going to try this out. I actually want to control the outbound for each network. BTW whats your router config cpu/memory etc. since your taking full tables from each peer ?
Thanks again . Merry Christmas
The router in question is running a 1.7Ghz CPU and has 512Meg of ram. It normally averages under 10% utilization and roughly 400Meg free, so the box is probably way more than is actually needed.
One note though, while this box does have firewall rules, the connection table tracking has been turned off. If that was turned on I’d expect CPU usage to be a lot higher.
I need to clarify this statement, I didn’t word it very well.
If you’re taking full tables from your peer, the default route is going to go unused for the most part because you’ve probably got a more specific route in your tables from a peer. Still, I wouldn’t have those peers send you a default, I’d set it locally as a static route if I felt the need for it to exist.
Another problem you could well run into is that a peer might send you routes that they’ve artificially weighted to favor them. What you’ll see is that all of your outbound traffic favors that peer unless the destination is so close that one of the other peers becomes a better path. That’s the case I’m fighting at the moment, I believe. Unfortunatly, MikroTik doesn’t appear to have a way to view the raw routing tables instead of just the current best route, and it certainly doesn’t have a way to alter the weights of incoming routes the way some other routers have. This leaves me with no way to verify my idea that one peer is messing with the weights of their routes, and no way to undo that tampering if I could verify it.
This means that even with multiple peers, our outbound traffic is almost exclusivly through just one peer. If that peer goes down, the traffic does flip to the secondary as it should, but only after the routes from the first peer have slowly gotten removed from the routing table.
this means your wasting bw on the other peer by not using it? . Are your peers actually weighting – are you really landing up using just one ? . It also means do you have to have static routing enabled and some sort of failover script changing the static routing where if you loose a connection you would want to change the static route which i think would be advertised by our bgp router to the internet ???.
Just wondering how this would be done . Also wondering are we the only 2 people here concerned with multihomed bgp for seamless failover?
The second peer is very underutilized, yes. My guess is that it’s the weighting causing it, and the fact I can’t verify that because MT provides no way to see the raw routing table is one of my complaints with them.
There is no static route to change, we use BGP to advertise our networks by placing them in the list of networks it knows about and turning off all the other forms of announcements it can make. When a peer goes down, those announcements stop and our traffic flows through the remaining peer as the original routes timeout on the network backbone.
There might not be a lot of people using BGP on their border router or subscribed to these forums to get a good sample, but I know I’ve ran into enough problems with it that I’d prefer I didn’t have to use it. The problems I have with trying to force the speed/duplex settings of an ethernet interface and the complete lack of error counters just make things worse on top of the limitations on how you can configure and monitor BGP.
One other thing you’re going to want to watch out for is accidently re-announcing routes from one peer through the other. When I added our networks to the bgp config to announce them, I quickly discovered that those were in addition to all the routes I was learning via bgp through each peer. I wasn’t a leaf node, I was an interchange point between the two peers.
My solution was to add an outbound prefix list to limit what I announced. I created a new prefix list with a default “reject” rule, added my networks and installed it. Despite that default “reject”, I was still announcing everything and I believe that’s a bug in the MT. To stop it from announcing anything except my specific networks I had to add an explicit “reject everthing else” as the last rule of the prefix list.
I’m setting up something very similar to this, but I’ve got one annoying problem I haven’t yet figured out.
I’m taking a full BGP feed (all 150,000 routes in the global table) from my upstream. (It was easy enough to set it up this way, as the router has plenty of RAM and CPU to spare.) When you do an /ip route print, or look at the routes list in Winbox, it shows you those routes. ALL of them. Takes Winbox about half an hour to load the routes window.
I’m using the same router to handle our internal stuff (about 50 static routes), and trying to find those few routes mixed up in the giant routing table is, um, a pain in the behind. (I just keep a printout of our static routes handy, just in case…)
Is there some hidden option to get RouterOS to not show me dynamic BGP routes? That would make my life so very much easier. (I could also do this by getting a second device that just speaks BGP, and creating a /30 between it and the internal router, but that’ll cost a few dollars that I’d rather not spend if I can avoid it.)
Nope, and that’s another problem report I sent in last summer. Even if you close that window in winbox, look at the yellow activity dot in the upper right corner, it’s still solid yellow. Closing the box early doesn’t stop the download process, it simply means it hasn’t got a place to be displayed.
If you open up a terminal window though and display the route table from there, you can page through it one page at a time.
Displaying routes in winbox REALLY needs a setup screen to select what you want to look at. When I have to do this on my setup, I get lucky in that the routes are loading from the high numbered IP range downward, so I’ve got a few minutes to edit my static routes and then exit winbox to stop the download from eating all the memory in my workstation.
If you’re having this problem, that implies you’ve used Mikrotik’s BGP implementation to handle a full feed in a production environment, and that it didn’t die. That’s reassuring, as today was the first time I fired it up.
As a fix for this “problem” doesn’t appear to be forthcoming, I’ll probably just get a second rackmount PC to do the job. (A RouterBoard with enough memory could probably handle it, but a RB230 plus enough memory to be sure it’ll work ends up costing about as much as a mid-range 1U, something in the 1.5GHz range. For the $100 difference, might as well get something that I know will work for the next few years.)
Is there any advantage of taking full feeds from both providers???
I have to decide whether my peers give me
Full routes will include the entire routing table minus the default.
Partial routes will include only localy originated routes
customer routes(no transit).
Default will be only 0.0.0.0/0
What should i choose
This is an option given by one of the peers . They also want us to join radb.net . Is it really so essential after paying arin to get the AS number ?
We have 1.1.1.x/24 +other /24s from one provider A [public ips]
and 2.2.2.x/24 from the other B [public ips]
I am also using policy routes at the moment to send specific 1.1.1.x ipaddress (not all of the range) data of A out through other B .
Wonder what will happen to this . BGP info is very scanty in MT
If you don’t have an AS number, you’re wasting time with BGP because the point of using it is to advertise your own routes. If your peers are doing the annoucing for you, then if you go down they still announce and traffic to you gets thrown away, you get blackholed on the network. If you announce, then when a peer goes down all the traffic starts coming in through the good peer because your announcements through the down one have stopped.
If you’re going to take a BGP feed, you might as well take the full table. Smaller tables are a hack to try and let people with old routers that don’t have enough memory to handle the full feed, still run BGP.
