Mikrotik & BGP

RouterOS General
1

Hi all,

I’m in the mood to jump from the next bridge.

A few years ago I had a Linux BGP Router with 2 upstreams (full-feed) - nothing special… in total ~250.000 routes. Since I now have to work with a few hundred Mikrotik boards, I thought, it would be a nice idea, to change from static routing and RIP to BGP.
Yes, yes, yes - the idea was nice… but as far as I can see → uhmz… I’ve never seen a fork of freeware, which works so worse.

My BGP experience (so far) was: 2 upstreams - if one fails, the other take over in almost realtime (Debian GNU / Zebra). My experience with Mikrotik is: “hope, that nothing restarts… if it happens, you can go on an reboot the entire network”.

Example:
Router A - 10.0.1.1
Router B - 10.0.2.1
Router C - 10.0.3.1

Router A & B have an BGP connection (Router B is recieving from router xyz via BGP - Router C recieves routes vom 1234). Router A & C are no peers. The entire network has about 50 boards and 100 routes.
In my opinion - if the upstream of router B goes down (the peer “abc”) the router “b” would be down. So far, so right.
The funny thing: router “b” marked the routes as not available (in blue color) - what did router “a”? how funny… he hold the routes… I simply added a default route to router “c” and all was fine.

Hence - boys? whats the sence of BGP, if the partner keeps routes up, when the “gateway” is down? If a BGP session goes down, the routes need to be destroyed. And further more this information must go to all other peers.

Furthermore we also have another funny part in BGP:
Set up a BGP session and unplug the power. Then wait a minute und plug in the power - funny to see, that the session will not go up again (tested on a WRAP board with 2.8.x and and routerboard with 2.9.x - but also happens within routerboard with different versions).

Since I thought, that I’m unable to make such simple things, I mounted a CF card in my PC and take a look at the Image. Yes - fine… zebra.conf - so why it’s not possible to make a fine working image with this freeware? I know, that this software is working famous - without problems…

It would be nice, if anyone can explain me this.

Thanks in advance & best regards,
softdown1

p.s.: please rename zebra.conf to mikrotik-bgp.conf… i think, that the mainteners of zebra / quagga are not happy, that you are using this name and producing such a bullshi*** with their source (disregarding the fact, that - if you’re using GPL software - it would be nice, to mention those people).

2

we don’t use Zebra in later versions of RouterOS, please upgrade

3

Hi,

I’ve spent some time with BGP on MT. Some experiences which might help:

  1. Dont use Routerboard/WRAP for BGP if you do more than receiving a default route
    from your uplink. They are to slow to handle BGP. When your uplink goes up/down
    you’ve to wait some minutes until routing is stable again.
    So use faster hardware. Use IDE-Flash as Harddisks die more often.
    You need at least 256MB if you’ve full Internet Routingtables.

  2. Dont try to use winbox to show/enter routes.
    Use terminal and commands like this:
    /ip route print from= [/ip route find ospf=yes]
    /ip route print from= [/ip route find dst-address=0.0.0.0/0]
    Even this commands are frustrating if you’ve no fast hardware.

  3. Use 2.9.45/.46. Everything below is unstable. Above I’ve not tested.
    May work also. Use routing-test.

  4. Dont rely on routing protocols to find the way to your BGP Neighbor.
    This routes have to be static. This is a source of strange effects.
    In combination with a slow system you’re getting killed.

  5. Dont expect that every option has the effect you’re expecting.
    Especially if you’re used to cisco. E.g. the parameter Prepend works
    not as expected.


    If you’re thru the learing curve you’ve a stable system which is the
    same as your other routers. I’ve had a linux-quagga system before.
    quagga is faster, but has it’s drawbacks and flaws, too.
    And you’ve a lot of work to build a quagga system which fits
    on a IDE-Flash. And no winbox, and …

Stefan

4

Hi,

@normis

Thanks for your post. I’ve never read such a usefull posting. I try to read it, ok? → you’re not able to implement a working GPL software and we have have to upgrade.
As stated, we have a few hundred boards… time to cut “RouterOS” down to "OS?
If you - as admin - have nothing more to say than “upgrade” - thanks. I’ve tried a lot of boards & combinations… and nothing realy works (despite a simple debian with zebra / quagga).
Btw.: BGP is a protocol - it’s absolutly stable in zebra / quagga. I don’t understand, how you’re able, to damage it.

@Stefan

  1. I only (try) to use BGP for default routes and internel routes (~100 routes). As so far - nothing special.
  2. OSPF is not in use… and winbox also :wink:
  3. Everything below 2.9.45 is unstable? oh… sorry - I thaught, it’s a ROUTERos - time to flash openwrt or ddwrt?
    4 / 5. uhm… thats a good deal - I buy a “routeros” which isn’t able to make routing :frowning: - shit happens

Anyway - many thanks for your information!

Best regards,
softdown1

5

You’ve misunderstood some of my points:

  1. Only the BGP-Part is unstable below 2.9.45. Dont believe openwrt/ddwrt build stable BGP-Routers.

  2. I talked about the route to your bgp neighbor. Making this dependent on a routing
    protocol (ospf) makes problems as you may receive routes via another routingprotocol (bgp)
    with the same target but with better metric. So no bug but a likely config error.

Zebra/Quagga is stable if you’ve the right version. I’ve had problems with two quaggas
deadlocking themself under some conditions. Yes!! I’ve found my main and my backuprouter
dead (Perfect backup). I’ve had problems with quagga to recognize the state of
ethernet-interfaces correctly. Sometimes the quagga routing table and the
kernel-routing table was out of sync. I dont want to blame quagga as I have no
right to do so. But it’s not the solution I stay with.

So every solution has it’s drawback and the few bucks to spend for ROS are nothing
compared to cisco/juniper and others.

You use BGP for internal routes? It’s an external Rounting protocol. It has a slow
convergence. Better use OSPF.

Stefan

6

we use BGP across 8-10 routers all connected with l2tp. We have no problems like you specify, but we are also using > 2.9.45. We also use routing-test. Anything from 2.9.27 - .44 is not the best BGP version. We also put in place good routing filters that won’t cause loops or lockups when one router decides to advertise routes that conflict.

Also - something you might not have known … try to put all bgp peers in the same instance on each router, otherwise as-path / length won’t be used to determine the shortest hops. Each bgp instance is separate in the routing table and not used to compare which route wins.

7

Agree with changeip,
Version below 2.9.44 having so much problems with BGP, and OSPF also.
I tried the crappy version 2.9.27 and compare to 2.9.48, so different.
I got serious problems with OSPF, specially with the link cost.

:laughing: