I’m about to put a RB1000 into production shortly. I’ll probably be running ROS v3.21 (beta) on it, as it will be used as a HTTP proxy forwarder to a parent proxy, and I need to able to forward HTTPS that is fixed in v3.21.
This router will also be eBGP peering with two peers and receive a few hundred prefixes from each peer. One peering directly over an ethernet if and the other over an IPIP over IPsec tunnel.
This has worked flawlessly in my lab environment, but now that I’ll put this into production, I’d just like to ask the community, what is the recommended ROS version to run BGP on that is most stable?
Stock routing package or routing-test? Or perhaps a backlevel? If a backlevel is required to get a stable router, I’ll have to get two of them, one to do the routing and one to use as a proxy forwarder.
ROS 3.18 has MD5 issues with cisco and juniper router’s peerings
ROS 3.20 has some problems with routes that disappear when a new peer is established
ROS 3.20+ Routing Test looks good, i’m using it and works fine.. but sometimes it hangs for a while when acquiring the full internet routing table. And looks like it doesn’t use all the multi-cpu of a quad-core x86. Might work better in a RB1000
RB1000 has the same issue with the cli locking up when querying the route table. Ironically it always works fast when you query specific prefixes (eg. /ip route print where dst-address=n.n.n.n/xx).
It doesn’t seem to be loosing traffic when it locks up, so it’s not harmful in any way.
3.21 is not released yet. It’s only in beta. Search for “3.21” and “https” here in the forum and you will find links to the beta. But keep in mind that it is a beta and not for production use.
It would still be nice to know if it fixes any BGP bugs though.
As long I use ROS (since 2.8.x), I do not use its BGP implementation.
Our network is big (more than 800 private AS’es now) and reveals easily any problem or bug. Most stable is probably cisco and quagga’s stable versions.
Many people here around, say that mikrotik is not bad with BGP. We (in AWMN) have test it extremely, trying to avoid the use of quagga with extra router (which makes it more difficult to administer). Today I cannot trust mikrotik’s BGP implementation. I would not even try it.
I use mikrotik with success for, firewall, wireless, hotspots, vlan and special interface manipulation, but not for routing.
I’m evaluating platforms for bgp. We have been using zebra/quagga for many years to provide bgp multihoming. Also using ospfd.
I’m questioning it right now because of some problems I’m seeing. I’m interested in Medianet’s comments about quagga “losing routes”. I’m seeing some unreachable destinations (eg. ns1.google.com!) when I have 2 bgp peers installed, but if I remove either of the 2 bgp sessions then it works fine. Is this similar to the behaviour others have seen?
I was considering Mikrotik bgp, but maybe not now. I was kind of assuming that Mikrotik’s bgp is really just quagga in disguise or something like that.
Cisco would be an expensive experiment, so if I have other options then I’d rather explore them all first.
Mojiro: Have you reported your findings to MT? We use a rather small configuration of Mikrotiks BGP routing, and would love to utilize it more, but if indeed there are these bugs you describe, I would really appreciate if you took the time to report them to Mikrotik, so they can be fixed.
@scottt, Eising: don’t use it it has so many bugs and is so unflexible, i would not recommend it for anything more than two peers with just the smallest acls or filters you can think of as mojiro said, you can’t use it in anything bigger than that.
get a juniper 2320 if a cisco is too expensive for you. rockstable, all features of junos is also in this box and great for traffic <350mbit/s. and it costs around 1600 euro streetprice including two ge-interfaces. and you get loads of free support in irc or forums, as many people are actually using it with small and big bgp-setups.
i personally haven’t found that much bugs, simply because many things cannot be configured (as they are non-existant) and can’t generate errors because of that
but there were many times of session-disconnects, no communication after the hand-shake, no propagation of routes. most times a restart fixed everything, but in a production environment you cannot restart a core-router once a week just because it stopped working
i tried to debug this a few times with mt-support, and many others did the same as you can read in quite a few threads on this forum, but mostly the only solution was a restart or an update which created other problems not only in the bgp-stack.
so for me this experiment is done, i won’t try it until there have been some really really really promising case studies with traffic more than 300mbit/s and more than two peers. as openbgpd is a fine and stable software-router and juniper makes really very cheap routers there is no need to use mt desperately in every part of our network.
don’t get me wrong, i like the software and most of the routerboards are fine, but there are parts where mt is good at (wireless, pppoe) and there are parts where mt (in my opinion) should have left the market for another company, as a bad protocol-implementation is worse than no implementation, especially in a thing like bgp, which the big mass of their customers is not needing, but is a backbone technology, so they cannot depend on their customers to beta-test it as they do with other technologies.
i can test a stupid-l3-switch in any part of my network without any problems, but a bgp router can only be tested in a realistic environment with working peers and live traffic, so i need to replace a maybe working bgp-setup to test a new product. that will happen only rarely i think
What version did you test on ? IPv6 BGP was a problem in a few versions but seems to work now in my setup. I am using 3.17 and 3.19 now and there are some lingering bugs in 3.x BGP. BGP on 2.9.51 is very stable with 100k - 200k routes.
Can anyone really say that any other brand of BGP handles 500k+ routes without a problem? I don’t even think crisco handles it very well.
i talked to marlow yesterday (thats the guy who wrote this blog-entry) and he said, the issues he had are fixed in the last versions.
nonetheless he is not using it anymore, as there is too less support in the rest of the system (dual stack or single stack) for v6 and he also had stability/performance issues.
@changeip: junos is really good nowadays, i would presume it can handle this without any glitches.
i tested with some 3.x version, but cannot really remember which specific one, was somehting in the middle of 3.x
Probably right now I cannot remember all the problems we had in past with mikrotik’s BGP. In general, the problems had to do with the peers, not with the number of routes.
ex. One peer → no problems, Second peer → problems arise…
My router BGP v3.20 restart and freeze randomly from 2 till 8 days. Technical support could not help me. I switched off SNMP, changed all network cards on Intel-Pro, changed motherboards, memory and the processor - is useless. I exclude hardware incompatibility since the same equipment without problems works pptp-server with the big loading - more than 1000 simultaneous connections. Problems really do not exist at 1 BGP connection as soon as you connect the second - casual reboots and freezing. I very much would wish to hear people at which without problems works BGP FULL VIEW more than on 2 links.
3.20 stably works more month without reboots in quality pptp a server.
Tell please - v3.10 for you works stably and without problems BGP as a router with two and more full BGP links?
MultiCPU use?
it has so many bugs and is so unflexible, i would not recommend it for anything more than two peers with just the smallest acls or filters you can think of