BFD + OSPF + CCR1036 Issue

Hi

I have a few CCR1036-8G-2S+, (6.22) I tried turning on BFD with OSPF.
My first hurdle was BFD doesn’t work by itself !
I then associated it with a OSPF interface, which had 4 nodes on it. Turned it on all 4 nodes.. worked for a while then up / down flapping …
I ran some tests sending flood pings down the line, no missed packets !

My final test, 2 of these boxes side by side, 1 cross over cable. Turned on OSPF + BFD, started to fail :frowning:

I have a nice simple firewall rule
10 ;;; Allow BFD
chain=input action=accept protocol=udp dst-port=3784 log=no log-prefix=“”

I do have a drop invalid before it, but that should affect it and for a period of time BFD does come up.

I have it attached to one of my BGP peers on a ROS VM, it seems to work fine. So I’m leaning toward an issue with CCR’s

So anyone else using BFD with CCR’s and OSPF ?

I’m not going to lodge a support ticket till I upgrade the firmware :slight_smile:

I have the exact same issue with this model. Are you using the 1 Gbps copper or 10 Gbps SFP+ interfaces?

I currently have two CCR1036-8G-2S+ in a datacenter and saw this issue when turning BFD on a 20 Gig LACP channel. After some review of the logs and BFD neighbor states, I was able to determine that it drops right at 60 seconds every time. I haven’t seen this issue on other routerboards. They are both on 6.19 code, so I’m going to try and upgrade to see if the behavior changes.

Here is a copy of the BFD Neighbor output showing a drop right at 1 Minute

[admin@CCR-CORE-1] > routing bfd neighbor print detail 
Flags: U - up 
 0 U interface=Po1 address=172.31.31.1 protocols=ospf multihop=no state=up state-changes=2 uptime=55s 
     desired-tx-interval=0.9s actual-tx-interval=0.9s required-min-rx=0.9s remote-min-rx=0.9s 
     multiplier=5 hold-time=4.5s packets-rx=72 packets-tx=77 
[admin@CCR-CORE-1] > routing bfd neighbor print detail 
Flags: U - up 
 0 U interface=Po1 address=172.31.31.1 protocols=ospf multihop=no state=up state-changes=2 uptime=57s 
     desired-tx-interval=0.9s actual-tx-interval=0.9s required-min-rx=0.9s remote-min-rx=0.9s 
     multiplier=5 hold-time=4.5s packets-rx=74 packets-tx=79 
[admin@CCR-CORE-1] > routing bfd neighbor print detail 
Flags: U - up 
 0 U interface=Po1 address=172.31.31.1 protocols=ospf multihop=no state=up state-changes=2 uptime=1m 
     desired-tx-interval=0.9s actual-tx-interval=0.9s required-min-rx=0.9s remote-min-rx=0.9s 
     multiplier=5 hold-time=4.5s packets-rx=78 packets-tx=83 
[admin@CCR-CORE-1] > routing bfd neighbor print detail 
Flags: U - up 
[admin@CCR-CORE-1] > routing bfd neighbor print detail 
Flags: U - up 
 0   interface=Po1 address=172.31.31.1 protocols=ospf multihop=no state=down state-changes=0 
     desired-tx-interval=0.9s actual-tx-interval=1s required-min-rx=0.9s remote-min-rx=0s multiplier=5 
     hold-time=0s packets-rx=0 packets-tx=3 
[admin@CCR-CORE-1] >

My initial test was a 20Gb/s LACP Hash 2-3 with vlans.
Then i moved to 1Gb/s copper cross over cable

Good news I suppose that other poeple have the issue.

I’m on 6.22. was going to try an upgrade this weekend … so I can send in a support rif

So I have upgraded to 6.26 and firmware upgrade to 3.22

and now it seems to be stable !!!
I have a OSPF BFD link up for 5min… way longer than what I was able to get before

Hi

Spoke to soon, patched the rest of my ccr’s and tried to bring together 4 devices, falls over around 60-70 seconds

With this rule you get state ‘UP’ on both ends on Routing > BFD > Neighbors?

In my case I didn’t so I had to allow UDP 3784 & 3785 for it to work.

https://tools.ietf.org/html/rfc5881

BFD Control packets MUST be transmitted in UDP packets with
destination port 3784, within an IPv4 or IPv6 packet. The source
port MUST be in the range 49152 through 65535. The same UDP source
port number MUST be used for all BFD Control packets associated with
a particular session. The source port number SHOULD be unique among
all BFD sessions on the system.

BFD Echo packets MUST be transmitted in UDP packets with destination
UDP port 3785 in an IPv4 or IPv6 packet.

Hello

do not waste time on BFD
the issue is known and will be fixed in v7

this as been told to me by Mikrotik support in mai 2014

best regards
Thierry

What’s the issue exactly?

BFD is not working and session are lost after few seconds even with a timer at 1000 ms

So do not you BFD or you will get a lot of OSPF flaps

A+
Thierry

I’ve never had this problem. It may be specific to your configuration.

All my BFD peers are configured with the default values (which amount to hold-time: 1second)

Other than the fact that rarely it will drop all peers if one gets down, I don’t have the problem you describe.

Are you sure you configured it properly? Sound’s like a firewall to me since it will timeout after a few seconds.

Here’s a screenshot of a CCR using BGP, OSPF & BFD with uptimes over 8days.
Screenshot_1.png
So apparently it works. I am aware that there will be some fixes on v7 but they will be for specific problems (wrong src address, no multihop support, etc).
Generally BFD works relatively fine if you don’t mess with it often.

BFD is working for BGP not for OSPF

just setup BFD on OSPF P2P link and you will see that it colapse after a few seconds

cheers
Thierry

Did you see the screenshot I posted?

There are 2 OSPF peers (one IPv4 one IPv6) with 8-9days uptime.
And yes bonding1-ibgp is a p2p link between the 2 CCRs running iBGP with OSPF.

BFD + OSPF is stable for us between 9 peers (all CCR)

So you are lucky guys but watch up ospf flapping !!!

Below Mikrotik answer from my original ticket

Hello,

Thank you for the report. BFD issues will be fixed in ROS v7
Regards,
Maris

05/03/2014 16:28 - Thierry wrote:

Hello

we have detected issue between BFD and OSPF

if your are using BFD on Point to Point OSPF link become unstable because BFD is
detecting loss even with high timer values ( link 1 seconde )
this does not occure if you are using NBMA configuration for OSPF

Thierry

Cheers
Thierry

Anyone figured out how to work around this problem? We are trying to build a mpls network and some links are not stable 100% so we need a fast failover mechanism for ospf and bfd is the only solution. I dont want to set the regular ospf timers lower.

Regards,
Joerg

Are the links stable when you don’t use BFD?

Yes, they are stable. I also see this problem when i connect two CCR’s with a direct cable.

Offical response from MT is: “do not use BFD, wait for RouterOS 7 where it is planned to be fixed” :frowning:

But this is no solution for us

Not sure what your convergence requirements are, but I regularly use 1 second hello and 3 second dead timer on OSPF for MPLS based networks and adjust LDP timers accordingly (usually close to or slightly above OSPF) and that seems to provide very rapid and stable convergence without BFD. May not meet your requirements, but it is much fast than going with default timers.

just as a “fun fact” note: RouterOS v6.36 on a CCR1036 speaking bfd on a single 1G copper link (OSPF) with a Cisco (IOS 15.2(4)S7) does seem to carry a (at least for now) stable BFD interface (29 minutes and counting)

@hknet

It seemed stable for us too, but as our network grew we experienced a lot of flapping.

It is best to avoid BFD until v7 is released.