Community discussions

 
AlexS
Member Candidate
Member Candidate
Topic Author
Posts: 258
Joined: Thu Oct 10, 2013 7:21 am

BFD + OSPF + CCR1036 Issue

Fri Feb 06, 2015 3:46 am

Hi

I have a few CCR1036-8G-2S+, (6.22) I tried turning on BFD with OSPF.
My first hurdle was BFD doesn't work by itself !
I then associated it with a OSPF interface, which had 4 nodes on it. Turned it on all 4 nodes.. worked for a while then up / down flapping ...
I ran some tests sending flood pings down the line, no missed packets !

My final test, 2 of these boxes side by side, 1 cross over cable. Turned on OSPF + BFD, started to fail :(

I have a nice simple firewall rule
10 ;;; Allow BFD
chain=input action=accept protocol=udp dst-port=3784 log=no log-prefix=""

I do have a drop invalid before it, but that should affect it and for a period of time BFD does come up.

I have it attached to one of my BGP peers on a ROS VM, it seems to work fine. So I'm leaning toward an issue with CCR's

So anyone else using BFD with CCR's and OSPF ?

I'm not going to lodge a support ticket till I upgrade the firmware :)
 
User avatar
IPANetEngineer
Trainer
Trainer
Posts: 1053
Joined: Fri Aug 10, 2012 6:46 am
Location: Jackson, MS, USA
Contact:

Re: BFD + OSPF + CCR1036 Issue

Fri Feb 06, 2015 5:46 pm

I have the exact same issue with this model. Are you using the 1 Gbps copper or 10 Gbps SFP+ interfaces?

I currently have two CCR1036-8G-2S+ in a datacenter and saw this issue when turning BFD on a 20 Gig LACP channel. After some review of the logs and BFD neighbor states, I was able to determine that it drops right at 60 seconds every time. I haven't seen this issue on other routerboards. They are both on 6.19 code, so I'm going to try and upgrade to see if the behavior changes.

Here is a copy of the BFD Neighbor output showing a drop right at 1 Minute
[admin@CCR-CORE-1] > routing bfd neighbor print detail 
Flags: U - up 
 0 U interface=Po1 address=172.31.31.1 protocols=ospf multihop=no state=up state-changes=2 uptime=55s 
     desired-tx-interval=0.9s actual-tx-interval=0.9s required-min-rx=0.9s remote-min-rx=0.9s 
     multiplier=5 hold-time=4.5s packets-rx=72 packets-tx=77 
[admin@CCR-CORE-1] > routing bfd neighbor print detail 
Flags: U - up 
 0 U interface=Po1 address=172.31.31.1 protocols=ospf multihop=no state=up state-changes=2 uptime=57s 
     desired-tx-interval=0.9s actual-tx-interval=0.9s required-min-rx=0.9s remote-min-rx=0.9s 
     multiplier=5 hold-time=4.5s packets-rx=74 packets-tx=79 
[admin@CCR-CORE-1] > routing bfd neighbor print detail 
Flags: U - up 
 0 U interface=Po1 address=172.31.31.1 protocols=ospf multihop=no state=up state-changes=2 uptime=1m 
     desired-tx-interval=0.9s actual-tx-interval=0.9s required-min-rx=0.9s remote-min-rx=0.9s 
     multiplier=5 hold-time=4.5s packets-rx=78 packets-tx=83 
[admin@CCR-CORE-1] > routing bfd neighbor print detail 
Flags: U - up 
[admin@CCR-CORE-1] > routing bfd neighbor print detail 
Flags: U - up 
 0   interface=Po1 address=172.31.31.1 protocols=ospf multihop=no state=down state-changes=0 
     desired-tx-interval=0.9s actual-tx-interval=1s required-min-rx=0.9s remote-min-rx=0s multiplier=5 
     hold-time=0s packets-rx=0 packets-tx=3 
[admin@CCR-CORE-1] > 
Global - MikroTik Support & Consulting - English | Francais | Español | Portuguese +1 855-645-7684
https://iparchitechs.com/services/mikro ... l-support/ mikrotiksupport@iparchitechs.com
 
AlexS
Member Candidate
Member Candidate
Topic Author
Posts: 258
Joined: Thu Oct 10, 2013 7:21 am

Re: BFD + OSPF + CCR1036 Issue

Fri Feb 06, 2015 9:14 pm

My initial test was a 20Gb/s LACP Hash 2-3 with vlans.
Then i moved to 1Gb/s copper cross over cable

Good news I suppose that other poeple have the issue.

I'm on 6.22. was going to try an upgrade this weekend ... so I can send in a support rif
 
AlexS
Member Candidate
Member Candidate
Topic Author
Posts: 258
Joined: Thu Oct 10, 2013 7:21 am

Re: BFD + OSPF + CCR1036 Issue

Fri Feb 06, 2015 10:43 pm

So I have upgraded to 6.26 and firmware upgrade to 3.22

and now it seems to be stable !!!
I have a OSPF BFD link up for 5min... way longer than what I was able to get before
 
AlexS
Member Candidate
Member Candidate
Topic Author
Posts: 258
Joined: Thu Oct 10, 2013 7:21 am

Re: BFD + OSPF + CCR1036 Issue

Wed Feb 18, 2015 9:57 pm

Hi

Spoke to soon, patched the rest of my ccr's and tried to bring together 4 devices, falls over around 60-70 seconds
 
User avatar
Cha0s
Forum Veteran
Forum Veteran
Posts: 899
Joined: Tue Oct 11, 2005 4:53 pm

Re: BFD + OSPF + CCR1036 Issue

Sun Aug 02, 2015 3:37 pm

10 ;;; Allow BFD
chain=input action=accept protocol=udp dst-port=3784 log=no log-prefix=""
With this rule you get state 'UP' on both ends on Routing > BFD > Neighbors?

In my case I didn't so I had to allow UDP 3784 & 3785 for it to work.

https://tools.ietf.org/html/rfc5881
BFD Control packets MUST be transmitted in UDP packets with
destination port 3784, within an IPv4 or IPv6 packet. The source
port MUST be in the range 49152 through 65535. The same UDP source
port number MUST be used for all BFD Control packets associated with
a particular session. The source port number SHOULD be unique among
all BFD sessions on the system.

.....

BFD Echo packets MUST be transmitted in UDP packets with destination
UDP port 3785 in an IPv4 or IPv6 packet.
 
User avatar
wulfgard
Frequent Visitor
Frequent Visitor
Posts: 86
Joined: Wed Oct 17, 2012 1:06 pm
Location: France
Contact:

Re: BFD + OSPF + CCR1036 Issue

Sun Aug 02, 2015 4:41 pm

Hello

do not waste time on BFD
the issue is known and will be fixed in v7

this as been told to me by Mikrotik support in mai 2014

best regards
Thierry
System and Network Engineer
Mikrotik Trainer - MTCNA MTCRE
Official French Mikrotik Distributor
 
User avatar
Cha0s
Forum Veteran
Forum Veteran
Posts: 899
Joined: Tue Oct 11, 2005 4:53 pm

Re: BFD + OSPF + CCR1036 Issue

Sun Aug 02, 2015 4:47 pm

the issue is known and will be fixed in v7
What's the issue exactly?
 
User avatar
wulfgard
Frequent Visitor
Frequent Visitor
Posts: 86
Joined: Wed Oct 17, 2012 1:06 pm
Location: France
Contact:

Re: BFD + OSPF + CCR1036 Issue

Sun Aug 02, 2015 5:00 pm

BFD is not working and session are lost after few seconds even with a timer at 1000 ms

So do not you BFD or you will get a lot of OSPF flaps

A+
Thierry
System and Network Engineer
Mikrotik Trainer - MTCNA MTCRE
Official French Mikrotik Distributor
 
User avatar
Cha0s
Forum Veteran
Forum Veteran
Posts: 899
Joined: Tue Oct 11, 2005 4:53 pm

Re: BFD + OSPF + CCR1036 Issue

Sun Aug 02, 2015 6:25 pm

BFD is not working and session are lost after few seconds even with a timer at 1000 ms

So do not you BFD or you will get a lot of OSPF flaps

A+
Thierry
I've never had this problem. It may be specific to your configuration.

All my BFD peers are configured with the default values (which amount to hold-time: 1second)

Other than the fact that rarely it will drop all peers if one gets down, I don't have the problem you describe.

Are you sure you configured it properly? Sound's like a firewall to me since it will timeout after a few seconds.

Here's a screenshot of a CCR using BGP, OSPF & BFD with uptimes over 8days.
Screenshot_1.png
So apparently it works. I am aware that there will be some fixes on v7 but they will be for specific problems (wrong src address, no multihop support, etc).
Generally BFD works relatively fine if you don't mess with it often.
You do not have the required permissions to view the files attached to this post.
 
User avatar
wulfgard
Frequent Visitor
Frequent Visitor
Posts: 86
Joined: Wed Oct 17, 2012 1:06 pm
Location: France
Contact:

Re: BFD + OSPF + CCR1036 Issue

Wed Aug 05, 2015 12:46 am

BFD is working for BGP not for OSPF

just setup BFD on OSPF P2P link and you will see that it colapse after a few seconds

cheers
Thierry
System and Network Engineer
Mikrotik Trainer - MTCNA MTCRE
Official French Mikrotik Distributor
 
User avatar
Cha0s
Forum Veteran
Forum Veteran
Posts: 899
Joined: Tue Oct 11, 2005 4:53 pm

Re: BFD + OSPF + CCR1036 Issue

Wed Aug 05, 2015 3:23 am

BFD is working for BGP not for OSPF

just setup BFD on OSPF P2P link and you will see that it colapse after a few seconds

cheers
Thierry
Did you see the screenshot I posted?

There are 2 OSPF peers (one IPv4 one IPv6) with 8-9days uptime.
And yes bonding1-ibgp is a p2p link between the 2 CCRs running iBGP with OSPF.
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 1818
Joined: Mon Jan 14, 2008 1:53 pm
Location: Straya
Contact:

Re: BFD + OSPF + CCR1036 Issue

Wed Aug 05, 2015 1:48 pm

BFD + OSPF is stable for us between 9 peers (all CCR)
http://thebrotherswisp.com/ | Mikrotik MTCNA, MTCRE, MTCINE | Fortinet FTCNA, FCNSP, FCT | Extreme Networks ENA
 
User avatar
wulfgard
Frequent Visitor
Frequent Visitor
Posts: 86
Joined: Wed Oct 17, 2012 1:06 pm
Location: France
Contact:

Re: BFD + OSPF + CCR1036 Issue

Fri Aug 28, 2015 2:03 pm

So you are lucky guys but watch up ospf flapping !!!

Below Mikrotik answer from my original ticket
-----------------------------------------------------------------------------------------------------------------------
Hello,

Thank you for the report. BFD issues will be fixed in ROS v7
Regards,
Maris

05/03/2014 16:28 - Thierry wrote:

> Hello
>
> we have detected issue between BFD and OSPF
>
> if your are using BFD on Point to Point OSPF link become unstable because BFD is
> detecting loss even with high timer values ( link 1 seconde )
> this does not occure if you are using NBMA configuration for OSPF
>
> Thierry

Cheers
Thierry
System and Network Engineer
Mikrotik Trainer - MTCNA MTCRE
Official French Mikrotik Distributor
 
brainy
Member Candidate
Member Candidate
Posts: 149
Joined: Fri Sep 29, 2006 3:08 pm
Location: Unterschleissheim, Germany
Contact:

Re: BFD + OSPF + CCR1036 Issue

Mon Sep 07, 2015 9:33 pm

Anyone figured out how to work around this problem? We are trying to build a mpls network and some links are not stable 100% so we need a fast failover mechanism for ospf and bfd is the only solution. I dont want to set the regular ospf timers lower.

Regards,
Joerg
 
User avatar
IPANetEngineer
Trainer
Trainer
Posts: 1053
Joined: Fri Aug 10, 2012 6:46 am
Location: Jackson, MS, USA
Contact:

Re: BFD + OSPF + CCR1036 Issue

Tue Sep 08, 2015 4:01 pm

Are the links stable when you don't use BFD?
Global - MikroTik Support & Consulting - English | Francais | Español | Portuguese +1 855-645-7684
https://iparchitechs.com/services/mikro ... l-support/ mikrotiksupport@iparchitechs.com
 
brainy
Member Candidate
Member Candidate
Posts: 149
Joined: Fri Sep 29, 2006 3:08 pm
Location: Unterschleissheim, Germany
Contact:

Re: BFD + OSPF + CCR1036 Issue

Tue Sep 08, 2015 4:04 pm

Yes, they are stable. I also see this problem when i connect two CCR's with a direct cable.

Offical response from MT is: "do not use BFD, wait for RouterOS 7 where it is planned to be fixed" :(

But this is no solution for us
 
User avatar
IPANetEngineer
Trainer
Trainer
Posts: 1053
Joined: Fri Aug 10, 2012 6:46 am
Location: Jackson, MS, USA
Contact:

Re: BFD + OSPF + CCR1036 Issue

Tue Sep 08, 2015 6:24 pm

Not sure what your convergence requirements are, but I regularly use 1 second hello and 3 second dead timer on OSPF for MPLS based networks and adjust LDP timers accordingly (usually close to or slightly above OSPF) and that seems to provide very rapid and stable convergence without BFD. May not meet your requirements, but it is much fast than going with default timers.
Global - MikroTik Support & Consulting - English | Francais | Español | Portuguese +1 855-645-7684
https://iparchitechs.com/services/mikro ... l-support/ mikrotiksupport@iparchitechs.com
 
User avatar
hknet
Frequent Visitor
Frequent Visitor
Posts: 88
Joined: Sun Jul 17, 2016 6:05 pm
Location: Vienna, Austria
Contact:

Re: BFD + OSPF + CCR1036 Issue

Tue Aug 09, 2016 1:14 am

just as a "fun fact" note: RouterOS v6.36 on a CCR1036 speaking bfd on a single 1G copper link (OSPF) with a Cisco (IOS 15.2(4)S7) does seem to carry a (at least for now) stable BFD interface (29 minutes and counting)
 
User avatar
nz_monkey
Forum Guru
Forum Guru
Posts: 1818
Joined: Mon Jan 14, 2008 1:53 pm
Location: Straya
Contact:

Re: BFD + OSPF + CCR1036 Issue

Tue Aug 09, 2016 1:18 am

@hknet

It seemed stable for us too, but as our network grew we experienced a lot of flapping.

It is best to avoid BFD until v7 is released.
http://thebrotherswisp.com/ | Mikrotik MTCNA, MTCRE, MTCINE | Fortinet FTCNA, FCNSP, FCT | Extreme Networks ENA
 
User avatar
hknet
Frequent Visitor
Frequent Visitor
Posts: 88
Joined: Sun Jul 17, 2016 6:05 pm
Location: Vienna, Austria
Contact:

Re: BFD + OSPF + CCR1036 Issue

Tue Aug 09, 2016 11:34 pm

It is best to avoid BFD until v7 is released.
well, removed it for the time being, as I like stable :)
might try to run it in the future, if MT care to comment on the issue.
 
lrl
newbie
Posts: 25
Joined: Sat Sep 28, 2013 10:31 am

Re: BFD + OSPF + CCR1036 Issue

Thu Sep 08, 2016 7:43 am

I played with BFD on OSPF links for quite a while. The max stable I could get was about 8 days, but that was two routers and only one ospf link on each. As OSPF links grew the BFD went flaky quicker until we were down to mere hours. A reboot of the affected router always resolved the issue temporary. As above we just use really aggressive timers.
 
ppwicho
newbie
Posts: 32
Joined: Fri Nov 23, 2012 11:45 pm

Re: BFD + OSPF + CCR1036 Issue

Fri Nov 04, 2016 1:57 am

BFD and CCR are something we are expecting since a lot of time ago.

We are waiting for v7 since our network grow so much with multiple paths, three BGP full tables with route reflectors and moving more than 2 Gbps in the backbone. RouterBoards can't handle changes quickly when something just go broke in peak hours!.

@Mikrotik please give more attention to CCR family!.
 
krdx
just joined
Posts: 11
Joined: Wed Jul 15, 2015 1:22 pm

Re: BFD + OSPF + CCR1036 Issue

Sun Nov 27, 2016 5:38 pm

From the topic I take it that there are problems with BFD with OSPF on CCR platform.

However, I'd like to ask those using BFD as I might soon need to use BFD with BGP because of a peering requirement:

- Does BFD work with BGP on CCR Platform?
- Does BFD work with BGP on X86 RouterOS?
- Does BFD work with OSPF on X86 RouterOS?

What are your experiences?
 
rclewis
just joined
Posts: 4
Joined: Wed Jun 29, 2011 8:45 pm

Re: BFD + OSPF + CCR1036 Issue

Thu Dec 29, 2016 10:37 pm

BFD on BGP is stable and works for us. (Been using it since early 6.x, currently on 6.36.4)
BFD on OSPF was not stable for us at the time of testing ( a few years back).
Saw many posts of others having problems so I just disabled it and dealt with it.

Who is online

Users browsing this forum: No registered users and 6 guests