Community discussions

MikroTik App
 
User avatar
homerwsmith
Member Candidate
Member Candidate
Topic Author
Posts: 166
Joined: Fri Dec 02, 2011 3:01 am
Location: Ithaca, NY
Contact:

ospf nbma

Tue Jul 13, 2021 6:16 am

Dear Gentle Folk,

I have a baseball diamond network, with 5 miks one at each base and one at pitchers mound.

These are ubiquiti wireless links, mostly Rockets from base to base with miks at either end.

Sometimes the OSPF goes very unstable on one link , and I wanted to see if declaring that link an NBMA would stop
it from flapping.

I don't understand the relationship between the nbma links and the normal neighbors. If I want to declare a link between A and B to be
a NBMA link do I have to remove the normal neighbor link or can I leave it in and let them both work together. The NBMA link is prio 1.

How do I view the status of the NBMA links, routing ospf nbma print just shows me the existence of the link, not the
activity that has transpired across it.

Thanks in advance. Homer W Smith lightlink.com
\
 
User avatar
StubArea51
Trainer
Trainer
Posts: 1739
Joined: Fri Aug 10, 2012 6:46 am
Location: stubarea51.net
Contact:

Re: ospf nbma

Tue Jul 13, 2021 4:46 pm

NBMA on a WISP PTP link only masks an underlying problem with either:

1. The backhaul itself dropping traffic or a misconfiguration or issue with OSPF multicast streams
2. Traffic congestion tearing down OSPF neighbor adjacencies without QoS for the control plane
3. Misconfiguration of OSPF like a duplicate subnet, loopback, router id, etc
4. OSPF/BFD timers that are too aggressive
4. RouterOS versions mismatched and not using the same long term version

On the vast majority of WISP clients we design and configure OSPF on with MikroTik, using the network type PTP (which is not NBMA) works just fine. When we see OSPF flapping, it's almost always an underlying issue that needs to be fixed.

Here is an OSPF troubleshooting presentation I did at the Baltimore US MUM in 2018:

https://mum.mikrotik.com/presentations/ ... 892575.pdf

https://www.youtube.com/watch?v=WCbpUMMmR6E
 
User avatar
homerwsmith
Member Candidate
Member Candidate
Topic Author
Posts: 166
Joined: Fri Dec 02, 2011 3:01 am
Location: Ithaca, NY
Contact:

Re: ospf nbma

Wed Jul 14, 2021 3:21 am

Thank you for your beautiful work, I read the entire thing, and vereified all he errors you mentioned none apply.

All of our OSPF links tend to drop and restart randomly and periodically but not important enough to notice, except two bad boy,
the one from home to first base and from first base to second.

During heavy rain or thunderstorms it is most noticeable, but it is happening pretty constantly every few minutes to an hour or two or more.
Thus we see by the adjancencies that its been stable for 6 to 10 minutes say, then it resets.

First it goes down only for a moment, but it takes a minute or two to come back and traffic flowing again.

It would be nice if this stopped happening as customers notice it during zooms and movies, and i would be nice that when it does happen the routers wouldn't lose the entire database so traffic would continue to flow even though the adjacency was recouping, or is that totally meaningless?

From what I have read NBMA was designed to deal with WISP links with packet loss etc, so it would be nice to know how to set it up
in order test it.

Is there a whitepaper or web page detailing this configuration? The mikrotik manual is lacking...

Thanks Homer W Smith, CEO Lightlink
 
User avatar
homerwsmith
Member Candidate
Member Candidate
Topic Author
Posts: 166
Joined: Fri Dec 02, 2011 3:01 am
Location: Ithaca, NY
Contact:

Re: ospf nbma

Wed Jul 14, 2021 5:19 am

Perhaps what I mean is not to use nbma but PTP or PTMP

Something running on TCP/IP and not multicast or are they both tcp/ip?

Bad thunder storm, BANG, ospf goes down and doesn't come back. Logs say ignoring LSA, wrong peer state, state = exstart

Both ends were up and radios connected to each other.

Homer
 
millenium7
Long time Member
Long time Member
Posts: 538
Joined: Wed Mar 16, 2016 6:12 am

Re: ospf nbma

Wed Jul 14, 2021 6:15 am

If you are losing OSPF adjacency its because you are losing packets, there is a link problem. You mention this is happening during weather well......... theres ur answer. Wireless link is not able to handle the weather conditions
Essentially you either improve the wireless links themselves (ensuring they definitely are aligned, increasing power, different antenna's, different frequencies, physically moving them to a better location, etc) or if you can't actually solve the problem then you have to bandaid the problem to lessen the impact. But ultimately theres no getting around a bad link, there is no magical config that will fix the conditions of it

In essence, NBMA and PTMP have bugs and periodic issues on MikroTik. Only 2 stable types are broadcast or point-to-point, and especially in the case of wireless you should always use PTP type. Even in a point-to-multipoint setup, its still technically a bundle of point-to-point links and you should use 1 VLAN per router. This has some other benefits as well and in general there are no benefits to PTMP or NBMA types

Above and beyond all this though, the main issue is the routing space has a total lack of protocols built specifically for wireless, it shits me when I think about it. OSPF is a TERRIBLE protocol for WISP networks (i've illustrated in detail why on this forum before) but even if we had the choice of another protocol, there's nothing built with the concept of wireless networks in mind. Ideally we need a protocol that can have all devices on that segment inform each other of whats going on. So the actual routing protocol can see what the conditions are, what the potential throughput it, how much retransmission there is, what the modulation is, when a frequency or power change happens etc so it can then dynamically steer and load balance traffic according to those conditions. What we have to work with are very old protocols that don't fit the bill properly, heck OSPF doesn't even handle 1-way transmission properly hence the need for BFD slapped onto it (which also couples as a fast link failure detection protocol) however its still terribad for flapping links or those that change conditions rapidly
 
User avatar
homerwsmith
Member Candidate
Member Candidate
Topic Author
Posts: 166
Joined: Fri Dec 02, 2011 3:01 am
Location: Ithaca, NY
Contact:

Re: ospf nbma

Thu Jul 15, 2021 7:43 am

With the rough wireless seas this link is in, fixing the wireless link is out of the question.

How many packets of loss are we talking about? Or how long must a total lack of packets have to take place before
adjacency is lost.

Would increasing any of the timers help the link stay adjacenct during momentary disconnects?

Since multicast is UDP and PTP is TCP, wouldn't that mean that PTP would inherently be more stable against random lossiness in
a wireless link?
 
millenium7
Long time Member
Long time Member
Posts: 538
Joined: Wed Mar 16, 2016 6:12 am

Re: ospf nbma

Thu Jul 15, 2021 11:47 am

It depends how OSPF is configured since you can set the dead timer. This just means if the hello packets havn't been received, the neighbor is considered down
It could be that 99.99% of traffic gets through just fine but it just so happens to be that the 0.01% were OSPF hello packets, lose enough in a row and its considered down. As soon as a successful hello gets through it resets the countdown
You could adjust these (they have to match on both routers) but it may not be a great solution

Keep in mind also that if the interface physically goes down (the ethernet side) then it will immediately consider it down, hence a flapping link is going to cause a lot of disconnections
PTP is not TCP..... OSPF uses neither UDP nor TCP, it uses its own protocols

In your case if you absolutely must deal with unreliable connections, then you are best off setting a static route and use a higher Distance value than 110 (or whatever you have OSPF using). That way if link A goes down but B is still active, OSPF will still route it appropriate. But if both A and B are down, its going to fallback to using the static route and just shove traffic out that interface
This won't help with anything that relies on OSPF (like MPLS/VPLS) but raw traffic will just go wherever the best active route is

Who is online

Users browsing this forum: No registered users and 19 guests