Community discussions

MikroTik App
 
Brough
just joined
Topic Author
Posts: 18
Joined: Sun Jul 18, 2010 7:30 pm
Location: Boston, MA, USA
Contact:

OSPF - How large can a flat network grow?

Thu Jun 07, 2018 9:35 pm

We're an urban WISP with a dense mesh of wireless links and a router per building. I am concerned that, without paying attention, we have grown to 600+ routers and ~2550 routes in one OSPF domain. This network has a diverse mix of routers from CCR1036s down to RB750UPs. We're not having any OSPF problems at this time and I have plenty of other things to worry about, but I'd hate to hit some limit and have the whole thing blow up.

Does anyone have experience (positive or negative) with large flat OSPF networks based on MikroTik routers?
And, if you have had problems, what were the problems?
 
User avatar
Anumrak
Forum Guru
Forum Guru
Posts: 1174
Joined: Fri Jul 28, 2017 2:53 pm

Re: OSPF - How large can a flat network grow?

Fri Jun 08, 2018 4:00 pm

In a single ospf area must be not more than 100 routers I believe. I hope you using lsa type 3 between areas in order to save processors time to recalculate shortest path. With OSPF help you have to announce only tech nets. For large scale better use BGP.
 
Brough
just joined
Topic Author
Posts: 18
Joined: Sun Jul 18, 2010 7:30 pm
Location: Boston, MA, USA
Contact:

Re: OSPF - How large can a flat network grow?

Fri Jun 08, 2018 4:07 pm

In a single ospf area must be not more than 100 routers I believe. I hope you using lsa type 3 between areas in order to save processors time to recalculate shortest path. With OSPF help you have to announce only tech nets. For large scale better use BGP.
Well we have 600+ routers in our largest network and it is working and it is stable. Are you quoting a rule of thumb from the 1990s or do you have experience with a 100+ router network that had problems?

The only measurable issue I can see is the time for routes to propagate. For example if I add a route or I throw traffic in one part of the network onto a backup path, the route change can take 2-3 seconds to appear on a router that is far removed from where the change happened. But that's the only issue I can see today.
 
User avatar
Anumrak
Forum Guru
Forum Guru
Posts: 1174
Joined: Fri Jul 28, 2017 2:53 pm

Re: OSPF - How large can a flat network grow?

Fri Jun 08, 2018 5:15 pm

Do you use bfd protocol between peers?
 
Brough
just joined
Topic Author
Posts: 18
Joined: Sun Jul 18, 2010 7:30 pm
Location: Boston, MA, USA
Contact:

Re: OSPF - How large can a flat network grow?

Fri Jun 08, 2018 6:31 pm

Yes we use BFD, with MikroTik default timers.

I'm assuming BFD is entirely local between two peers and doesn't relate to stability of the whole network. In any event, we've never had occasion to look at other than BFD defaults.
 
User avatar
jprietove
Trainer
Trainer
Posts: 212
Joined: Fri Jun 03, 2016 3:00 pm
Location: Cádiz, Spain
Contact:

Re: OSPF - How large can a flat network grow?

Sat Jun 09, 2018 10:39 am

In a WISP covering a very extensive area with 30 PPPoE servers, more than 300 PtMP and more than 2000 CPE we had problem with MPLS: sometimes MPLS forwarding table doesn't follow OSPF.
We decided to split the very big OSPF domain into several little ones using iBGP. Routing tables have diminished from almost 800 routes to 30.
Troubleshooting is now easier than before.
 
User avatar
Anumrak
Forum Guru
Forum Guru
Posts: 1174
Joined: Fri Jul 28, 2017 2:53 pm

Re: OSPF - How large can a flat network grow?

Sat Jun 09, 2018 11:40 am

In a WISP covering a very extensive area with 30 PPPoE servers, more than 300 PtMP and more than 2000 CPE we had problem with MPLS: sometimes MPLS forwarding table doesn't follow OSPF.
We decided to split the very big OSPF domain into several little ones using iBGP. Routing tables have diminished from almost 800 routes to 30.
Troubleshooting is now easier than before.
Yes. First of all need to understand how big you need to create mpls cloud with ospf area 0.0.0.0, then you can delegate diff ospf areas with none Dijkstra's algorithm routing(lsa type 3). After it you can create bgp relationships between ospf loopbacks. With this design, the network will feel better.
 
sri2007
Member Candidate
Member Candidate
Posts: 206
Joined: Wed May 20, 2015 10:14 pm
Location: Lake Grove, NY

Re: OSPF - How large can a flat network grow?

Wed Jun 13, 2018 4:23 am

Hi!! the craziest network that I've seen is about 700subs in a flat bridged/switched one... but we really recommend to everyone move into a routed network instead of a switched one, by using OSPF with BGP, you can configure OSPF as an IGP which can handle everything related to convergence, load balancing and loopbacks while iBGP can be used for managing the rest of your subnets. It's really a most effective way of building a WISP/ISP because it's scalable and easy to manage, once that you get the idea of how everything works.
 
User avatar
awacenter
Member Candidate
Member Candidate
Posts: 201
Joined: Thu Dec 09, 2004 12:58 pm
Location: Castellón
Contact:

Re: OSPF - How large can a flat network grow?

Fri Jun 15, 2018 5:40 pm

Hi!! the craziest network that I've seen is about 700subs in a flat bridged/switched one... but we really recommend to everyone move into a routed network instead of a switched one, by using OSPF with BGP, you can configure OSPF as an IGP which can handle everything related to convergence, load balancing and loopbacks while iBGP can be used for managing the rest of your subnets. It's really a most effective way of building a WISP/ISP because it's scalable and easy to manage, once that you get the idea of how everything works.
That's true! I conform this solution. Besides of routing protocols for your distribution network, I recomend implenting BGP in PPPoE servers which connect with you central connnection node when you provide Public addressing to your customers.

Who is online

Users browsing this forum: No registered users and 7 guests