Our network is growing!

We started with one location and added another a while back. Ignoring best practices, we extended Mikrotik RouterOS bridging and VLANs to the limit of their combined capacity. It is time for us to grow up and become a real WISP.

In so doing, we have built some PowerRouter 732 equivalents and replaced a wireless link with a fiber point to point link. We finally have decent equipment that connects both main locations, but have continued bridging while we look for ways of migrating customers to new IP space so we can divide the bridge and begin routing like grown ups.

Here are the resources we have:

1. Plenty of unused public IPs.
2. Three Beefy routers running RouterOS 5.0beta2 (It was the only version of RouterOS that has drivers for our ethernet cards.)
3. A generic network map.
My goal is to reconfigure the network so that has the following features:

1. It is internally routed (not bridged)
2. Customers (IP addresses, walled garden, etc) are manageable via Platypus from Tucows.
3. It is multihomed, including running BGP, etc.

My plan is to work on the goals in the order expressed above. But I am starting at zero. The purpose of this post is to get ideas on how to establish OSPF in my network. I have read the examples of OSPF on the Mikrotik site, but do not yet understand how OSPF should be implemented on a production network without causing downtime.

Specifically, I am looking for suggestions on how the background area should be configured. What routers should be in the background area and which routers should be in transit and/or stub areas.

Any thoughts are appreciated.

Lorell Hathcock

Set up IP addresses on routers, enable OSPF and then one by one remove bridges.

Thanks for the response.

What about OSPF areas? Is the whole network in the backbone area? Or are they each in their own area?

It depends on how many routers and how many routes you will have in total. As from your diagram all three routers can be in backbone area.

How about the CPE radios? Do they go into the backbone area too?

Here's how I see them working.

1. The CPE radio powers up.
2. It makes a DHCP request for its wlan1 interface.
3. It makes a PPPoE request (of a local radius/PPPoE server) for an IP block for its ether1 interface.
4. It injects that customer IP block into the network via OSPF.
5. All routing and authentication are handled.

Is this possible, desirable?

Just a quick point of clarification. Does an OSPF area fill out all of a broadcast/collision domain? From what I am reading, I see that OSPF likely does not want two areas sharing the same broadcast domain. And in that sense, I want to route between each of the three routers in my drawing, so does that mean a different OSPF area for each broadcast domain?

Here's how it would break out:

1. One OSPF area for the ether interface of router 1 for the APs and CPEs
2. One OSPF area for the interfaces for the fiber connection between router 1 and router 2
3. One OSPF area for the interfaces for the fiber connection between router 2 and router 3.
4. One OSPF area for the ether interface of router 3 for the APs and CPEs

Thoughts? Or is this getting too crazy?

To answer the question about how many routes will be on the network, right now with about one hundred customers I have about one hundred routes on one router for them. I expect that number to be the same for each main router, so three hundred routes right now. And that is just if we stop growing now which is not the point of this whole excercise.

I have 64 routers all in the backbone area. Running without a problem.

Are those routers both CPE radios and infrastructure routers?

As a fellow WISP operator I feel your pain. We made this transition a while ago.

OSPF does routing. It just tells stuff where to go. Breaking up broadcast domains is done with clever IP addressing.

When you define the networks on an area router, make sure to define the whole network (10.x.x.0/21)
Sure, setting areas will limit things but that can also be done other ways (firewall rules). Looking at the diagram that you provided, I would strongly suggest one area, and work your way out later.

My suggestion is to create a few NAT pools, one for each router perhaps with a network of say /24 for each. That gives you 253 available for customers on each router. Maybe a larger pool, that is up to you.
router 1= 172.16.0.0/24 (routing ospf network add network=172.16.0.0/24 area=backbone)
router 2= 172.16.1.0/24
router 3= 172.16.2.0/24

That covers the customers. Then put in your networks for Infrastructure.
router 1 = 10.0.0.0/24 ... etc.

Then you are controlling your broadcast domains by addressing, not excessive ospf areas. The routers will relay what they know to each other and take care of what goes where. If you further want to control things create a small net for only between routers, and then put that into the ospf.

We run 30+ routers in this fashion and have very few issues. All in the same area, but broken up by addressing. I can only imagine if you keep growing, static routing becomes plain insanity.

in wisp hotspot set up is ok to put into ospf only networks for Infrastructure. ex.10.0.0.1 (router 1), 10.0.0.2 ( router 2 ) , etc ...


thex for your help

It's technically OK, but pretty pointless. Those IPs are already on the same subnet so you don't need a routing protocol for them - you need a routing protocol so that via the shared subnet the subnets BEHIND the routers can be found by other routers.

Unless I'm misunderstanding your question.

Sorry for confusion.

Here is my set up of WISP hotspot in prototype stage for now.



------------------------------ROUTER A --------------------------------

I have 1 dsl line 5Mbit/800kbs rb 750G,
I have 4 hotspots configures behind NAT masqueraded.
the hotspots subnets are as follow.
192.168.1.1 -x.x.x.254 (dhcp)
192.168.2.1 -x.x.x.254 (dhcp)
192.168.3.1 - x.x.x.254 (dhcp)
192.168.4.1 - x.x.x.254 (dhcp)
The WAN ip lets call it 2.4.53.3

and the future OSP interface ip is: 10.0.1.1


-------------------ROUTER B------------------------------------------


The same configuration like on router A ffor Hotspot behind NAT, with OSP interface ip: 10.0.2.1


--------------------------ROUTER C ---------------------------------


the same configuration like on router A and B for hotspot, behind NAT with OSP interface IP
10.0.0.3.1



-----------------------------CONCLUSION -------------------------
Do i have to put all the Hotspot Subnets to OSP "backbond" network?
or i don't have to, cause it use NAT to translated the IP address,

I only will put the 10.0.x.y ips to OSP. Thats all, it doesn't need to know what the enterface with 10.0.x.y ip will do with it , from that with its point of view. From OSP point of view after connecting to that IP OSP job is done.Am i correct?



I don't understand why you put subnets behind NAT to OSPF?
when OSPF uses only 10.0.x.y subnets.
Just curious.




Correct if i wrong. or missing something


thx

You typically do not run OSPF with an ISP. OSPF is what is called an interior protocol that you run on your own network.


You really should not NAT to private IPs.

i guess u haven't read my questions

How about the CPE radios? Do they go into the backbone area too?

Here's how I see them working.

1. The CPE radio powers up.
2. It makes a DHCP request for its wlan1 interface.
3. It makes a PPPoE request (of a local radius/PPPoE server) for an IP block for its ether1 interface.
4. It injects that customer IP block into the network via OSPF.
5. All routing and authentication are handled.

Is this possible, desirable?

Dont talk OSPF to your CPEs. If you've an instable connection to a cpe
you see routes coming and going. Each OSPF router on the whole area
has to recalculate its network map. Our border routers talk OSPF and
BGP and therefor have huge routing tables. So each OSPF change gives
stress to them...
We have a complete routed network (since some proxim devices in the very
past caused problems in bridging mode).
But we use OSPF only where there are alternative routes possible. In leaf
parts of the network, where there is only one way out, we use static routes.
This causes some work to setup but keeps the OSPF area smaller.

You typically do not run OSPF with an ISP. OSPF is what is called an interior protocol that you run on your own network.


You really should not NAT to private IPs.

Ah - why? All internal adresses etc. of an ISP are pretty much an example of an internal network, or? With BGP on the edges to handle the border gateway routing.

What is wrong using OSPF within your own network, even as an ISP?

I meant you as a customer don't run OSPF with your ISP. Either party can run OSPF within their AS, but you don't run OSPF between the two.

I apparently wildly misinterpreted his post (I've read it several times since and still don't understand it).

make backbone with ospf and built mpls .