Big subnets vs small subnets

RouterOS General
1

This is probably a bit of a noob question, but sometimes we have to go back to the basics.

Is it better to use /30 subnets between routers or should routers connected to the same backbone all have their gateways on the same subnet?

For example:

Let’s say you have an RB800.

ether1 is 192.168.1.1
It is connected to a Cisco managed switch port 1.
port 2 of the managed switch is plugged into a RB493 ether1.
port 3 of the managed switch is plugged into a RB493 ether1.
port 4 of the managed switch is plugged into a RB493 ether1.
port 5 of the managed switch is plugged into a RB493 ether1.
fibre 1 on the Cisco is connected to another Cisco.
Now we can repeat
port 1 of the 2nd Cisco is plugged into a RB493 ether1.
port 2 of the 2nd Cisco is plugged into a RB493 ether1.
port 3 of the 2nd Cisco is plugged into a RB493 ether1.

I’m using this scenario in a 10 floor block of flats with 6 flats per floor.

So then ether2 from the RB493 would go to flat 1, ether3 to flat 2, ether 3 etc.

At the moment, I have /30’s from the RB800 to ether1 of each RB493.

I then have /29’s on each of the ether ports on each RB493.

The switches are all in a /29 with one address of the /29 also on ether1 of the RB800

In total there are something like 10 /30 IP addresses and 1 /29 IP address all on ether1 of the RB800.

This works very well.

Nobody is complaining. The people in the apartments are all very happy. The router never goes over 30% CPU and average traffic through the router is only around 15Mb.

However, I’ve been thinking of putting all the infrastructure into a /24. This would mean that all the switches and the routerboard ether1’s would be on the same subnet.

The reason for this is that there’s a tower block next door that belongs to the same owners who want the same service, but this time we’re talking of a 24 floor block with 8 apartments per floor, so I’m probably going to have to put an RB1100 on each floor and they are all going to link back to fibre 2 on the one Cisco.

I can continue as I have been doing, but then we’re going to have something in the region of 35 /30 IP addresses all on ether1 of the RB800 and I don’t know if this is such a good idea.

Ether2 and ether3 on the RB800 are used for other things so they can’t be used for client access. However, I could pull the RB800 and replace it with an RB1100 and then split some of the /30’s onto other ports, but that would mean running a LOT more cable.

There’s a large amount of internal traffic on the network. One guy has set up a DC++ server and people are moving stuff around on that. Another guy one has a couple of games servers in one of our wiring closets and he and his neighbours shoot each other. There’s also an inhouse asterisk system. All that traffic stays inside the network, but I’m worried that having it on the same subnet could cause broadcast storms or other forms of network noise that could have a negative effect on everyone.

There’s 100Mb “metro ethernet” going into ether2 of the RB800. With one block only using 15Mb, there’s enough capacity for the second block - for now…

2

I think I would deploy that completely differently. I would use one router and managed switches, and use one VLAN per flat. The switches simply trunk to the router, which handles all the routing between VLANs where required. The router and switches would share a /28 management VLAN.
By default all routing between flat VLANs would be disabled, when flats request to be able to talk to one another that gets set up for a (moderate) service fee.

An RB1100 should be able to handle that just fine - as should an RB800. What’s your reason for deploying that many RB493s?

3

You may want to look into Private VLAN.

Cisco PVLAN

I’m not aware of any other vendor which implements such functionality. If I were I’d post it here.

4

Network has been built to client specs with client supplied equipment.

One of the people on the management committee is an “IT specialist” and he’s dictated what hardware gets purchased from where.

My job has been to get the stuff working.

5

Then I would prefer a /30 per router, even if that means many subnets. It’s both easier to secure and to troubleshoot. Not ideal, though, in my opinion.

Just on a side note, if you do ever implement PVLANs make sure you also implement a router side ACL/firewall that drops all traffic into the VLAN from addressing space carried on the VLAN interface not sourced from the router IP, and all traffic from the VLAN destined to addressing space carried on the VLAN interface that isn’t going right to the router - without that PVLANs are pretty easy to circumvent with ARP spoofing.

6

There are different schools of thought when it comes to routing.

In general it’s best practice to have a point to point link as a /30 as it conserves IPv4 addresses.

In your scenario I would suggest creating a core switched network, bond interfaces if you have to but it’s the foundation you build on so it’s important. Then at each floor put a router like you are to go to the clients.

It’s perfectly ok to put all the routers on one subnet over the core network on a setup your size. I haven’t done it with Mikrotik but I have done setups like that with Cisco and Brocade though on the Cisco core we were bonding two 10Gbps fiber lines between the core switches and bonded 5 1Gbps for the Brocade network. We switched as much as possible and only routed where we had to.

A small low thorough network like yours is much more forgiving but it doesn’t hurt to prepare for the future. I suggest finding some Cisco network design books and getting some ideas from them. I take pride in getting the networks I work on to be fault tolerant and fast while minimizing latency.