So I currently have a single router setup that is then distributed by a hierarchy of L2 smart switches. We also have multiple VLANs on our network. Lets say for instance that our router and all of our L2 switches reside on a management network of 192.168.1.1/24. I am wanting to reduce broadcast domains. So my thought is it is time to start to implement more routers at key points in the topology and implement OSPF on those routers. My question (and I may be over thinking it) what is the difference between these two topology layouts? It seems to me that the second example would not reduce broadcast domains because the main backbone connection of the network is running on the L2 switches before hitting the routers. Help me out to settle my confusion.
Hello,
Honestly, I would not use either designs. They look like they would be for high redundancy, and if so, you need a fully meshed setup. That means that you need extra like between opposite switches or routers, depending on the diagramm you’re looking at. You could loose any two device and still have communications between the remaining ones. The extra links also make for a more direct route for the two opposite routers or switches, thus reducing latency.
Now, from my understanding, you have one router and multiple switches and need to limit the broadcast domains. VLANs do that right, not only collision domains. I would configure my router with all the VLANs that need to be present and configure as many trunk ports as you have switches. Then, one ethernet port to one switch and voilà!
In a fully redundant setup, you would have two routers, each with a link (trunk) to all switches and one link between each router. That way, if one router dies, then the other takes automatically. You would use VRRP to make the gateways highly available. If a link breaks, everything is still available.
Hope this helps,
AC
Yes this does help. I thought of your scenario as well, just setting up a VRRP router which I think I will do. I know that VLANs increase the number of broadcast domains which is good because they reduce the size of them… I just worry as my management subnet is starting to fill up, getting close to 200 devices on it. Not sure if that will be problematic as those broadcasts hit every switch trunk and device on that subnet. Opinions?
I think it’s unavoidable. Even if you did /25 and doubled up on VLANs, the trunks would still carry all the broadcasts.
Remember that broadcasts are normal part of any network.
[thinking out loud]If some app starts broadcasting big times, maybe you could put it on a VLAN which would NOT go beyond a certain point. Maybe certain bridge filters to block that traffic. Have trunks that don’t carry that particular VLAN (although I’d have to see how it’s done in ROS)[/thinking out loud].
Also, if you go VRRP, you could play with (r)stp and OSPF timers for a faster recovery. It could be tricky depending on network size and topology, but it’s worth giving it a shot.
That is why I was looking at doing multiple routers so it would break up all the switches etc into groups. Routers don’t forward broadcasts so it would help in that area…
But if you do that, you break up all vlans and subdivide your network… And costs more. Is that what you want? Your approach will work, but your lan becomes like multiple wans in one building.
Sent from Tapatalk
I get what you are saying about cost but I did not clarify about our situation. We are a smaller city so we have several geographically different locations. All of our locations are connected through either our own fiber (yes we literally built out our own fiber infrastructure) or point to point off of our water tower. It is all L2 switches right now with a single router, several VLANs. Single WAN connection for internet shared across the network. So… multiple routers wouldn’t be a bad thing to replace some of the switches at key intersect points. Like I said I just worry about the broadcasts getting too big at some point. Maybe not now but in the future, just trying to plan ahead.
Oh! That’s different.
Lucky guy with his own Fiber!!!
Then YES! Router for all sites. Allows you to add QoS and proper firewalling. Don’t forget about a proper IP address plan also. Critical always, more so with multiple sites. And for what MT costs, no big deal when your goal is a solid network.
So, I would go with L3 as main access for each sites and the switches for end-user/device connectivity. Makes sense now 
LOL yeah oops should have mentioned that in original post… I have to wrap my head around the IP plan… ugh. Our network is rapidly expanding and now is the time to do it. IP cameras, public wifi, blah blah blah continue to expand and come online. I have several of the small routers just to setup a test lab to make sure my thinking is all okay. I suppose I could use the same subnet for my routers but just do it with /30 address masking? Does that make sense?
Yes it is nice (and fun) to build out our own fiber. We even do the fusion splicing etc.
IP addressing is a can of worms for many people… until you beging and then it’s ok.
Here are some suggestions I give to my students:
- Take all 3 private address spaces and give them rolls. For example, I use Class A (10.0.0.0) for loopbacks, Class B (172.16.0.0 - 172.31.0.0) for subnets and Class C (192.168.0.0) for WAN links.
- Start assigning subnets from an even number (172.16.0.0 rather than 172.16.1.0). It’ll kill you when you start playing with subnets sizes.
- Make friends with /32. Some will rave against it, but there’s no reason. It works for my L2TP links. Hell, I have no choice with my L2TP links. When your local address is always the same, how can you differentiate link addresses when not using /32?
- Think “binary”. Don’t reserve three subnets for one site, reserve 1, 2, 4, 8, etc.. Think 2 to the power of whatever.
- All sites should have the same amount of subnets (if they’re pretty much all the same size) and all subnets should have the same function. For example, 1 is Network mngmt, 2 is servers, 3 is PCs, 4 is phones, etc. Repetition is your best ally in the long run. You’ll always know what all subnets do.
- Segment your address space by district, neighborhood, whatever makes sense for you. You’ll come to know certain addresses for certain areas.
- IP addresses is all fine and dandy, but use a good naming convention also. Ask other departments if such a convention exists and use it. DONT reinvent the wheel. I worked for an accountant firm and we named our equipement based on the office’s name. Makes it MUCH simpler for all to understand each other. I worked for a company that didN,t do that because “IT knows it all”. THREE different naming conventions, total chaos!!
I could go on, but I’d have to charge you 
I’ll just say this; if your address and naming plans are good, the rest is almost just plug and play, because you’ll already know where you’re going. And since you do your own fiber (again LUCKY!!!), remember to build redundant link, and NOT coming from the same pipes. Kinda useless when a dozer rips through one
Think I’ll go to bed now.
Cheers.
Great info. Thanks for everything.
please explain it
I had a similar setup with similar questions a while back. My design looks something like the 2nd picture but with fewer routers and no loop
My primary design goal was to have L2 connectivity between “sites” (really buildings about 1/4 mile apart) in my setup, and I have single mode fiber between buildings. Multiple, for redundancy.
Based on that, I have a “top of building” L2 switch at each site carrying lots of vlans direct attached to 2 large routers doing VRRP in an active/standby setup. The routers are in separate buildings for even better redundancy. We control broadcast domains by just adding more vlans and pushing them to whichever building that needs them.
Most people don’t have stuff that needs to be in the same subnet to work, but I do, which is what made the decision for a design like this. Enough to where tunneling ethernet would be counter-productive. Sure is simple from a routing perspective though.
Thanks for the info. Based on your description it does sound very similar to what our setup is. I went back through stats on some of our very busy switches and looked at how many broadcast packets it recorded compared to total packets on the various port interfaces. In all instances the calculation came out to less than 0.50% of broadcast packets compared to normal packets exchanged. In fact most of the time it was less than 0.10%. So I may be worried about nothing since everything broadcast wise is way less than 1% packets exchanged… Seems like broadcast traffic is kept to a minimum since we have everything VLAN out.