This is a really long post but you guys wanted to know the details so here you go…
This is again where a diagram would be infinitely useful. It can be ascii art or whatever.
See below and I’ll explain in more detail after I answer your questions
What is the situation? Am I correct:
- The thingies on the 10.0.0.0/8 subnet are connected to your L3 switch?
Yes. Correct
What is the rest?
- Clearly the L3 switch is connected to the Mikrotik. On one port only? 10Gbps?
- Are the 192.168.0.0/23 devices connected to the other ports on the Mikrotik? Are they connected to the switch?
The L3 switch is connected to the Mikrotik by a 10 Gbit SFP+ connection. It is the only connection at this time, but the open ports are there should I need them.
Everything else is connected to the L3 switch. There is a second L2 switch that is also on the network and connected to the L3 switch.
Another question: What is your goal exactly: Do you want the Mikrotik to serve only as a DHCP server and handle NAT? Or would you like it to do other things a well (filter traffic, so for example the 192.168.0.0/23 can “talk to” the 10.0.0.0/8 devices, but not vice versa, etc., or maybe provide a VPN gateway…)?
Either way you will have to do this additional routing subnet thing. Just other configuration afterwards depends on this choice.
I want the Mikrotik (or whatever router) to act as DHCP server, DNS relay, NAT, VPN gateway, firewall, router, etc.
I “could” use the L3 switch as a “LAN” router where I set things up where it is the gateway for all the devices on my LAN. It works as I had it that way yesterday, but then anything on the LAN traversing to the outside world (most of the traffic) would need to be handled at the L3 level by both the switch and the router as opposed to if the router was the gateway, then the switch is only operating on L2 for any traffic to the outside world.
Below is the network diagram scribbled by me. An artist I am not! Let me explain the VLANs as we use them as part of distributing IP video over a network. I write software to work with video devices from a company called Just Add Power (wwww.justaddpower.com). At a basic level there are two sorts of video devices - an encoder and a decoder. The encoder takes and HDMI signal from a video device (camera, cable box, AppleTV, Roku, etc) and encodes that into an IGMP multicast stream. On the other end the decoder, receives the IGMP multicast stream and converts it back to HDMI where it connects to a TV. Because we want video to have priority above all else on the network where the encoder resides, it is set up to basically just transmit data as fast as possible. If you were to put one of these devices on your LAN, it would take down your LAN very rapidly. So we isolate each different video source into its own VLAN. Now whatever decoders are in an encoder’s VLAN receive its signal. If I want to watch something different I move the encoder connected to the TV I am watching to a different VLAN. We do all of our video switching at the Layer 2 level and we are actively changing the programming of the switch depending on what we want to watch. Using this topology we can build literally any size video system. Examples of places using this product are Ceasar’s Palace in Las Vegas, multiple Twin Peaks and Buffalo Wild Wings restaurants. PGA Pro Superstores, and more. I personally manage a hockey arena in Florida that has something like 250 displays and 60 video sources spread across 8 switches. And we do similar routing with all of these and I have never had this issue show up at any customer nor have I had anyone complain to me. This situation I am having is unique and we have been building out these systems for the past 10 years in the manner I will explain. It’s a little “non-standard” in terms of normal networking but it works.
We set up ports on our switches to use “General” mode where you need to assign both a VLAN and a PVID. For those who don’t know, the PVID is what is used as the “transmit” VLAN while listening happens on any other VLAN(s) assigned to that switch port. All traffic by the way is untagged.
All the decoders are put into their own VLAN which we typically call VLAN 10. Their PVID is then 10. So in my network, all the decoders get an IP address in the 10.0.10.0/8 space. But here’s where it gets interesting. The VLAN on the switch is set up as 10.0.10.1/24. And that’s on purpose. Yes it break some rules but it works quite well for the way we do things. This way only traffic that is in the 10.0.10.1/24 network is allowed on the VLAN but the device can talk to any other device in the 10.0.0.0 subnet. Remember I said before that we move the decoders between transmitter VLANs to decide what we want to watch. But then I said they are on VLAN 10. This is the magic of “general” switchport mode. You can have that port in multiple VLANs. So the primary VLAN that device will transmit on is the PVID. But it will LISTEN on all other VLANs that it is connected do.
Now the encoders we want to completely isolate. So they can be put into much smaller networks. I’m using pretty big and inefficient subnets here basically waisting a ton of addresses in a class A subnet. Who cares as it’s a private network and I can configure it how I want! In our bigger installations we use much smaller subnets for everything (and actually have a pretty slick standard for how we do it), but the same concept still applies. Anyhow, the transmitters get a similar type of arrangement. In my case the first encoders gets assigned to VLAN 11 with an IP address of 10.0.11.100/8. VLAN 11 is given an address of 10.0.11.1/24. Now the port this encoder is connected to on the switch gets assigned to VLAN 11 with a PVID of all. But it also gets assigned to VLAN 10.
Do you follow what is happening now? Transmitters talk to receivers on their VLAN and receivers talk back to transmitters on their VLAN. Where IGMP multicasting is in reality a single duplex one way form of communication, we’ve now enabled two way communication and ability for our devices to easily communicate with each other and the outside world.
In a lot of commercial cases, what we call the “video switch” just handles only video and any control traffic on the LAN is just routed into it. We tell the IT people not to think of it as a network switch but a video appliance on the network that gets an IP and traffic routed to it like anything else. Now what they do on their routers to set the routes - I don’t know. I’ve seen specific VLANs that they create just to put our switches on their network. In other cases it just goes on the LAN.
In smaller places and like in my home, the switch shares both video traffic and normal LAN traffic.
My diagram is below.
I
- we will probably never know what was wrong. Neither routing/rules or raw firewall helped or will help you with that.