Is this a valid network design?

spyghost · Sat Mar 04, 2023 6:20 am

I'm fairly new to network and I'm trying to set up a small home lab. Due to space limitations, I'm using a single hEX router.

ether1 is a DHCP client of the ISP modem
ether2 is connected to a TP-Link Deco mesh for LAN devices, phone, laptop, etc...
ether3 is connected to a proxmox server with a single NIC that will host 3 VLANs, management interface of proxmox, lab vlan and dmz vlan

Considerations:

All networks should have internet access
Only Lab network has access to DMZ, DMZ cannot get in to Lab nor LAN
Only LAN network has access to Lab, LAB cannot get in to LAN

In the VLAN diagrams that I see, it's normally router with a trunk connecting multiple switches. Due to constraints, I want to limit the design to just 1 hEX router, hence the question if this is a 'valid' design for a simple home setup?

If not, what would be the recommended way to gain VLAN separation for my proxmox hypervisor while using just 1 hEX router for this use case?

I've been playing with multiple configurations and I can't get in to any of my proxmox VLANs, hence the question if this design is even feasible/possible?

For simplicity, isolation, and easier troubleshooting, I only used ether3 to connect to a proxmox hypervisor with an IP of 192.168.68.10 (vlan99), my laptop with an IP of 192.168.68.254 (untagged) c/o mikrotic DHCP. From the config below, ping doesn't work from either the proxmox hypervisor nor my laptop

# jan/02/1970 00:12:33 by RouterOS 7.8
# software id = 8GAA-890C
#
# model = RB750Gr3
# serial number = HDJ08PDPNZ1
/interface bridge
add name=bridge1 vlan-filtering=yes
/interface vlan
add interface=bridge1 name=mgmt vlan-id=99
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile
set [ find default=yes ] html-directory=hotspot
/ip pool
add name=bridge1 ranges=192.168.68.100-192.168.68.254
/ip dhcp-server
add address-pool=bridge1 interface=bridge1 name=bridge_dhcp
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge1 interface=ether3
add bridge=bridge1 interface=ether5 pvid=99
/interface bridge vlan
add bridge=bridge1 tagged=ether3,bridge1 untagged=ether5 vlan-ids=99
/ip address
add address=192.168.68.1/24 interface=bridge1 network=192.168.68.0
add address=192.168.68.1/24 interface=mgmt network=192.168.68.0

anav · Sat Mar 04, 2023 6:35 pm

There are many ways to accomplish what you have done, I prefer one bridge all vlan, in other words your home subnet would also be on a vlan.
Since I am unfamiliar with other ways, wont attempt to say good or bad, but if it works for you, then thats your answer.

tdw · Sat Mar 04, 2023 8:05 pm

The fundamental error is the same subnet is assigned to multiple L2 interfaces (bridge1 and mgmt). You have various options:
Use a different subnet on these two interfaces.
Combine the interfaces, this in turn can be achieved in two ways:
Do away with the mgmt VLAN and address, use the untagged the bridge and ports for your LAN & mgmt network.
Do away with the bridge1 address, set the PVID on the LANport to be placed on the mgmt VLAN network.

spyghost · Sun Mar 05, 2023 4:51 pm

With the latest code...

/interface bridge
add name=bridge1 vlan-filtering=yes
/interface vlan
add interface=bridge1 name=mgmt vlan-id=99
/interface wireless security-profiles
/ip pool
add name=mgmt_pool ranges=192.168.68.100-192.168.68.254
/ip dhcp-server
add address-pool=mgmt_pool interface=mgmt name=dhcp1
/interface bridge port
add bridge=bridge1 interface=ether3
add bridge=bridge1 interface=ether5 pvid=99
/interface bridge vlan
add bridge=bridge1 tagged=ether3,mgmt untagged=ether5 vlan-ids=99
/ip address
add address=192.168.68.1/24 interface=mgmt network=192.168.68.0
/ip dhcp-server network
add address=192.168.68.0/24 dns-server=192.168.68.1 gateway=192.168.68.1

here's what I can do (successful)....

ping promox from laptop

here's what i cannot do (unsuccessful)...

ping laptop from proxmox
ping 192.168.68.1 from laptop
ping 192.168.68.1 from proxmox
get DHCP address to my laptop (i just assigned static IP)
hEX doesn't appear in winbox when in ether5 port (had to transfer to ether4 to gain access)

Attached is the simplified diagram of my troubleshooting...

Design2.jpg

I'm pretty sure, what i'm missing is just there, but not sure what it is...

tdw · Sun Mar 05, 2023 5:18 pm

The /interface bridge vlan section should only refer to ports in /interface bridge port and the implict bridge-to-CPU port which has the same name as the bridge itself. For more detail see viewtopic.php?f=2&t=173692

The /interface vlan section defines objects which add a tag when passing traffic to the parent interface and removes a tag when receiving traffic from the parent interface.

/interface bridge vlan
add bridge=bridge1 tagged=bridge1,ether3,mgmt untagged=ether5 vlan-ids=99

spyghost · Mon Mar 06, 2023 2:06 pm

Gotcha, thanks. I'm still trying to get my head around the virtual concepts in networking, but I think I got it sort of...

Here's my config now which is working - all pings to relevant interfaces are OK.

/interface bridge
add name=bridge1 vlan-filtering=yes
/interface vlan
add interface=bridge1 name=mgmt vlan-id=99
/ip pool
add name=mgmt_pool ranges=192.168.68.100-192.168.68.254
/ip dhcp-server
add address-pool=mgmt_pool interface=mgmt name=dhcp1
/interface bridge port
add bridge=bridge1 interface=ether3
add bridge=bridge1 interface=ether5 pvid=99
/interface bridge vlan
add bridge=bridge1 tagged=ether3,bridge1 untagged=ether5 vlan-ids=99
/ip address
add address=192.168.68.1/24 interface=mgmt network=192.168.68.0
/ip dhcp-client
add interface=ether1
/ip dhcp-server network
add address=192.168.68.0/24 dns-server=8.8.8.8,8.8.4.4 gateway=192.168.68.1 netmask=24
/ip firewall nat
add action=masquerade chain=srcnat out-interface=ether1

What's left for me to work on this time are the additional VLANs for proxmox. Just to take one step at a time, I've added VLAN77 in proxmox and from the picture this is in ether3. Now ether3 carries VLAN99 and VLAN77, hence called a trunk?

/interface vlan
add interface=ether3 name=lab vlan-id=77
/interface bridge vlan
add bridge=bridge1 tagged=ether3 untagged=bridge1 vlan-ids=77
/ip address
add address=192.168.77.1/24 interface=lab network=192.168.77.0

Based on the code edits I recently made for VLAN77, it seems that they 'all' see each other - ie VLAN77 can ping VLAN99 and vice versa. Is this an expected behavior, which can only be limited by FW rules if I only want VLAN99 to reach VLAN77 and not the other way around?

My VLAN table shows this

Flags: D - DYNAMIC
Columns: BRIDGE, VLAN-IDS, CURRENT-TAGGED, CURRENT-UNTAGGED
#   BRIDGE   VLAN-IDS  CURRENT-TAGGED  CURRENT-UNTAGGED
0   bridge1        99  bridge1         ether5          
                       ether3                          
1 D bridge1         1                  bridge1         
                                       ether3          
2   bridge1        77  ether3          bridge1

Buckeye · Tue Mar 07, 2023 11:10 am

The problem with ROS is that it is like writing code in assembly language, and it is happy to let you create invalid configurations without any complaint.

It isn't clear what you are attempting to do, but I don't think the following is going to do what you expect.

/interface vlan
add interface=ether3 name=lab vlan-id=77

The above is creating a vlan interface under ether3.

**** Edit **** when coming back and reading the post, I see I didn't complete what I mean to say about the above. Normally when using the vlan-filtering bridge configuration, you would have ether3 as a member of the bridge, and you would create the vlan interface under the bridge1 device, not ether3.

E.g. I think you should be using
/interface vlan
add interface=bridge1 name=lab vlan-id=77

**** end edit ****

This will be tagged on the "internal" link between the CPU and the Switch. You control whether the switch ports are tagged or untagged for the vlan using the /interface bridge vlan and /interface bridge port commands.

/interface bridge vlan
add bridge=bridge1 tagged=ether3 untagged=bridge1 vlan-ids=77

bridge1 vlans other than the pvid associate with the bridge device need to be specified as tagged. Since you didn't specify the pvid on the /interface bridge command, it will default to pvid 1. So the part in red is incorrect. It should be

/interface bridge vlan
add bridge=bridge1 tagged=bridge1,ether3 vlan-ids=77

In addition to the other references you have been given, here is a writeup about the hEX S (which is very similar to the RB750Gr3, it just has an SFP port as well). Secure Mikrotik Router Hex S RB760iGS for Home Office Use by David Hall (hallzhallz on github)

It's worth reading. I haven't tested it, but I don't see any errors when scanning it.

I just watched How To Create VLANs in Proxmox For a Single NIC and it has some good pointers. I would get the main vlans working before converting to a pure trunk to the Proxmox server so you don't lock yourself out (since you don't have any vlan aware switch other than the hEX).

For background on vlans and networking, I think that Ed Harmoush's Practical Networking is very good. Virtual Local Area Networks (VLANs)

anav · Tue Mar 07, 2023 1:17 pm

Yes, you should always be able to ping interfaces, that has nothing to do with passing data.
If users/devices are on separate vlans they will not be accessible UNLESS you screw up the firewall rules and allow the router to do so LOL.

As for the first statement all the interfaces are considered to be part of the router and thus why firewall rules in the forward chain dont block pinging the interface, nor vlan interfaces.
You could block this in the input chain but work for nothing really, unless its a corporate IT rule.

Buckeye · Wed Mar 08, 2023 8:54 am

When re-reading post #7 I saw I had made a comment about

/interface vlan
add interface=ether3 name=lab vlan-id=77

but didn't complete it.

I went back and edited the post to add what I meant to include.

See this @tdw post in the thread he linked before.

Is this a valid network design?

Is this a valid network design?

Re: Is this a valid network design?

Re: Is this a valid network design?

Re: Is this a valid network design?

Re: Is this a valid network design?

Re: Is this a valid network design?

Re: Is this a valid network design?

Re: Is this a valid network design?

Re: Is this a valid network design?

Who is online