I've tried to finally setup some VLAN separation in my home network due to upcoming changes. I was relying on:
- viewtopic.php?t=143620
- https://wiki.mikrotik.com/wiki/Manual:S ... our_Router
- viewtopic.php?t=182373
- and a lot of posts related to Unify & Mikrotik combo
So far so good, my setup is working as I expected (more or less ). I have some doubts or I need some clarification on some topics (which I could not understand on my own even after many time spent on reading docs).
I want to show you my network diagram with some description what is what, purpose of resources, etc.
1. Network diagram
2. Description
HOME_VLAN => Internet WIFI access (2G & 5G separated SSID): all users devices like laptops, PCs, phones, TV, Chromecast, etc.
IOT_VLAN => no Internet WIFI access (2G SSID): *future* all IoT devices without access to Internet like IP cams, etc.
TESTING_VLAN => no Internet WIFI & ETH access (2G SSID): VLAN for launching isolated machines (VMs) or connecting untrusted stuff for testing
IOT_INTERNET_VLAN => Internet WIFI access (2G SSID): all IoT devices which requires cloud access, but have to be isolated, e.g. Roborock (don't want to root my S5 Max vacuum cleaner due to Valetudo limitations compared to Roborock app)
DMZ_VLAN => Internet ETH access: VLAN for launching VMs on Proxmox where all virtual servers will be kept and services, e.g. Gitlab and other stuff (some of those will be exposed to Internet via Cloudflare Access)
MANAGEMENT_VLAN => Internet ETH access: physical servers, APs, etc. - all "admin" stuff that I want to have isolated from above cases
The idea is to create 4 "management" ethernet ports on Mikrotik to keep required physical stuff in MANAGEMENT_VLAN and to have 1 free port for "management" if required (e.g. to easily connect my laptop)
I'm using Android USB Tethering (lte2 is my WAN) because LTE modem in Mikrotik hAP ac3 LTE6 sucks a lot Weak antenas, only 2CA so my old OnePlus 6 (LineageOS 19) beats it, but thinking about switching to some LTE/5G modem, e.g. ZTE MF286D, but for now, it's sufficient.
Proxmox servers requires to have static IP (not reserved from DHCP) - underlay stuff rely on this IP address, so I that's why I left some IPs free in range to have no conflict in DHCP with duplicated IP addresses).
All other stuff I want to have visible in Mikrotik DHCP leases (some addresses are reserved as you can see in below config).
3. Configuration
3.1 Mikrotik configuration
Code: Select all
/interface bridge
add name=bridge-custom protocol-mode=none vlan-filtering=yes
/interface vlan
add interface=bridge-custom name=HOME_VLAN vlan-id=10
add interface=bridge-custom name=IOT_VLAN vlan-id=20
add interface=bridge-custom name=TESTING_VLAN vlan-id=30
add interface=bridge-custom name=IOT_INTERNET_VLAN vlan-id=40
add interface=bridge-custom name=DMZ_VLAN vlan-id=50
add interface=bridge-custom name=MANAGEMENT_VLAN vlan-id=99
/interface list
add name=WAN
add name=VLAN
add name=MANAGEMENT
/ip pool
add name=HOME_POOL ranges=192.168.10.2-192.168.10.254
add name=IOT_POOL ranges=192.168.20.2-192.168.20.254
add name=TESTING_POOL ranges=192.168.30.2-192.168.30.254
add name=IOT_INTERNET_POOL ranges=192.168.40.2-192.168.40.254
add name=DMZ_POOL ranges=192.168.50.2-192.168.50.254
add name=MANAGEMENT_POOL ranges=192.168.99.5-192.168.99.254
/ip dhcp-server
add address-pool=HOME_POOL interface=HOME_VLAN name=HOME_DHCP
add address-pool=IOT_POOL interface=IOT_VLAN name=IOT_DHCP
add address-pool=TESTING_POOL interface=TESTING_VLAN name=TESTING_DHCP
add address-pool=IOT_INTERNET_POOL interface=IOT_INTERNET_VLAN name=IOT_INTERNET_DHCP
add address-pool=DMZ_POOL interface=DMZ_VLAN name=DMZ_DHCP
add address-pool=MANAGEMENT_POOL interface=MANAGEMENT_VLAN name=MANAGEMENT_DHCP
/interface bridge port
add bridge=bridge-custom interface=ether1 pvid=99
add bridge=bridge-custom interface=ether3 pvid=99
add bridge=bridge-custom frame-types=admit-only-untagged-and-priority-tagged interface=ether4 pvid=99
add bridge=bridge-custom frame-types=admit-only-untagged-and-priority-tagged interface=ether5 pvid=10
add bridge=bridge-custom interface=ether2 pvid=99
/ip neighbor discovery-settings
set discover-interface-list=MANAGEMENT
/interface bridge vlan
add bridge=bridge-custom tagged=bridge-custom,ether1 untagged=ether5 vlan-ids=10
add bridge=bridge-custom tagged=bridge-custom,ether1 vlan-ids=20
add bridge=bridge-custom tagged=bridge-custom,ether1,ether2,ether3 vlan-ids=30
add bridge=bridge-custom tagged=bridge-custom untagged=ether1,ether2,ether3,ether4 vlan-ids=99
add bridge=bridge-custom tagged=bridge-custom,ether1 vlan-ids=40
add bridge=bridge-custom tagged=ether2,ether3,bridge-custom vlan-ids=50
/interface list member
add interface=lte2 list=WAN
add interface=MANAGEMENT_VLAN list=VLAN
add interface=HOME_VLAN list=VLAN
add interface=IOT_VLAN list=VLAN
add interface=TESTING_VLAN list=VLAN
add interface=IOT_INTERNET_VLAN list=VLAN
add interface=DMZ_VLAN list=VLAN
add interface=MANAGEMENT_VLAN list=MANAGEMENT
/ip address
add address=192.168.10.1/24 interface=HOME_VLAN network=192.168.10.0
add address=192.168.20.1/24 interface=IOT_VLAN network=192.168.20.0
add address=192.168.30.1/24 interface=TESTING_VLAN network=192.168.30.0
add address=192.168.40.1/24 interface=IOT_INTERNET_VLAN network=192.168.40.0
add address=192.168.50.1/24 interface=DMZ_VLAN network=192.168.50.0
add address=192.168.99.1/24 interface=MANAGEMENT_VLAN network=192.168.99.0
/ip dhcp-server lease
add address=192.168.99.13 client-id=*** comment=\
"Laptop ETH" mac-address=*** server=MANAGEMENT_DHCP
add address=192.168.10.4 client-id=*** comment="TV WIFI" \
mac-address=*** server=HOME_DHCP
add address=192.168.10.13 client-id=*** comment=\
"Laptop WIFI" mac-address=*** server=HOME_DHCP
add address=192.168.10.3 client-id=*** comment="Printer WIFI" \
mac-address=*** server=HOME_DHCP
add address=192.168.99.5 client-id=*** comment=\
"Unify AP AC LR ETH" mac-address=*** server=MANAGEMENT_DHCP
add address=192.168.40.2 client-id=*** comment=\
"Roborock S5 Max" mac-address=*** server=IOT_INTERNET_DHCP
/ip dhcp-server network
add address=192.168.10.0/24 dns-server=192.168.99.1 gateway=192.168.10.1
add address=192.168.20.0/24 dns-server=192.168.99.1 gateway=192.168.20.1
add address=192.168.30.0/24 dns-server=192.168.99.1 gateway=192.168.30.1
add address=192.168.40.0/24 dns-server=192.168.99.1 gateway=192.168.40.1
add address=192.168.50.0/24 dns-server=192.168.99.1 gateway=192.168.50.1
add address=192.168.99.0/24 dns-server=192.168.99.1 gateway=192.168.99.1
/ip dns
set allow-remote-requests=yes servers=8.8.8.8,8.8.4.4
/ip dns static
add address=192.168.99.1 name=router.lan
/ip firewall filter
add action=accept chain=input comment="Allow Estab & Related" \
connection-state=established,related
add action=accept chain=input comment="Allow VLAN" in-interface-list=VLAN
add action=accept chain=input comment="Allow MANAGEMENT_VLAN Full Access" \
in-interface=MANAGEMENT_VLAN
add action=drop chain=input comment=Drop
add action=accept chain=forward comment="Allow Estab & Related" \
connection-state=established,related
add action=drop chain=forward comment="Drop IOT_VLAN from Internet" \
in-interface=IOT_VLAN out-interface-list=WAN
add action=drop chain=forward comment="Drop TESTING_VLAN from Internet" \
in-interface=TESTING_VLAN out-interface-list=WAN
add action=accept chain=forward comment="VLAN Internet Access only" \
connection-state=new in-interface-list=VLAN out-interface-list=WAN
add action=drop chain=forward comment=Drop
/ip firewall nat
add action=masquerade chain=srcnat out-interface-list=WAN
/ip service
set telnet address=192.168.99.0/24 disabled=yes
set ftp address=192.168.99.0/24 disabled=yes
set www address=192.168.99.0/24
set ssh address=192.168.99.0/24
set www-ssl address=192.168.99.0/24
set api address=192.168.99.0/24
set winbox address=192.168.99.0/24
set api-ssl address=192.168.99.0/24
/ipv6 firewall address-list
add address=::/128 comment="defconf: unspecified address" list=bad_ipv6
add address=::1/128 comment="defconf: lo" list=bad_ipv6
add address=fec0::/10 comment="defconf: site-local" list=bad_ipv6
add address=::ffff:0.0.0.0/96 comment="defconf: ipv4-mapped" list=bad_ipv6
add address=::/96 comment="defconf: ipv4 compat" list=bad_ipv6
add address=100::/64 comment="defconf: discard only " list=bad_ipv6
add address=2001:db8::/32 comment="defconf: documentation" list=bad_ipv6
add address=2001:10::/28 comment="defconf: ORCHID" list=bad_ipv6
add address=3ffe::/16 comment="defconf: 6bone" list=bad_ipv6
/ipv6 firewall filter
add action=accept chain=input comment=\
"defconf: accept established,related,untracked" connection-state=\
established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
invalid
add action=accept chain=input comment="defconf: accept ICMPv6" protocol=\
icmpv6
add action=accept chain=input comment="defconf: accept UDP traceroute" port=\
33434-33534 protocol=udp
add action=accept chain=input comment=\
"defconf: accept DHCPv6-Client prefix delegation." dst-port=546 protocol=\
udp src-address=fe80::/10
add action=accept chain=input comment="defconf: accept IKE" dst-port=500,4500 \
protocol=udp
add action=accept chain=input comment="defconf: accept ipsec AH" protocol=\
ipsec-ah
add action=accept chain=input comment="defconf: accept ipsec ESP" protocol=\
ipsec-esp
add action=accept chain=input comment=\
"defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=input comment=\
"defconf: drop everything else not coming from VLAN" in-interface-list=\
!VLAN
add action=accept chain=forward comment=\
"defconf: accept established,related,untracked" connection-state=\
established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
connection-state=invalid
add action=drop chain=forward comment=\
"defconf: drop packets with bad src ipv6" src-address-list=bad_ipv6
add action=drop chain=forward comment=\
"defconf: drop packets with bad dst ipv6" dst-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: rfc4890 drop hop-limit=1" \
hop-limit=equal:1 protocol=icmpv6
add action=accept chain=forward comment="defconf: accept ICMPv6" protocol=\
icmpv6
add action=accept chain=forward comment="defconf: accept HIP" protocol=139
add action=accept chain=forward comment="defconf: accept IKE" dst-port=\
500,4500 protocol=udp
add action=accept chain=forward comment="defconf: accept ipsec AH" protocol=\
ipsec-ah
add action=accept chain=forward comment="defconf: accept ipsec ESP" protocol=\
ipsec-esp
add action=accept chain=forward comment=\
"defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=forward comment=\
"defconf: drop everything else not coming from VLAN" in-interface-list=\
!VLAN
/system clock
set time-zone-name=Europe/Warsaw
/system scheduler
add name=default-wan-on-boot on-event=set-lte2-as-wan policy=\
ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon \
start-time=startup
/system script
add dont-require-permissions=no name=set-lte2-as-wan owner=admin policy=\
ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon source=":\
delay 30\r\
\n/system routerboard usb power-reset bus=1 duration=5\r\
\n:delay 10\r\
\n/interface list member set interface=lte2 list=WAN numbers=1"
/tool mac-server
set allowed-interface-list=MANAGEMENT
/tool mac-server mac-winbox
set allowed-interface-list=MANAGEMENT
3.3. Proxmox NIC configuration
I tried to follow this:
https://pve.proxmox.com/wiki/Network_Co ... twork_vlan
but ended up with following for 1st server:
Code: Select all
root@pve1:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback
iface enp2s0 inet manual
auto vmbr0
iface vmbr0 inet static
address 192.168.99.2/24
gateway 192.168.99.1
bridge-ports enp2s0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
Code: Select all
root@pve2:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback
iface enp2s0 inet manual
auto vmbr0
iface vmbr0 inet static
address 192.168.99.3/24
gateway 192.168.99.1
bridge-ports enp2s0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
Questions
1. I thought if I have rule:
Code: Select all
/ip firewall filter
add action=accept chain=input comment="Allow MANAGEMENT_VLAN Full Access" \
in-interface=MANAGEMENT_VLAN
it'll allow me to easily access all hosts, services, etc. which are running in other VLANS via HTTP, SSH, ICMP etc., but it seems I need specific firewall rules for it which I do not like. Am I right?
2. Let's say I have cams in IOT_VLAN, but my Cam Software will be launched in DMZ_VLAN -> I assume I need to create specific rule in firewall in order to allow it, something like sharing printer in "Using RouterOS to VLAN your network" examples, yes?
3. Unify AP is not propagating stuff from DHCP from tagged VLANs with
Code: Select all
frame-types=admit-only-untagged-and-priority-tagged
Code: Select all
admit-all
4. Same story with Proxmox servers - VMs on DMZ_VLAN were not getting IPs from DHCP with config included in 3.3 and same setting as above (
Code: Select all
frame-types=admit-only-untagged-and-priority-tagged
Code: Select all
frame-types=admit-all
5. I wanted to use "tagged" MANAGEMENT_VLAN on eth4, but when I connect my laptopt (Windows 11) WSL2 has no Internet access (looks like WSL2 is not supporting tagged VLANs), but on Windows I have Internet access, etc. Is it secure as well?
6. How to block access to 192.168.99.1 (Mikrotik router)? Should I disable IP services in Mikrotik or maybe some firewall rules? If I go with firewall what about accessing DNS from all VLANs? I've limited access for both Mikrotik user and all IP services (www, winbox) to MANAGEMENT_VLAN, but don't like that still I can hit login page from e.g. TESTING_VLAN (still some injection is possible to access Mikrotik )
7. I've completly disabled FASTTRACK - not sure is it worth to use when I have VLANS. Not sure if it will be a problem when I'll launch some MQTT server, etc.
8. Will it be safe to create WIFI for MANAGEMENT_VLAN? I know I can not broadcast SSID, but there are a lot of software to find hidden SSID so I'm broadcasting all SSID. I don't think that launching MANAGEMENT WIFI is safe (without RADIUS, etc.), but for sure it'd help me a lot for management Just asking what you think
9. I tried to create setup on Proxmox as per Proxmox docs (linked in 3.3.) (thought that I can pass MANAGEMENT_VLAN as tagged not untagged) and have to bridges on physical server, but it was not working Not sure why... I have only 1 ethernet card in my server.
In near future I'll add some switch and IP cams, NAS server, but with current knowledge, it shouldn't be difficult to change setup.
Thanks a lot for any answers, suggestions and help!