Simple Miktorik + Proxmox VLAN config

Dear all,

since the “new” changes to VLAN configuration I am blocked trying to configure a personal lab with Mikrotik and Proxmox. My goal is to achieve the scenario shown in this picture:
network.png
NOTE: BOCA means port.

The problem here is that Proxmox is connected with only 1 interface on port 4, therefore I need to allow tagged (ISOLAN + HISOLAN) and untagged (LAN) traffic. How can I get this configuration ?

Best regards,
Xavi

BTW, the device used is hAP Lite.

Thanks,
Xavi

Xavi your written description does not match your diagram.
You show ports 2,3,4 connecting to 3 switches.

Switch 2 feeds to your private LAN
Switch 3 feeds ISOLAN (vlan100)
Switch 4 feeds HISOLAN (vlan200)

Okay I understand in reality you only have one input for all of BOCA4
So port 4 on the router feeds (one or two managed switches) and two VLANS.
I still dont see why you need untagged on port four, your diagram does not show any private LAN on the BOCA 4???

While you figure out what I am asking or not understanding :slight_smile:

What I would do is create a single bridge on the router
bocabridge LOL
create the two vlan interfaces with interface of bridge
create ip address structure for vlans with interface of vlans
create ip pool
create dhcp-server with interface of vlans
create dhcp-server network

Will use default bridge of pvid=1 to our benefit and
create the private lan and attach it to the bridge (192.168.90.x)
ip address… interface=bridge
ip dhcp-server interface=bridge

Create the bridge port structure (ingress behaviour of vlans especially if access ports)
/interface bridge port
add bridge=bocabridge comment=defconf interface=ether2 (pvid=1 default)
add bridge=bocabridge comment=defconf interface=ether4 (trunk port)

Create the interface vlan structure
/interface bridge vlan
add bridge=bocabridge tagged=bocabridge,ether4 vlan-ids=100,200

That should get you well on the way…
The way I am migrating too is not to use pvid=1 for any vlans as it gets confusing.
But the answer to my above questions will help us decide what is the best direction.
In the meantime please read this link with examples for best practices…
http://forum.mikrotik.com/t/using-routeros-to-vlan-your-network/126489/1

Hi, sorry, as you stated, what I am looking for is not completely shown in the diagram. As a context, this “project” is for a vocational school trying to build a very simple configuration in order to allow them to play with unsafe software in two levels (**):

  • ISOLAN (vlan 100): must be fully isolated, no communication from ISOLAN can be made outside (Internet and LAN). Even though, traffic from LAN can achieve any VM in ISOLAN. Traffic between VMs in the same VLAN must be possible as well.
  • HISOLAN (vlan 200): It’s the same scenario than ISOLAN but allowing only DNS, HTTP, HTTPS queries to Internet.

In the simplest scenario I want:

  • ether1: is connected to Internet.
  • ether2-ether4: are LAN connections. But, ether4 is directly connected to a Proxmox.

Afterwards, the Mikrotik firewall will apply the specifications defined before (**).

Proxmox:
The Proxmox machine, for the lab, will be working in a Virtualbox with only one bridge mode NIC. Proxmox, internally, creates the vmbr0 bridge connected to the (virtualbox bridge mode) virtual NIC and all the new containers or VMs will be created in this bridge using either the tags 100 (ISOLAN) or 200 (HISOLAN).

Therefore, port 4 requires untagged traffic to provide connectivity to the real machine, the Proxmox itself and requires Mikrotik being able to route and firewall the packets from tag 100 and 200.

I am not sure if this text was useful, but I will be happy to provide any further explanation.

Best regards,
Xavi

Well, the promox bit is zing over my head but since your name is one of my favourite Barca footballers I will do my best with the rest…

Hi,

just aiming to help to find the proper solution. This is the script I used before the “new” VLAN scheme.

Best regards,
Xavi

/interface bridge
add name=bridge-HISOLAN-200 protocol-mode=none
add name=bridge-ISOLAN-100 protocol-mode=none
add name=bridge-LAN protocol-mode=none
/interface vlan
add interface=ether4 name=vlan-ether4-HISOLAN-200 vlan-id=200
add interface=ether4 name=vlan-ether4-ISOLAN-100 vlan-id=100
/interface list
add name=WAN
add name=LAN
add name=ISOLAN
add name=HISOLAN
/ip pool
add name=pool-LAN ranges=192.168.90.100-192.168.90.200
add name=pool-ISOLAN-100 ranges=192.168.100.100-192.168.100.200
add name=pool-HISOLAN-200 ranges=192.168.200.100-192.168.200.200
/ip dhcp-server
add address-pool=pool-LAN disabled=no interface=bridge-LAN lease-time=1w name=\
    dhcp-server-LAN
add address-pool=pool-ISOLAN-100 disabled=no interface=bridge-ISOLAN-100 \
    lease-time=1w name=dhcp-server-ISOLAN
add address-pool=pool-HISOLAN-200 disabled=no interface=bridge-HISOLAN-200 \
    lease-time=1w name=dhcp-server-HISOLAN
/interface bridge port
add bridge=bridge-LAN interface=ether2
add bridge=bridge-LAN interface=ether3
add bridge=bridge-ISOLAN-100 interface=vlan-ether4-ISOLAN-100
add bridge=bridge-HISOLAN-200 interface=vlan-ether4-HISOLAN-200
add bridge=bridge-LAN interface=ether4
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface list member
add interface=ether1 list=WAN
add interface=bridge-LAN list=LAN
add interface=bridge-ISOLAN-100 list=ISOLAN
add interface=bridge-HISOLAN-200 list=HISOLAN
/ip address
add address=192.168.90.1/24 interface=bridge-LAN network=192.168.90.0
add address=192.168.200.1/24 interface=bridge-HISOLAN-200 network=192.168.200.0
add address=192.168.100.1/24 interface=bridge-ISOLAN-100 network=192.168.100.0
/ip dhcp-client
add dhcp-options=hostname,clientid disabled=no interface=ether1
/ip dhcp-server network
add address=192.168.90.0/24 dns-server=8.8.8.8 domain=lomio.lan gateway=\
    192.168.90.1 netmask=24
add address=192.168.100.0/24 dns-server=8.8.8.8 domain=lomio.isolan gateway=\
    192.168.100.1 netmask=24
add address=192.168.200.0/24 dns-server=8.8.8.8 domain=lomio.hisolan gateway=\
    192.168.200.1 netmask=24
/ip firewall filter
add action=accept chain=input connection-state=established,related,untracked
add action=drop chain=input connection-state=invalid
add action=accept chain=input protocol=icmp
add action=drop chain=input in-interface-list=!LAN
add action=fasttrack-connection chain=forward connection-state=\
    established,related
add action=accept chain=forward connection-state=established,related,untracked
add action=drop chain=forward connection-state=invalid
add action=drop chain=forward connection-nat-state=!dstnat connection-state=new \
    in-interface-list=WAN
add action=drop chain=forward in-interface-list=ISOLAN
add action=accept chain=forward dst-port=53 in-interface-list=HISOLAN protocol=\
    udp
add action=accept chain=forward in-interface-list=HISOLAN protocol=icmp
add action=accept chain=forward dst-port=80,443 in-interface-list=HISOLAN \
    protocol=tcp
add action=drop chain=forward in-interface-list=HISOLAN
/ip firewall nat
add action=masquerade chain=srcnat out-interface-list=WAN
/tool mac-server
set allowed-interface-list=none
/tool mac-server mac-winbox
set allowed-interface-list=LAN
/tool mac-server ping
set enabled=no

Okay,
(1) Change both to bridge for interface
/interface vlan
add interface=ether4 name=vlan-ether4-HISOLAN-200 vlan-id=200
add interface=ether4 name=vlan-ether4-ISOLAN-100 vlan-id=100

add interface=bridge-LAN name=vlan-ether4-HISOLAN-200 vlan-id=200
add interface=bridge-LAN name=vlan-ether4-ISOLAN-100 vlan-id=100

(2) Delete the two separate bridges you made for hisoland and isolan

(3) Bridge ports, besides the ether2 and ether3, only need to keep the last rule modify 4 as follows
add bridge=bridge-ISOLAN-100 interface=vlan-ether4-ISOLAN-100
add bridge=bridge-HISOLAN-200 interface=vlan-ether4-HISOLAN-200
add bridge=bridge-LAN interface=ether4
To:
add bridge=bridge-LAN interface=ether4
Note: None of the etherports are access ports to my knowledge (all are trunk ports).

(4) Sever setup, change interfaces to vlan interface…
/ip dhcp-server
add address-pool=pool-LAN disabled=no interface=bridge-LAN lease-time=1w name=
dhcp-server-LAN
add address-pool=pool-ISOLAN-100 disabled=no interface=bridge-ISOLAN-100
lease-time=1w name=dhcp-server-ISOLAN
add address-pool=pool-HISOLAN-200 disabled=no interface=bridge-HISOLAN-200
lease-time=1w name=dhcp-server-HISOLAN
To:
/ip dhcp-server
add address-pool=pool-LAN disabled=no interface=bridge-LAN lease-time=1w name=
dhcp-server-LAN
add address-pool=pool-ISOLAN-100 disabled=no interface=vlan-ether4-ISOLAN-100
lease-time=1w name=dhcp-server-ISOLAN
add address-pool=pool-HISOLAN-200 disabled=no interface=vlan-ether4-HISOLAN-200
lease-time=1w name=dhcp-server-HISOLAN

(5) Interface (remove hiso and iso entries)
add vlanmembers

(6) Interface list members (modify)
add interface=bridge-ISOLAN-100 list=vlanmembers
add interface=bridge-HISOLAN-200 list=vlanmembers

(7) Modify IP addresses.
/ip address
add address=192.168.90.1/24 interface=bridge-LAN network=192.168.90.0
add address=192.168.200.1/24 interface=bridge-HISOLAN-200 network=192.168.200.0
add address=192.168.100.1/24 interface=bridge-ISOLAN-100 network=192.168.100.0
To:
/ip address
add address=192.168.90.1/24 interface=bridge-LAN network=192.168.90.0
add address=192.168.200.1/24 interface=vlan-ether4-HISOLAN-200 network=192.168.200.0
add address=192.168.100.1/24 interface=vlan-ether4-ISOLAN-100 network=192.168.100.0

Next have to fix your dns being all over the map and clean up firewall rules.

(8) Instead of drop rule in input chain… the purpose is more clearly to ONLY allow you the admin to access the router.
Thus delete it and instead put these…
add chain=input action=accept in-interface=bridge-LAN source-address-list=adminaccessonly
add (at end of input chain) chain=input action=drop comment=“Drop all else”
(create the necessary firewall address list consisting of your pc workstation and laptop etc… or perhaps an IP range or the lan subnet your choice).
(dont forget this should be also similar to your winbox settings…)

(9) After the admin access rule, add the following allow rules for DNS so that all other users can ONLY access the router for DNS.
add chain=input action=accept in-interface-list=LAN protocol=tcp/udp port=53
connection-state=new

(10) Adjust DHCP-server network rules so that both DNS and gateway are the main subnet IP

(11) add allow IP remote requests and add 8.8.8.8, 8.8.4.4 and perhaps 1.1.1.1 etc…

(12) Add the accept ICMP rule on the input chain.
add action=accept chain=input comment=“defconf: accept ICMP” protocol=icmp

(13) Remove DNS and ICMP rules you have on the forward chain.

(14) a. Remove these rules
add action=drop chain=forward in-interface-list=ISOLAN
add action=drop chain=forward in-interface-list=HISOLAN

b. keep but modify this this rule.
add action=accept chain=forward dst-port=80,443 in-interface-list=HISOLAN
protocol=tcp
to:
add action=accept chain=forward dst-port=80,443 protocol=tcp
in-interface
=vlan-ether4-HISOLAN-200
out-interface-list=WAN

(15) Add this rule at end of forward chain
add action=drop chain=forward comment=“Drop all else”

(16) Add this rule to the forward chain
add action=accept chain=forward in-interface=bridge-LAN source address=192.168.90.0/24
out-interface-list=WAN comment=“Allow LAN to WAN”

First of all, thanks a lot for your time!

Secondly, I tried to adapt your comments to the script posted before and I merged your comments as comments in the script. To simplify the scenario I prefer to delay the firewall modifications and I just changed to the default rules without IPSec rules. Once it works I’ll continue improving the firewall.

With this new configuration (code shown below) I have the next data:
*) Apparently DHCP protocol works properly.
*) There is no communication after the dhcp: no ping to 192.168.200.1, no ping between VLANS and LAN nodes, no ping to internet 8.8.8.8
*) I can see traffic in the vlan-bridge-LAN-HISOLAN.



/interface bridge
add name=bridge-LAN protocol-mode=none

/interface vlan
add interface=bridge-LAN name=vlan-bridge-LAN-HISOLAN-200 vlan-id=200
add interface=bridge-LAN name=vlan-bridge-LAN-ISOLAN-100 vlan-id=100

/interface list
add name=WAN
add name=LAN
add name=VLAN-MEMBERS

/ip pool
add name=pool-LAN ranges=192.168.90.100-192.168.90.200
add name=pool-ISOLAN-100 ranges=192.168.100.100-192.168.100.200
add name=pool-HISOLAN-200 ranges=192.168.200.100-192.168.200.200

/ip dhcp-server
add address-pool=pool-LAN disabled=no interface=bridge-LAN lease-time=1w name=\
    dhcp-server-LAN
add address-pool=pool-ISOLAN-100 disabled=no interface=vlan-bridge-LAN-ISOLAN-100 \
    lease-time=1w name=dhcp-server-ISOLAN
add address-pool=pool-HISOLAN-200 disabled=no interface=vlan-bridge-LAN-HISOLAN-200 \
    lease-time=1w name=dhcp-server-HISOLAN

/interface bridge port
add bridge=bridge-LAN interface=ether2
add bridge=bridge-LAN interface=ether3
add bridge=bridge-LAN interface=ether4


/ip neighbor discovery-settings
set discover-interface-list=LAN

/interface list member
add interface=ether1 list=WAN
add interface=bridge-LAN list=LAN
add interface=vlan-bridge-LAN-ISOLAN-100 list=VLAN-MEMBERS
add interface=vlan-bridge-LAN-HISOLAN-200 list=VLAN-MEMBERS

/ip address
add address=192.168.90.1/24 interface=bridge-LAN network=192.168.90.0
add address=192.168.200.1/24 interface=vlan-bridge-LAN-HISOLAN-200 network=192.168.200.0
add address=192.168.100.1/24 interface=vlan-bridge-LAN-ISOLAN-100 network=192.168.100.0

/ip dhcp-client
add dhcp-options=hostname,clientid disabled=no interface=ether1

/ip dhcp-server network
add address=192.168.90.0/24 dns-server=8.8.8.8 domain=lomio.lan gateway=\
    192.168.90.1 netmask=24
add address=192.168.100.0/24 dns-server=8.8.8.8 domain=lomio.isolan gateway=\
    192.168.100.1 netmask=24
add address=192.168.200.0/24 dns-server=8.8.8.8 domain=lomio.hisolan gateway=\
    192.168.200.1 netmask=24



# Firewall rules are redefined with the default settings (except IPSec stuff). Once VLANs works I'll apply them.
# -------------------
 /ip firewall {
   filter add chain=input action=accept connection-state=established,related,untracked comment="defconf: accept established,related,untracked"
   filter add chain=input action=drop connection-state=invalid comment="defconf: drop invalid"
   filter add chain=input action=accept protocol=icmp comment="defconf: accept ICMP"
   filter add chain=input action=drop in-interface-list=!LAN comment="defconf: drop all not coming from LAN"
   filter add chain=forward action=fasttrack-connection connection-state=established,related comment="defconf: fasttrack"
   filter add chain=forward action=accept connection-state=established,related,untracked comment="defconf: accept established,related, untracked"
   filter add chain=forward action=drop connection-state=invalid comment="defconf: drop invalid"
   filter add chain=forward action=drop connection-state=new connection-nat-state=!dstnat in-interface-list=WAN comment="defconf:  drop all from WAN not DSTNATed"
 }


/ip firewall nat
add action=masquerade chain=srcnat out-interface-list=WAN

/tool mac-server
set allowed-interface-list=none

/tool mac-server mac-winbox
set allowed-interface-list=LAN

/tool mac-server ping
set enabled=no

Please actually post your config not the replication of what I suggested its very confusing…
I need to see a clean latest config.

Hi again,

I’ve been sniffing some packets from the proxmox machine:

  • (WORKS) DHCP query/reply
root@pve:~# tcpdump -v  -e -i enp0s3 -n vlan and port 67 or 68
tcpdump: listening on enp0s3, link-type EN10MB (Ethernet), capture size 262144 bytes
11:47:16.040578 4a:9e:1b:95:5c:29 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 346: vlan 100, p 0, ethertype IPv4, (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 4a:9e:1b:95:5c:29, length 300, xid 0x50298735, Flags [none]
          Client-Ethernet-Address 4a:9e:1b:95:5c:29
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message Option 53, length 1: Request
            Requested-IP Option 50, length 4: 192.168.100.200
            Hostname Option 12, length 15: "isolan-100-test"
            Parameter-Request Option 55, length 13:
              Subnet-Mask, BR, Time-Zone, Default-Gateway
              Domain-Name, Domain-Name-Server, Option 119, Hostname
              Netbios-Name-Server, Netbios-Scope, MTU, Classless-Static-Route
              NTP
11:47:16.043501 cc:2d:e0:1b:45:b9 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 346: vlan 100, p 0, ethertype IPv4, (tos 0x0, ttl 16, id 0, offset 0, flags [none], proto UDP (17), length 328)
    192.168.100.1.67 > 255.255.255.255.68: BOOTP/DHCP, Reply, length 300, xid 0x50298735, Flags [none]
          Your-IP 192.168.100.200
          Server-IP 192.168.100.1
          Client-Ethernet-Address 4a:9e:1b:95:5c:29
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message Option 53, length 1: ACK
            Server-ID Option 54, length 4: 192.168.100.1
            Lease-Time Option 51, length 4: 604800
            Subnet-Mask Option 1, length 4: 255.255.255.0
            Default-Gateway Option 3, length 4: 192.168.100.1
            Domain-Name Option 15, length 12: "lomio.isolan"
            Domain-Name-Server Option 6, length 4: 8.8.8.8
            NTP Option 42, length 8: 178.255.228.77,109.74.206.120
  • (FAIL) Ping to 192.168.100.1
root@pve:~# tcpdump -v  -e -i enp0s3 -n vlan
tcpdump: listening on enp0s3, link-type EN10MB (Ethernet), capture size 262144 bytes
11:49:58.494482 4a:9e:1b:95:5c:29 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 100, p 0, ethertype ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.100.1 tell 192.168.100.200, length 28
11:49:59.498697 4a:9e:1b:95:5c:29 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 100, p 0, ethertype ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.100.1 tell 192.168.100.200, length 28
11:50:00.522806 4a:9e:1b:95:5c:29 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 100, p 0, ethertype ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.100.1 tell 192.168.100.200, length 28
  • (FAIL) Ping from mikrotik to 192.168.100.200 using the GUI (Tools->Ping) through the “vlan-bridge-LAN-ISOLAN-100” interface. Only these two packets below.
root@pve:~# tcpdump -v  -e -i enp0s3 -n vlan
tcpdump: listening on enp0s3, link-type EN10MB (Ethernet), capture size 262144 bytes
11:51:33.783534 cc:2d:e0:1b:45:b9 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 64: vlan 100, p 0, ethertype ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.100.200 tell 192.168.100.1, length 46
11:51:33.783624 4a:9e:1b:95:5c:29 > cc:2d:e0:1b:45:b9, ethertype 802.1Q (0x8100), length 46: vlan 100, p 0, ethertype ARP, Ethernet (len 6), IPv4 (len 4), Reply 192.168.100.200 is-at 4a:9e:1b:95:5c:29, length 28
  • (FAIL) Ping from mikrotik to 192.168.100.200 using the GUI (Tools->Ping).
    Does not show anything.
  • Why the GUI ping without specifying the device does not follows the same route ?, the route table is as follows:
[admin@MikroTik] /ip route> print 
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, 
B - blackhole, U - unreachable, P - prohibit 
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 ADS  0.0.0.0/0                          172.19.20.1               1
 1 ADC  172.19.20.0/24     172.19.20.10    ether1                    0
 2 ADC  192.168.90.0/24    192.168.90.1    bridge-LAN                0
 3 ADC  192.168.100.0/24   192.168.100.1   vlan-bridge-LAN...        0
 4 ADC  192.168.200.0/24   192.168.200.1   vlan-bridge-LAN...        0

Again, thanks a lot for your time!
Xavi

/export hide-sensitive file=yourlatestconfig

Oks, I didn’t know about hide-sensitive, really useful!
yourlatestconfig.rsc (3.46 KB)

  1. Missing /interface bridge vlan rules…

/interface bridge vlan
add bridge=bridge-LAN tagged=bridge-LAN,ether4 vlan-ids=100,200
(assuming only ether4 requires to be trunked with vlans at the moment)

  1. DNS is different from I am use too but if it works for you…

Damn, Still does not work.

I guessed [1] it required also to activate the /interface bridge vlan-filtering but neither in this operation mode worked.

I am absolutely lost.

Were you able to run this use case ?

Do you want me to sniff or debug in part of it ?

[1] - https://wiki.mikrotik.com/wiki/Manual:Basic_VLAN_switching

No worries… lets keep at it.
Post your latest config…

Ops, I forgot it, sorry.
yourlatestconfig-with-bridge-vlan-rule.rsc (3.55 KB)

While I am looking at this you should update your OS to 6.44!

Is the first switch on ether2 a managed switch??

Hi,

ok, updated but still does not work.
On ether2, ether3 there is nothing connected.

  • ether1: 172.19.20.0/24 which provides internet
  • ether4: proxmox

Regards,
Xavi

May be it can help: find attached a capture of the wireshark using Packet Sniffer from the interface “vlan-bridge-LAN-ISOLAN-100” and the current config.
current-config.rsc (4.17 KB)

I dont see any rule for your LAN to WAN (assuming no vlan to wan required)
I dont see any rule for admin access to vlans either.
the rest looks okay argggg??

Also I am assuming that the managed switch between the devices and the router is marking the packets!!
(ie its ports to the devices are access ports).