Hello,
Having some odd issues with bridge VLAN not working, thinking I'm either missing something basic. Come from a Juniper switch environment so I've got the basic understanding so I'm confident I'm breaking something.
The issues I'm seeing are:
1) A device plugged into ether2/ether3/ether4 does not get any dhcp. I see the dhcp discover on a wireshark dump but no answer.
2) A device plugged into ether5 (VLAN 200) is seeing ARP broadcasts from vlan100. I can also ping 192.168.88.1 (vlan100 router interface) from a device in vlan200 (on 192.168.20.254 / ether5 port) On a similar Juniper config arp requests would not be broadcast across vlans nor would an interface bound to vlan100 be directly pingable on interface vlan200.
Hardware: Hex S
Goal:
VLANS: 100,200,300,400,500
Internal IP on VLAN 100: 192.168.88.1/24 (pool 192.168.88.10-254)
Internal IP on VLAN 200: 192.168.20.1/24 (pool 192.168.20.10-254)
VLAN 300/400/500 are to carry customer tagged traffic being tagged
Ether1 = WAN uplink (this part's not really relevant/works fine)
Ether2 - Untagged (pcs/devices won't be adding any vlan tags to traffic) vlan 100, device tagged traffic allowed for vlan 200,300,400, 500.
Ether3 - Untagged (pcs/devices won't be adding any vlan tags to traffic) vlan 100, device tagged traffic allowed for vlan 200,300,400, 500.
Ether4 - Untagged (pcs/devices won't be adding any vlan tags to traffic) vlan 100, device tagged traffic allowed for vlan 200,300,400, 500.
Ether5 - Untagged (pcs/devices won't be adding any vlan tags to traffic) vlan 200, device tagged traffic allowed for vlan 100,300,400, 500.
Statements used (from a default config)
/interface bridge port remove [find where interface="ether2"]
/interface bridge port add bridge=bridge interface=ether2 pvid=100
/interface bridge port remove [find where interface="ether3"]
/interface bridge port add bridge=bridge interface=ether3 pvid=100
/interface bridge port remove [find where interface="ether4"]
/interface bridge port add bridge=bridge interface=ether4 pvid=100
/interface bridge port remove [find where interface="ether5"]
/interface bridge port add bridge=bridge interface=ether5 pvid=200
/interface bridge vlan add bridge=bridge vlan-ids=100 tagged=bridge,ether5 untagged=ether2,ether3,ether4
/interface bridge vlan add bridge=bridge vlan-ids=200 tagged=bridge,ether1,ether2,ether3,ether4 untagged=ether5
/interface bridge vlan add bridge=bridge vlan-ids=300 tagged=bridge,ether1,ether2,ether3,ether4,ether5
/interface bridge vlan add bridge=bridge vlan-ids=400 tagged=bridge,ether1,ether2,ether3,ether4,ether5
/interface bridge vlan add bridge=bridge vlan-ids=500 tagged=bridge,ether1,ether2,ether3,ether4,ether5
/interface vlan add name=vlan100 vlan-id=100 interface=bridge
/interface vlan add name=vlan200 vlan-id=200 interface=bridge
/interface vlan add name=vlan300 vlan-id=300 interface=bridge
/interface vlan add name=vlan300 vlan-id=400 interface=bridge
/interface vlan add name=vlan300 vlan-id=500 interface=bridge
/ip address set [find where address=192.168.88.1/24] interface=vlan100
/ip address add address=192.168.20.1/24 interface=vlan200
/ip dhcp-server
add address-pool=default-dhcp disabled=no interface=vlan100 name=defconf
add address-pool=vlan200 disabled=no interface=vlan200 name=vlan200
/ip dhcp-server network
add address=192.168.20.0/24 gateway=192.168.20.1
add address=192.168.88.0/24 comment=defconf gateway=192.168.88.1
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN src-address=192.168.88.0/24
add action=masquerade chain=srcnat out-interface-list=WAN src-address=192.168.20.0/24
/interface bridge set bridge vlan-filtering=yes
Full config:
# oct/12/2018 12:16:03 by RouterOS 6.43.2
# software id = GBVI-ZYLG
#
# model = RB760iGS
# serial number = 87F2093EE262
/interface bridge
add admin-mac=B8:69:F4:01:DF:4D auto-mac=no comment=defconf name=bridge vlan-filtering=yes
/interface ethernet
set [ find default-name=ether1 ] speed=100Mbps
set [ find default-name=ether2 ] speed=100Mbps
set [ find default-name=ether3 ] speed=100Mbps
set [ find default-name=ether4 ] speed=100Mbps
set [ find default-name=ether5 ] speed=100Mbps
/interface vlan
add interface=bridge name=vlan100 vlan-id=100
add interface=bridge name=vlan200 vlan-id=200
add interface=bridge name=vlan300 vlan-id=300
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile
set [ find default=yes ] html-directory=flash/hotspot
/ip pool
add name=default-dhcp ranges=192.168.88.10-192.168.88.254
add name=vlan200 ranges=192.168.20.10-192.168.20.254
/ip dhcp-server
add address-pool=default-dhcp disabled=no interface=vlan100 name=defconf
add address-pool=vlan200 disabled=no interface=vlan200 name=vlan200
/system logging action
set 3 remote=64.246.100.226
/interface bridge port
add bridge=bridge comment=defconf interface=sfp1
add bridge=bridge interface=ether2 pvid=100
add bridge=bridge interface=ether3 pvid=100
add bridge=bridge interface=ether4 pvid=100
add bridge=bridge interface=ether5 pvid=200
/ip neighbor discovery-settings
set discover-interface-list=none
/interface bridge vlan
add bridge=bridge tagged=bridge untagged=ether5 vlan-ids=200
add bridge=bridge tagged=bridge,ether1,ether2,ether3,ether4,ether5 vlan-ids=300
add bridge=bridge tagged=bridge,ether1,ether2,ether3,ether4,ether5 vlan-ids=400
add bridge=bridge tagged=bridge,ether1,ether2,ether3,ether4,ether5 vlan-ids=500
/interface ethernet switch vlan
add independent-learning=no ports=ether2,ether3,ether4,switch1-cpu switch=switch1 vlan-id=500
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=ether1 list=WAN
/ip address
add address=64.246.97.6/29 network=64.246.97.0
add address=192.168.88.1/24 interface=vlan100 network=192.168.88.0
add address=192.168.20.1/24 interface=vlan200 network=192.168.20.0
add address=192.168.30.1/24 interface=vlan300 network=192.168.30.0
/ip dhcp-client
add comment=defconf dhcp-options=hostname,clientid disabled=no interface=ether1
/ip dhcp-server network
add address=192.168.20.0/24 gateway=192.168.20.1
add address=192.168.88.0/24 comment=defconf gateway=192.168.88.1
/ip dns static
add address=192.168.88.1 name=router.lan
/ip firewall address-list
add address=censored.ips list=amplex-mgnt
add address=censored.ips list=amplex-mgnt
add address=censored.ips list=amplex-mgnt
add address=censored.ips list=amplex-mgnt
add address=censored.ips list=amplex-mgnt
add address=censored.ips list=amplex-mgnt
add address=censored.ips list=amplex-mgnt
add address=censored.ips list=amplex-mgnt
add address=censored.ips list=amplex-mgnt
add address=censored.ips list=amplex-mgnt
add address=censored.ips list=amplex-mgnt
/ip firewall filter
add action=accept chain=input comment=amplex-mgnt-in src-address-list=amplex-mgnt
add action=accept chain=forward comment=luckey-testaccount-subnet-500 dst-address=64.246.97.0/29
add action=accept chain=forward src-address=64.246.97.0/29
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related disabled=yes
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN src-address=192.168.88.0/24
add action=masquerade chain=srcnat out-interface-list=WAN src-address=192.168.20.0/24
/ip service
set telnet disabled=yes
set ftp port=8021
set www port=8080
set ssh port=8022
set winbox disabled=yes
/ip ssh
set strong-crypto=yes
/system clock
set time-zone-name=America/New_York
/system identity
set name=luckey-testaccount-mtr
/system logging
add action=remote topics=critical
add action=remote topics=error
add action=remote topics=info
add action=remote topics=warning
/system routerboard settings
set silent-boot=no
/tool bandwidth-server
set enabled=no
/tool graphing
set store-every=24hours
/tool graphing interface
add interface=ether1
add store-on-disk=no
/tool graphing resource
add
/tool mac-server
set allowed-interface-list=none
/tool mac-server mac-winbox
set allowed-interface-list=none
/tool mac-server ping
set enabled=no
Re: Bridge Vlan Help Request
'interface bridge vlan' entries specifies egress handling, in the full config listing you seem to be missing:
/interface bridge vlan
add bridge=bridge tagged=bridge untagged=ether2,ether3,ether4 vlan-ids=100
some of the 'interface bridge port' parameters handle ingress - in addition to 'pvid' for the VLAN ID to apply to untagged traffic, setting 'ingress-filtering=yes' only permits VLANs specifed in the bridge VLAN table to enter.
the 'interface ethernet switch vlan' entry is redundant and potentially confusing - currently VLAN-aware bridges disable hardware offload to the switch chip
traffic will be routed between the subnets, if you wish to isolate them you will need some firewall rules (probably drop forward with out interface list !WAN the they should only communicate with the outside world)
/interface bridge vlan
add bridge=bridge tagged=bridge untagged=ether2,ether3,ether4 vlan-ids=100
some of the 'interface bridge port' parameters handle ingress - in addition to 'pvid' for the VLAN ID to apply to untagged traffic, setting 'ingress-filtering=yes' only permits VLANs specifed in the bridge VLAN table to enter.
the 'interface ethernet switch vlan' entry is redundant and potentially confusing - currently VLAN-aware bridges disable hardware offload to the switch chip
traffic will be routed between the subnets, if you wish to isolate them you will need some firewall rules (probably drop forward with out interface list !WAN the they should only communicate with the outside world)
Re: Bridge Vlan Help Request
I cannot see any mistake in your configuration.
As for the arp broadcasting, there are two points - first, I'd also expect vlan-filtering not to let any frame (including a broadcast one) tagged with a VLAN ID which is not permitted on a port to get out through that port, but with some 6.43rc in the past, I've seen Mikrotik to happily egress tagged frames with VID "incompatible" with the port. On a different RB model, someone else has reported it to work the expected way with 6.43.2. So I'm not sure here. In my case, I had to add /interface bridge filter rules to get rid of those frames. On the other hand, it is kinda philosophical question, given how MSTP works - if you permit only some VLANs out of those belonging to an MSTP instance on some port and it later gets into forwading state because the previously working path breaks down, you'll be scratching your head trying to understand why frames tagged with the "forgotten" VLANs stopped getting through all of a sudden.
Second, and it is what will give you the ultimate answer whether your VLAN setup is working or not, is whether those arp requests get out tagged or tagless. If arp requests coming in tagless via ether5 can be seen also tagless on any of ether2-ether4, the vlan-filtering (and tagging/untagging) is broken; if you can see them tagged with VID 200, it works, only differently than you expect. However, there is a caveat - if you capture on Windows, you have to be aware that most of the network drivers strip the VLAN tag before showing the frame to winpcap, so Wireshark shows them as tagless although they aren't on the wire. So either use /tool sniffer on the 'Tik or capture using a linux device.
After running /tool sniffer on the interface where you expect the frames to egress tagged while you send the arp requests on the access interface to another VLAN, you can use /tool sniffer packet print detail where vlan~"[0-9]" to see the result directly, in my case it was:
0 time=17.877 num=509 direction=tx src-mac=64:D1:54:87:39:45 dst-mac=FF:FF:FF:FF:FF:FF vlan=125 interface=ether2 protocol=arp size=46 cpu=0 fp=no
1 time=18.874 num=538 direction=tx src-mac=64:D1:54:87:39:45 dst-mac=FF:FF:FF:FF:FF:FF vlan=125 interface=ether2 protocol=arp size=46 cpu=0 fp=no
EDIT: after removing ether2 from the tagged list in /interface bridge vlan vlan-ids=125, the arp frames did not get out the ether2. 6.43.2 on 952Ui-5ac2nD (hAP ac lite), with firmware updated to 6.42.3 as well.
This need not necessarily be caused by a VLAN issue, a dhcp server issue is equally likely. Can you configure one device in vlan100 / subnet 192.168.88.0/24 statically and another one in vlan200 / subnet 192.168.20.0/24, give both the Mikrotik's respective IP addresses as gateways, and see whether they can see each other (mind the firewalls on Mikrotik and the devices themselves of course)?1) A device plugged into ether2/ether3/ether4 does not get any dhcp. I see the dhcp discover on a wireshark dump but no answer.
Response to a ping request coming to an address not belonging to the interface through which the ping request came in is a non-issue: Mikrotik, like most other devices, will respond to ping coming to any of its IP addresses, regardless through which interface the request came. I'm even surprised that Jun doesn't.2) A device plugged into ether5 (VLAN 200) is seeing ARP broadcasts from vlan100. I can also ping 192.168.88.1 (vlan100 router interface) from a device in vlan200 (on 192.168.20.254 / ether5 port) On a similar Juniper config arp requests would not be broadcast across vlans nor would an interface bound to vlan100 be directly pingable on interface vlan200.
As for the arp broadcasting, there are two points - first, I'd also expect vlan-filtering not to let any frame (including a broadcast one) tagged with a VLAN ID which is not permitted on a port to get out through that port, but with some 6.43rc in the past, I've seen Mikrotik to happily egress tagged frames with VID "incompatible" with the port. On a different RB model, someone else has reported it to work the expected way with 6.43.2. So I'm not sure here. In my case, I had to add /interface bridge filter rules to get rid of those frames. On the other hand, it is kinda philosophical question, given how MSTP works - if you permit only some VLANs out of those belonging to an MSTP instance on some port and it later gets into forwading state because the previously working path breaks down, you'll be scratching your head trying to understand why frames tagged with the "forgotten" VLANs stopped getting through all of a sudden.
Second, and it is what will give you the ultimate answer whether your VLAN setup is working or not, is whether those arp requests get out tagged or tagless. If arp requests coming in tagless via ether5 can be seen also tagless on any of ether2-ether4, the vlan-filtering (and tagging/untagging) is broken; if you can see them tagged with VID 200, it works, only differently than you expect. However, there is a caveat - if you capture on Windows, you have to be aware that most of the network drivers strip the VLAN tag before showing the frame to winpcap, so Wireshark shows them as tagless although they aren't on the wire. So either use /tool sniffer on the 'Tik or capture using a linux device.
After running /tool sniffer on the interface where you expect the frames to egress tagged while you send the arp requests on the access interface to another VLAN, you can use /tool sniffer packet print detail where vlan~"[0-9]" to see the result directly, in my case it was:
0 time=17.877 num=509 direction=tx src-mac=64:D1:54:87:39:45 dst-mac=FF:FF:FF:FF:FF:FF vlan=125 interface=ether2 protocol=arp size=46 cpu=0 fp=no
1 time=18.874 num=538 direction=tx src-mac=64:D1:54:87:39:45 dst-mac=FF:FF:FF:FF:FF:FF vlan=125 interface=ether2 protocol=arp size=46 cpu=0 fp=no
EDIT: after removing ether2 from the tagged list in /interface bridge vlan vlan-ids=125, the arp frames did not get out the ether2. 6.43.2 on 952Ui-5ac2nD (hAP ac lite), with firmware updated to 6.42.3 as well.