simple two subnets using vlans, review please

I have a RB750Gr3 that I’m setting up to serve one trusted, and one untrusted subnet. Both have full access to the ethernet. Both are wired, but both also have an associated SSID on a single AP. (That’s what drives needing vlans, otherwise this would be a simple routing setup.) In addition, this is replacing a very similar setup with a different (aging) router, so I have some older hardware (switches) to contend with.

A picture. Pardon my drawing program – its label placement isn’t perfect. Labels on lines are all to the right of the line they’re labeling.
vlans.jpg
I have existing unmanaged switches on both the trusted and untrusted nets, so I can’t rely on (and don’t really need) a vlan tag being carried across those networks reliably.

I’ve gone through examples in other people’s posts, and read (what I think are) the relevant manual sections, but I’m stuck at this point. A device on ether3 doesn’t get an address from the DHCP server on that interface. If I watch the traffic counters, ether3 does get traffic, but the vlan10-untrusted interface does not. (BTW, there’s no DHCP server on the 211 side. That’s intentional – I have a different server for that network.)

I started from the default MikroTik config, and shrank the default bridge down to just ether1, and renamed it “bridge-safe”. So ether1 still has the .88 net on it. (I only locked myself out several times before realizing that was a good idea. :wink:

Here’s what I have:

# feb/19/2022 09:44:55 by RouterOS 6.49.3
# software id = M4E2-6RER
#
# model = RB750Gr3
# serial number = D5030F1290A5

/interface bridge
add name=bridge-mine vlan-filtering=yes
add admin-mac=DC:2C:6E:7B:22:DE auto-mac=no comment=defconf name=bridge-safe

/interface vlan
add interface=bridge-mine name=vlan10-untrusted vlan-id=10
add interface=bridge-mine name=vlan11-trusted vlan-id=11

/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN

/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik

/ip pool
add name=default-dhcp ranges=192.168.88.10-192.168.88.254
add name=untrusted-dhcp ranges=192.168.210.100-192.168.210.240

/ip dhcp-server
add address-pool=default-dhcp disabled=no interface=bridge-safe name=defconf
add address-pool=untrusted-dhcp disabled=no interface=vlan10-untrusted name=dhcp-untrusted

/interface bridge port
add bridge=bridge-safe comment=defconf interface=ether2
add bridge=bridge-mine interface=ether3 pvid=10
add bridge=bridge-mine interface=ether4
add bridge=bridge-mine interface=ether5 pvid=11

/ip neighbor discovery-settings
set discover-interface-list=LAN

/interface bridge vlan
add bridge=bridge-mine tagged=ether4 untagged=ether5 vlan-ids=11
add bridge=bridge-mine tagged=ether4 untagged=ether3 vlan-ids=10

/interface list member
add comment=defconf interface=bridge-safe list=LAN
add comment=defconf interface=ether1 list=WAN
add interface=vlan10-untrusted list=LAN
add interface=vlan11-trusted list=LAN

/ip address
add address=192.168.88.1/24 comment=defconf interface=bridge-safe network=192.168.88.0
add address=192.168.210.1/24 interface=vlan10-untrusted network=192.168.210.0
add address=192.168.211.1/24 interface=vlan11-trusted network=192.168.211.0

/ip dhcp-client
add comment=defconf disabled=no interface=ether1

/ip dhcp-server network
add address=192.168.88.0/24 comment=defconf dns-server=192.168.88.1 gateway=192.168.88.1
add address=192.168.210.0/24 dns-server=8.8.8.8,4.4.4.4 gateway=192.168.210.1

/ip dns
set allow-remote-requests=yes servers=8.8.8.8,4.4.4.4

/ip dns static
add address=192.168.88.1 comment=defconf name=router.lan

/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN

/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN

Once I have this working, I’ll need to figure out how to give the trusted net access to the untrusted side (but not vice versa), and also how to allow ssh to a single host on the trusted net, both from the untrusted net, and the WAN. So pointers to good examples of doing those things would be welcome, but getting the subnets working in the first place comes first.

Many thanks in advance!!
paul

Following up with further confirmation of the symptoms. I found the built-in packet sniffer (how cool is that, anyway?!?), and using it I can see the DHCP broadcasts arriving on ether3, but not going further. If I disable VLAN filtering on the bridge, then I also see the broadcasts arriving there (i.e., at interface “bridge-mine”).

So there’s something I don’t understand about my vlan configuration that isn’t right.

paul

Yes, you only need one bridge
Too Many other config errors to count…

For Accessing the router OFF the bridge, ie being SAFE, I wrote up this article SEE item A - https://forum.mikrotik.com/viewtopic.php?t=182373 if the below text and config dont make sense.
We will use your address 192.168.88.1 for this purpose as it already exists and assuming your wan is not 192.168.88.1 (if so use something completely different 10.0.1.1 for example )
Now all you will have to do is plug into ether2 give your laptop/desktop an IPV4 address of 192.168.88.5 and you will access the router, regardless if bridge config gets screwed up.

Fixed…

/interface bridge
add name=bridge-mine vlan-filtering=yes
/interface ethernet
set [ find default-name=ether2 ] name=ether2-access
/interface vlan
add interface=bridge-mine name=vlan10-untrusted vlan-id=10
add interface=bridge-mine name=vlan11-trusted vlan-id=11
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=untrusted-dhcp ranges=192.168.210.100-192.168.210.240
add name=trusted-dchp ranges=192.168.211.200-192.168.211.240
/ip dhcp-server
add address-pool=default-dhcp disabled=no interface=vlan11-trusted name=defconf
add address-pool=untrusted-dhcp disabled=no interface=vlan10-untrusted name=dhcp-untrusted
/interface bridge port
add bridge=bridge-mine interface=ether3 pvid=10  ingress-filtering=yes  frame-types=admit-only-untagged-and-priority-tagged
add bridge=bridge-mine interface=ether4 interface=ether2 ingress-filtering=yes  frame-types=admit-only-vlan-tagged
add bridge=bridge-mine interface=ether5 pvid=11  ingress-filtering=yes  frame-types=admit-only-untagged-and-priority-tagged
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface bridge vlan
add bridge=bridge-mine tagged=bridge,ether4 untagged=ether5 vlan-ids=11
add bridge=bridge-mine tagged=bridge,ether4 untagged=ether3 vlan-ids=10
/interface list member
add comment=defconf interface=vlan11-trusted list=LAN
add comment=defconf interface=vlan10-untrusted list=LAN
add comment interface=ether1 list=WAN
add interface=ether2-access list=LAN
/ip address
add address=192.168.210.1/24 interface=vlan10-untrusted network=192.168.210.0
add address=192.168.211.1/24 interface=vlan11-trusted network=192.168.211.0
add address=192.168.88.1/24 interface=ether2-access network=192.168.88.0
/ip dhcp-client
add comment=defconf disabled=no interface=ether1
/ip dhcp-server network
add address=192.168.210.0/24 dns-server=8.8.8.8,4.4.4.4 gateway=192.168.210.1
add address=192.168.211.0/24 dns-server=8.8.8.8,4.4.4.4 gateway=192.168.211.1
/ip dns
set allow-remote-requests=yes servers=8.8.8.8,4.4.4.4
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN

Thanks!!!

I didn’t realize that I needed to tag my vlans on the bridge, as well as on the etherX interfaces. Adding the bridge as a tagged interface in the bridge vlan config did the trick.
Works perfectly now! I knew it would turn out to be a simple misunderstanding on my part. But perhaps you can help me understand it further. I followed the example in the manual, here: https://wiki.mikrotik.com/wiki/Manual:Bridge_VLAN_Table#Trunk.2FAccess_port_setup. In the solution there, the bridge is not added as “tagged”, and in fact they say that it’s by default added as “untagged”. So I wonder why I needed this change?

As for the rest, well, “too many to count” is a little unfair, I think. :slight_smile:

The ingress-filtering on the vlans – I understand that that’s probably an important optimization on a bigger network, but I believe it’s just an optimization, right? After I get this configuration working, I hope to not have to look at it again for many years. So keeping it as simple as possible is definitely a goal. :slight_smile: Do you think this ingress-filtering is a necessity?

The DHCP server on net 211: I did say in my original post that I left it out intentionally. DHCP for that network is provided elsewhere, so it’s unnecessary here. I realize the omission looked odd, though.

I think I have no use for IP neighbor discovery, so I had changed that interface to none. Can you tell me why I’d want it?

And finally, I understand you prefer your technique for the “safe” change. But what I did was very very easy, with minimal config, which accomplishes just the same thing as your scheme, I believe. All I had to do was remove 3 ports from the default bridge. Nothing else. And then I was able to create my bridge, and not worry about it again. My technique “wastes” a bridge, “wastes” a DHCP server and, of course, “wastes” a physical port on the router, but otherwise is fine. I confess I prefer that hit has DHCP running on the “safe” port.

Thank you again for taking the time. I appreciate it!
paul

The better reference for this type of VLAN setup (bridge vlan filtering is best explained at item C. - https://forum.mikrotik.com/viewtopic.php?t=182373

Yes, its not optimization its best security practices. Not more efficient but more secure.

Yes untagging is added by default based on the use of PVID on bridge ports so not necessary in Bridge Vlans.
However, I do put them in manually like you did, otherwise they dont show up on a printout or export of the config (as they are created dynamically when needed only).
I prefer to cross check my rules with bridge port rules so I can visually see the connection.

IP neighbours discovery is very useful for winbox and admin to find other smart devices on the trusted network - thus set on the trusted interface is a good idea.

You can setup a safe port anyway you want, you can do it the hard way or my way :wink:

Thanks again. I’ll look at that page for more answers!
paul

I’m afraid things still aren’t quite right. :-/

I can now get traffic in and out of ether3 and ether5, and they can access the WAN, and they can access the “failsafe” port. So that’s all good. But no traffic ever appears on ether4, which is the trunk port that will someday connect to the AP. You can refer to the diagram in the first post, but since I’ve revised my config, I’ll include a new copy of that below.

If I connect to ether3, and commence pinging a non-existent address on the .210 network, then, using the internal packet sniffer, I can see the ARP packets on ether3, on the bridge (“bridge-mine”), and on vlan10-untrusted. But they don’t appear on ether4. (I’ve confirmed that with an external packet sniffer as well.) In addition, the packet sniffer doesn’t show the ARP packets as having a VLAN id anywhere where they do show up. I would have expected them to have had a VLAN id of 10 assigned when they entered ether3, since pvid for ether3 is set to 10. I assume that the VLAN id should be visible on vlan10-untrusted, and probably on the bridge, as well.

Again, I assume I’ve done something dumb.

Many thanks to anyone that can take a look!

My current config:

/interface bridge
add admin-mac=DC:2C:6E:7B:22:DE auto-mac=no comment=defconf name=\
    bridge-failsafe
add name=bridge-mine vlan-filtering=yes

/interface vlan
add interface=bridge-mine name=vlan10-untrusted vlan-id=10
add interface=bridge-mine name=vlan11-trusted vlan-id=11

/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN

/ip pool
add name=default-dhcp ranges=192.168.88.10-192.168.88.254
add name=untrusted-dhcp ranges=192.168.210.100-192.168.210.240
/interface ethernet
set [ find default-name=ether1 ] mac-address=C0:56:27:67:BB:1C

/ip dhcp-server
add address-pool=default-dhcp disabled=no interface=bridge-failsafe name=\
    dhcp-failsafe
add address-pool=untrusted-dhcp disabled=no interface=vlan10-untrusted name=\
    dhcp-untrusted

/interface bridge port
add bridge=bridge-failsafe comment=defconf interface=ether2
add bridge=bridge-mine interface=ether3 pvid=10
add bridge=bridge-mine interface=ether4 pvid=10
add bridge=bridge-mine interface=ether5 pvid=11

/interface bridge vlan
add bridge=bridge-mine tagged=ether4,bridge-mine untagged=ether5 \
    vlan-ids=11
add bridge=bridge-mine tagged=ether4,bridge-mine untagged=ether3 \
    vlan-ids=10

/interface list member
add comment=defconf interface=bridge-failsafe list=LAN
add comment=defconf interface=ether1 list=WAN
add interface=vlan10-untrusted list=LAN
add interface=vlan11-trusted list=LAN

/ip address
add address=192.168.88.1/24 comment=defconf interface=bridge-failsafe \
    network=192.168.88.0
add address=192.168.210.1/24 interface=vlan10-untrusted network=192.168.210.0
add address=192.168.211.1/24 interface=vlan11-trusted network=192.168.211.0

/ip dhcp-client
add comment=defconf disabled=no interface=ether1

/ip dhcp-server network
add address=192.168.88.0/24 comment=defconf dns-server=192.168.88.1 gateway=\
    192.168.88.1
add address=192.168.210.0/24 dns-server=8.8.8.8,4.4.4.4 gateway=192.168.210.1

/ip dns
set allow-remote-requests=yes servers=8.8.8.8,4.4.4.4

/ip dns static
add address=192.168.88.1 comment=defconf name=router.lan

/ip firewall filter
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    in-interface-list=!LAN
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-state=established,related
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new in-interface-list=WAN
add action=drop chain=forward comment="211 blocks 210" dst-address=\
    192.168.211.0/24 src-address=192.168.210.0/24
add action=drop chain=forward comment="88 blocks 210" dst-address=\
    192.168.88.0/24 src-address=192.168.210.0/24

/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" \
    ipsec-policy=out,none out-interface-list=WAN

There is good reason why its not working, Mainly because you decided that KISS was not good enough
and you wanted to interject needless things!!!
There is no purpose to having a second bridge for failsafe, simply not required.
There is no reasons for another dhcp server not required.

All you need is the IP address of the subnet, THATS IT.
Simply add the ether2 interface as a LAN list member and done!.
There is no reason for another bridge port entry…

In fact, by adding all this xtra SHIT, you screwed up your config… If you had not it would have been more obvious!!! :wink:
Look at this and you should spot the issue right away…

/ip dhcp-server
add address-pool=default-dhcp disabled=no interface=bridge-failsafe name=
dhcp-failsafe
add address-pool=untrusted-dhcp disabled=no interface=vlan10-untrusted name=
dhcp-untrusted

It should be:
/ip dhcp-server
add address-pool=trusted-dhcp disabled=no interface=vlan11-trusted name=
dhcp-trusted[

add address-pool=untrusted-dhcp disabled=no interface=vlan10-untrusted name=
dhcp-untrusted

AND AGAIN ARGGG

/ip dhcp-server network
add address=192.168.88.0/24 comment=defconf dns-server=192.168.88.1 gateway=
192.168.88.1
add address=192.168.210.0/24 dns-server=8.8.8.8,4.4.4.4 gateway=192.168.210.1

Should be:
/ip dhcp-server network
add address=192.168.211.0/24 comment=defconf dns-server=192.168.211.1 gateway=
192.168.211.1

add address=192.168.210.0/24 dns-server=8.8.8.8,4.4.4.4 gateway=192.168.210.1

No need for foul language. I thought we just went through this, in the posts above.

As I said before, I don’t want, or need, a DHCP server on the trusted net. The DHCP and DNS servers for that network lives somewhere else altogether, on a different machine. Please believe me.

And, I do want a DHCP server on the failsafe port, certainly while I’m connecting to it multiple times a day, in between using my laptop for other things. While maybe someday I’ll eliminate the bridge for the failsafe port, I doubt I’ll ever eliminate the DHCP server. Plus, I didn’t add the failsafe bridge. It was already there when I booted the router. It would have been more work to get rid of it.

Getting to the real issue – why do you think that the changes you’ve proposed could possibly cause the problem I described, which pretty clearly seems to me like a tagging issue?

Oh that LOL,
Okay fair enough its because you added a pvid to ether4, remove it, will work fine.

:smiley:

So, I thought PVID was only used on input, to assign a tag to untagged packets? I don’t ever expect untagged packets on that leg, since the only device will by the AP that’s configured with vlans. Not sure why I configured the PVID. In any case, sadly it didn’t help. Same symptoms as before. :-/

paul

An access port is one where you have a ‘dumb’ device attached that cannot read vlans. THus when said device sends traffic the router tags it inbound to the port and then strips the tag for the return traffic outbound back the the device.

If its not working its due to your bizarre setup.

It’s a Unifi U6-LR. It’s definitely capable of using vlans correctly.

And it’s not even connected, so it’s not part of the current problem. No packets are exiting the router on ether4 (i.e., the “trunk” line to the AP).

If the packet sniffer is to be believed (and I assume it is), then none of the packets transiting the router are tagged. That makes it seem like a problem with the inbound configuration on ether3.

I also tried doing pings from the CLI, to both a known and an unknown host (in order to force ARP broadcasts). Those packets don’t show a VLAN id either. But I’m not sure where they would have been tagged, so I don’t know if it proves anything or not.

paul

A trunk port, ether4 to a smart device does not have a PVID attached.
There is no need as the trunk carries one or more VLANS to the attached smart device.

However unifi devices sometimes require hybrid port depending on how its configured, probably its default setting.
In which case you need to setup a hybrid port.

in this case you would setup the trusted subnet as the PVID as the unifi might be expecting, and the untrusted subnet as a vlan travelling through it.

SO IN SUMMARY
IF ITS A TRUNK PORT { both vlans reach AP and no untagging or tagging by the MT }
/interface bridge port
add bridge=bridge-failsafe comment=defconf interface=ether2
add bridge=bridge-mine interface=ether3 pvid=10
add bridge=bridge-mine interface=ether4
add bridge=bridge-mine interface=ether5 pvid=11

/interface bridge vlan
add bridge=bridge-mine tagged**=ether**4,bridge-mine untagged=ether5
vlan-ids=11
add bridge=bridge-mine tagged=ether4,bridge-mine untagged=ether3
vlan-ids=10

IF ITS AN ACCESS PORT { only one vlan reaches the AP - whatever is put in for pvid, in this case the untrusted subnet }
/interface bridge port
add bridge=bridge-failsafe comment=defconf interface=ether2
add bridge=bridge-mine interface=ether3 pvid=10
add bridge=bridge-mine interface**=ether4 pvid=10**
add bridge=bridge-mine interface=ether5 pvid=11

/interface bridge vlan
add bridge=bridge-mine tagged=bridge-mine untagged=ether5
vlan-ids=11
add bridge=bridge-mine tagged=bridge-mine untagged=ether3**,ether4**
vlan-ids=10

IF ITS A HYBRID PORT { both vlans reach the AP, the trusted vlan is tagged and untagged by the MT device, the untrusted subnet flow through like it was in a trunk port }
/interface bridge port
add bridge=bridge-failsafe comment=defconf interface=ether2
add bridge=bridge-mine interface=ether3 pvid=10
add bridge=bridge-mine interface=ether4 pvid=11
add bridge=bridge-mine interface=ether5 pvid=11

/interface bridge vlan
add bridge=bridge-mine tagged=bridge-mine untagged=ether5,ether4
vlan-ids=11
add bridge=bridge-mine **tagged=ether4,**bridge-mine untagged=ether3
vlan-ids=10

Thanks. My current config is identical to your “if it’s a trunk port” section. If I need the others, because of my AP, it will be good to have the examples.

But first I have to get traffic out of ether4 at all! :slight_smile: I’ll be working on it more later today, but if anyone else has ideas!

Okay, I think that in the end my most recent config, above, was correct, and worked. My testing was thwarted by two mistaken assumptions:

First, I assumed that the built-in packet sniffer would display VLAN IDs (i.e., 802.1Q headers) at the points in the packet flow where I thought they shouldd be present. Perhaps the sniffer watches the “wrong” side of the interfaces for that, or something similar, because for whatever reason – you can’t rely on the sniffer to show you the tags (At least in my use case.) Too bad – it would have been an excellent debugging tool, rather than a huge red herring. :slight_smile:

Second, I made the incredibly poor assumption that traffic going out an interface would appear in the sniffer, and in the counters, even with nothing plugged into that interface. Clearly that was really dumb on my part. If the interface is down, nothing will flow there. “Doh!”

Between those two things, I didn’t finally decide everything was working until I’d dug out a couple of old laptops to use as network clients, and run wireshark on one of them to verify that the traffic I thought should be tagged really was.

paul

Thanks for the update. I always test with real traffic.