I've been experimenting, and indeed using the switch chip natively increases the performance 5x (going from ~200Mbit/s when using just one interface to 1Gbit/s per each interface). So that's a great win.
Security
I suppose that VLAN filtering in bridge mode, with some combinations of frame-type are the same as the VLAN mode and VLAN header settings in switch VLANs are doing the exact same thing? There is no gap in terms of security features there as far as I can understand these options. All the other fancy features I thought were nice in the bridge VLAN setup are not needed in the switch VLAN setup ( BPDU guard, trusted ports, and more bridge related options). So I'm not loosing anything, only gaining a simpler and faster setup that's just as (or maybe more secure).
I do have some final questions though. Below is my config, which is very basic, just played with the switch VLAN stuff.
/export hide-sensitive
# jun/04/2022 14:25:18 by RouterOS 7.2.3
# software id = A1GI-TFVF
#
# model = 960PGS
# serial number = 89F90861A06A
/interface bridge
add admin-mac=CC:2D:E0:81:0A:BE auto-mac=no comment=defconf name=bridge
/interface ethernet
set [ find default-name=ether2 ] poe-out=forced-on
set [ find default-name=ether3 ] poe-out=forced-on
set [ find default-name=ether4 ] poe-out=forced-on
set [ find default-name=ether5 ] poe-out=forced-on
/interface ethernet switch port
set 0 default-vlan-id=10 vlan-header=always-strip vlan-mode=secure
set 1 default-vlan-id=10 vlan-mode=secure
set 2 default-vlan-id=10 vlan-mode=secure
set 3 default-vlan-id=10 vlan-mode=secure
set 4 default-vlan-id=10 vlan-mode=secure
set 5 default-vlan-id=10 vlan-mode=secure
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=default-dhcp ranges=192.168.88.10-192.168.88.254
/ip dhcp-server
add address-pool=default-dhcp disabled=yes interface=bridge name=defconf
/interface bridge port
add bridge=bridge comment=defconf interface=ether2
add bridge=bridge comment=defconf interface=ether3
add bridge=bridge comment=defconf interface=ether4
add bridge=bridge comment=defconf interface=ether5
add bridge=bridge comment=defconf interface=sfp1
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface ethernet switch vlan
add comment=native independent-learning=yes ports=ether1,ether2,ether3,ether4,ether5,switch1-cpu switch=switch1 vlan-id=10
add comment=management independent-learning=yes ports=ether2,ether3,ether4,ether5 switch=switch1 vlan-id=11
add comment=replication independent-learning=yes ports=ether2,ether3,ether4,ether5 switch=switch1 vlan-id=12
add comment=public independent-learning=yes ports=ether2,ether3,ether4,ether5 switch=switch1 vlan-id=13
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=ether1 list=WAN
/ip dhcp-client
add comment=defconf interface=ether1
/ip dhcp-server network
add address=192.168.88.0/24 comment=defconf dns-server=192.168.88.1 gateway=192.168.88.1 netmask=24
/ip dns
set allow-remote-requests=yes
/ip dns static
add address=192.168.88.1 comment=defconf name=router.lan
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment="defconf: drop all not coming from LAN" disabled=yes in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related hw-offload=yes
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" disabled=yes ipsec-policy=out,none out-interface-list=WAN
/ip route
add disabled=no dst-address=172.27.13.0/24 gateway=ether1 routing-table=main suppress-hw-offload=no
/ipv6 firewall address-list
add address=::/128 comment="defconf: unspecified address" list=bad_ipv6
add address=::1/128 comment="defconf: lo" list=bad_ipv6
add address=fec0::/10 comment="defconf: site-local" list=bad_ipv6
add address=::ffff:0.0.0.0/96 comment="defconf: ipv4-mapped" list=bad_ipv6
add address=::/96 comment="defconf: ipv4 compat" list=bad_ipv6
add address=100::/64 comment="defconf: discard only " list=bad_ipv6
add address=2001:db8::/32 comment="defconf: documentation" list=bad_ipv6
add address=2001:10::/28 comment="defconf: ORCHID" list=bad_ipv6
add address=3ffe::/16 comment="defconf: 6bone" list=bad_ipv6
/ipv6 firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMPv6" protocol=icmpv6
add action=accept chain=input comment="defconf: accept UDP traceroute" port=33434-33534 protocol=udp
add action=accept chain=input comment="defconf: accept DHCPv6-Client prefix delegation." dst-port=546 protocol=udp src-address=fe80::/10
add action=accept chain=input comment="defconf: accept IKE" dst-port=500,4500 protocol=udp
add action=accept chain=input comment="defconf: accept ipsec AH" protocol=ipsec-ah
add action=accept chain=input comment="defconf: accept ipsec ESP" protocol=ipsec-esp
add action=accept chain=input comment="defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=input comment="defconf: drop everything else not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop packets with bad src ipv6" src-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: drop packets with bad dst ipv6" dst-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: rfc4890 drop hop-limit=1" hop-limit=equal:1 protocol=icmpv6
add action=accept chain=forward comment="defconf: accept ICMPv6" protocol=icmpv6
add action=accept chain=forward comment="defconf: accept HIP" protocol=139
add action=accept chain=forward comment="defconf: accept IKE" dst-port=500,4500 protocol=udp
add action=accept chain=forward comment="defconf: accept ipsec AH" protocol=ipsec-ah
add action=accept chain=forward comment="defconf: accept ipsec ESP" protocol=ipsec-esp
add action=accept chain=forward comment="defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=forward comment="defconf: drop everything else not coming from LAN" in-interface-list=!LAN
/system clock
set time-zone-name=Europe/Amsterdam
/system identity
set name=rt2
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN
As you can see I have set all ports to VLAN mode secure and the VLAN header to always-strip for ether1. This is because ether1 is not suppose to receive tagged frames. Maybe it will in the future, but for now it depends on the default VLAN tag. But no matter what VLAN header config I set for the ether1, it always works. Probably because of the default VLAN membership? What would then be the best setting for this port if I don't expect tagged traffic?
The other ports have it set to leave-as-is. ether2 to ether5 are connected to Raspberry Pi's that sent out tagged frames, but also depend on the default VLAN (untagged). So in a sense these are hybrid ports? I'm a bit confused about this. When I check the documentation, I suppose the ports should be set to add-if-missing. Because then you ensure the PVID is always set? For some reason all the VLANs work just fine in the current setting, but also when setting it to add-if-missing (trunk ports). When I remove a port member, the connection breaks between those VLAN members. So the VLAN setup does work as I intend it to. I just find these extra security options a bit hard to understand.
When I check
this table it basically doesn't seem to matter if I use add-if-missing or leave-as-is. Both ended being treated as "Tagged traffic is sent out, tag is already present on ingress port" and otherwise it's dropped. Is there still some reason to chose for any of these modes?
As mentioned in the
host table section of the documentation:
"Packets without VLAN tag are treated just like if they had a VLAN tag with port default-vlan-id." This means that the secure mode works because I specified some VLAN tags, but also defined a default VLAN with number 10. And thus the following applies to my configuration and makes the secure mode work: "vlan-mode=check or secure" to be able to forward packets without VLAN tags you have to add a special entry to VLAN table with the same VLAN ID set according to default-vlan-id.
VLAN learning
I've enabled independent-learning, because it seems like it's best to keep smaller VLAN linked tables rather than one large table. Does this have any benefit? Does this improve isolation and perhaps also performance? The documentation isn't really clear about why you would turn this on or off and what you might gain or lose.
Spanning tree protocol
In the bridge VLAN setup I configured it with MSTP, which was recommended in the documentation when using a multi VLAN setup. It's unclear to me which protocol I'm using now. Maybe this is a dumb question, because of the setup I'm using now (not bridge VLAN).
Firewall
With the bridge VLAN setup I could enable the IP firewall, for also VLAN traffic. This option is now not present in the switch VLAN setup. But maybe this is a dumb questions as well and I can just always use the firewall for this. But it just needs an extra layer for bridges and therefore it's an extra option? But it's enabled regardless for the switch VLAN setup? Just want to confirm. I can of course still use a layer 3 firewall and use input and output ports. So I guess it doesn't matter.
Switch1 interface
In
documentation it states that "All switch chips have a special port that is called switchX-cpu, this is the CPU port for a switch chip, it is meant to forward traffic from a switch chip to the CPU, such a port is required for management traffic and for routing features. By default the switch chip ensures that this special CPU port is not congested and sends out Pause Frames when link capacity is exceeded to make sure the port is not oversaturated, this feature is called CPU Flow Control. Without this feature packets that might be crucial for routing or management purposes might get dropped."
However, this interface is also available for selecting VLAN membership. I already noticed that this interface is important for the default VLAN. Because when I remove switch1 from the default VLAN 10, I cannot connect to the Mikrotik device anymore. I can still reach the other devices connected to the Mikrotik. Acts this a bit the same as the bridge1 interface in the bridge? If so, can maybe someone explain how this exact works and in which situations I should include it in VLAN membership and when not. Maybe some other gotchas as well.