Router & Switch intercommunication only partially functional

RouterOS
1

Mikrotik hardware and ROS have opened an interesting and fun rabbit hole for me!

I have acquired 2 L009 devices and am playing with them in a test setup before moving them to my home network. They both run ROS 7.19.4 with the appropriate matching firmware updates. One (router-1) is set up to provide router + access ports and the other (switch-1) to only provide switch functionality. The sfp ports provide the inter-device trunk. Ether1 on the router is the WAN port and is connected as a dhcp client to my ISP fiber via an ONT that provides basic firewall services. Router-1 has the default Mikrotik firewall active. For now, I have off bridge ports configured on both devices. With both devices in their default state (no vlans setup) all functions well. But with my current vlan setup there are problems.

Router-1:

Physical Function/VLAN VLAN ID Subnet
ether1 WAN dhcp client
ether2 OffBridge 192.168.75.0/28
ether3 Management 99 192.168.0.0/24
ether4 & 5 PC 10 10.0.10.0/24
ether6 Guest 20 10.0.20.0/24
ether7 & 8 IOT 30 10.0.30.0/24
sfp Trunk 1

Switch-1:

Physical Function/VLAN VLAN ID Subnet
ether1 Management 99 192.168.0.0/24
ether2 & 3 PC 10 10.0.10.0/24
ether4 & 5 Guest 20 10.0.20.0/24
ether6 & 7 IOT 30 10.0.30.0/24
ether8 OffBridge 192.168.76.0/28
sfp Trunk 1

With the current setup Router-1 seems to function mostly as expected. I have access to the web with ping and with a browser from the various vlan access ports. Curiously nslookup does not work on the PC, Guest & IOT vlans. The vlans are properly segregated. The management port on the router can open WinBox on the router and also on the switch.

Switch-1 can open WinBox on the switch and on the router from the management port. From the management port only pings 192.168.0.1 and to the web succeed. The browser succeeds but nslookup fails. From the PC vlan ports pings to all *.1 ips in the LAN work as do pings to the web (eg ping 1.1.1.1). However, nslookup yahoo.com fails. The browser succeeds in reaching yahoo.com. Within a vlan (eg vlan id 10) one host cannot ping another. DHCP seems to function on both the router and the switch.

These behaviors show that I clearly do not have the two devices correctly setup. The exported setups for each are below. If requested I can upload the verbose versions of the exported files.

Can someone suggest where I have gone wrong?

Switch-1

# 2025-07-29 19:36:47 by RouterOS 7.19.4
# software id = IZ2D-1YF9
#
# model = L009UiGS
# serial number = HHG0A5XTN3E

/interface bridge
add name=BR1 vlan-filtering=yes

/interface vlan
add interface=BR1 name=Guest vlan-id=20
add interface=BR1 name=IOT vlan-id=30
add interface=BR1 name=Management vlan-id=99
add interface=BR1 name=PC vlan-id=10

/interface list
add name=OFFBR
add name=LAN
add include=OFFBR,LAN name=OFFBRandLAN

/ip pool
add name=OffBR-Pool-L009 ranges=192.168.76.2-192.168.76.14

/ip dhcp-server
add address-pool=OffBR-Pool-L009 interface=ether8 name=OffBridge-dhcp-serv

/port
set 0 name=serial0

/interface bridge port
add bridge=BR1 frame-types=admit-only-untagged-and-priority-tagged interface=\
    ether1 pvid=99
add bridge=BR1 frame-types=admit-only-untagged-and-priority-tagged interface=\
    ether2 pvid=10
add bridge=BR1 frame-types=admit-only-untagged-and-priority-tagged interface=\
    ether3 pvid=10
add bridge=BR1 frame-types=admit-only-untagged-and-priority-tagged interface=\
    ether4 pvid=10
add bridge=BR1 frame-types=admit-only-untagged-and-priority-tagged interface=\
    ether5 pvid=20
add bridge=BR1 frame-types=admit-only-untagged-and-priority-tagged interface=\
    ether6 pvid=20
add bridge=BR1 frame-types=admit-only-untagged-and-priority-tagged interface=\
    ether7 pvid=30
add bridge=BR1 frame-types=admit-only-vlan-tagged interface=sfp1

/ip neighbor discovery-settings
set discover-interface-list=OFFBRandLAN

/interface bridge vlan
add bridge=BR1 comment=Management tagged=sfp1,BR1 untagged=ether1 vlan-ids=99
add bridge=BR1 comment=PC tagged=sfp1 vlan-ids=10
add bridge=BR1 comment=Guest tagged=sfp1 vlan-ids=20
add bridge=BR1 comment=IOT tagged=sfp1 vlan-ids=30

/interface list member
add interface=ether8 list=OFFBR
add interface=BR1 list=LAN
add interface=Management list=LAN

/ip address
add address=192.168.76.1/28 comment="Off Bridge" interface=ether8 network=\
    192.168.76.0
add address=192.168.0.2/24 comment=Management interface=Management network=\
    192.168.0.0
add address=192.168.88.2/24 interface=BR1 network=192.168.88.0

/ip dhcp-server network
add address=192.168.76.0/28 dns-server=192.168.76.1 gateway=192.168.76.1

/ip route
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=sfp1 routing-table=\
    main scope=30 suppress-hw-offload=no target-scope=10
add disabled=no dst-address=10.0.10.0/24 gateway=PC routing-table=main \
    suppress-hw-offload=no

/system clock
set time-zone-name=America/Chicago

/system identity
set name=Switch-1

/system routerboard settings
set auto-upgrade=yes enter-setup-on=delete-key

/tool mac-server
set allowed-interface-list=OFFBRandLAN

/tool mac-server mac-winbox
set allowed-interface-list=OFFBRandLAN

Router-1

# 2025-09-05 09:34:00 by RouterOS 7.19.4
# software id = 2Y0J-5V4S
#
# model = L009UiGS
# serial number = HGA09ZQDNJ0

/interface bridge
add admin-mac=D4:01:C3:68:A9:1F auto-mac=no comment=BR1 name=BR1 \
    port-cost-mode=short vlan-filtering=yes

/interface ethernet
set [ find default-name=ether1 ] comment=WAN
set [ find default-name=ether2 ] comment="Off Bridge"

/interface vlan
add interface=BR1 name=Guest vlan-id=20
add interface=BR1 name=IOT vlan-id=30
add interface=BR1 name=Management vlan-id=99
add interface=BR1 name=PC vlan-id=10

/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
add name=OFFBR
add include=OFFBR,LAN name=OFFBRandLAN

/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik

/ip pool
add name=OffBridge-Pool ranges=192.168.75.3-192.168.75.14
add name=PC-Pool ranges=10.0.10.10-10.0.10.254
add name=Guest-Pool ranges=10.0.20.10-10.0.20.254
add name=IOT-Pool ranges=10.0.30.10-10.0.30.254
add name=Management-Pool ranges=192.168.0.10-192.168.0.254

/ip dhcp-server
add address-pool=OffBridge-Pool comment="Off Bridge" interface=ether2 name=\
    OffBridge-dhcp-serv
add address-pool=PC-Pool comment=PC interface=PC name=PC-dhcp-serv
add address-pool=Guest-Pool comment=Guest interface=Guest name=\
    Guest-dhcp-serv
add address-pool=IOT-Pool comment=IOT interface=IOT name=IOT-dhcp-serv
add address-pool=Management-Pool comment=Management interface=Management \
    name=Management-dhcp-serv

/port
set 0 name=serial0

/interface bridge port
add bridge=BR1 comment=defconf frame-types=\
    admit-only-untagged-and-priority-tagged interface=ether3 \
    internal-path-cost=10 path-cost=10 pvid=99
add bridge=BR1 comment=defconf frame-types=\
    admit-only-untagged-and-priority-tagged interface=ether4 \
    internal-path-cost=10 path-cost=10 pvid=10
add bridge=BR1 comment=defconf frame-types=\
    admit-only-untagged-and-priority-tagged interface=ether5 \
    internal-path-cost=10 path-cost=10 pvid=10
add bridge=BR1 comment=defconf frame-types=\
    admit-only-untagged-and-priority-tagged interface=ether6 \
    internal-path-cost=10 path-cost=10 pvid=20
add bridge=BR1 comment=defconf frame-types=\
    admit-only-untagged-and-priority-tagged interface=ether7 \
    internal-path-cost=10 path-cost=10 pvid=30
add bridge=BR1 comment=defconf frame-types=\
    admit-only-untagged-and-priority-tagged interface=ether8 \
    internal-path-cost=10 path-cost=10 pvid=30
add bridge=BR1 comment=defconf frame-types=admit-only-vlan-tagged interface=\
    sfp1 internal-path-cost=10 path-cost=10

/ip firewall connection tracking
set udp-timeout=10s

/ip neighbor discovery-settings
set discover-interface-list=OFFBRandLAN

/ip settings
set max-neighbor-entries=14336

/ipv6 settings
set max-neighbor-entries=7168

/interface bridge vlan
add bridge=BR1 comment=IOT tagged=sfp1,BR1 vlan-ids=30
add bridge=BR1 comment=Guest tagged=sfp1,BR1 vlan-ids=20
add bridge=BR1 comment=PC tagged=sfp1,BR1 vlan-ids=10
add bridge=BR1 comment=Management tagged=sfp1,BR1 untagged=ether3 vlan-ids=99

/interface list member
add comment=defconf interface=BR1 list=LAN
add comment=defconf interface=ether1 list=WAN
add interface=ether2 list=OFFBR
add interface=Management list=LAN

/interface ovpn-server server
add mac-address=FE:17:CA:6F:03:26 name=ovpn-server1

/ip address
add address=192.168.88.1/24 comment=defconf interface=BR1 network=\
    192.168.88.0
add address=192.168.75.1/28 comment="Off Bridge" interface=ether2 network=\
    192.168.75.0
add address=10.0.10.1/24 comment=PC interface=PC network=10.0.10.0
add address=10.0.20.1/24 comment=Guest interface=Guest network=10.0.20.0
add address=10.0.30.1/24 comment=IOT interface=IOT network=10.0.30.0
add address=192.168.0.1/24 comment=Management interface=Management network=\
    192.168.0.0

/ip dhcp-client
add comment=defconf interface=ether1 use-peer-dns=no

/ip dhcp-server
add address-pool=*1 interface=BR1 lease-time=10m name=defconf

/ip dhcp-server network
add address=10.0.10.0/24 comment=PC gateway=10.0.10.1
add address=10.0.20.0/24 comment=Guest gateway=10.0.20.1
add address=10.0.30.0/24 comment=IOT gateway=10.0.30.1
add address=192.168.0.0/24 comment=Management gateway=192.168.0.1
add address=192.168.75.0/28 comment="Off Bridge" dns-server=1.1.1.1,8.8.8.8 \
    gateway=192.168.75.1

/ip dns
set allow-remote-requests=yes servers=1.1.1.1,1.0.0.1

/ip dns static
add address=192.168.88.1 comment=defconf name=router.lan type=A

/ip firewall filter
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment=\
    "defconf: accept to local loopback (for CAPsMAN)" dst-address=127.0.0.1
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
    ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
    ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" \
    connection-state=established,related hw-offload=yes
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop all from WAN not DSTNATed" connection-nat-state=!dstnat \
    connection-state=new in-interface-list=WAN

/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" \
    ipsec-policy=out,none out-interface-list=WAN

/ip ipsec profile
set [ find default=yes ] dpd-interval=2m dpd-maximum-failures=5

/ipv6 firewall address-list
add address=::/128 comment="defconf: unspecified address" list=bad_ipv6
add address=::1/128 comment="defconf: lo" list=bad_ipv6
add address=fec0::/10 comment="defconf: site-local" list=bad_ipv6
add address=::ffff:0.0.0.0/96 comment="defconf: ipv4-mapped" list=bad_ipv6
add address=::/96 comment="defconf: ipv4 compat" list=bad_ipv6
add address=100::/64 comment="defconf: discard only " list=bad_ipv6
add address=2001:db8::/32 comment="defconf: documentation" list=bad_ipv6
add address=2001:10::/28 comment="defconf: ORCHID" list=bad_ipv6
add address=3ffe::/16 comment="defconf: 6bone" list=bad_ipv6

/ipv6 firewall filter
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMPv6" protocol=\
    icmpv6
add action=accept chain=input comment="defconf: accept UDP traceroute" port=\
    33434-33534 protocol=udp
add action=accept chain=input comment=\
    "defconf: accept DHCPv6-Client prefix delegation." dst-port=546 protocol=\
    udp src-address=fe80::/10
add action=accept chain=input comment="defconf: accept IKE" dst-port=500,4500 \
    protocol=udp
add action=accept chain=input comment="defconf: accept ipsec AH" protocol=\
    ipsec-ah
add action=accept chain=input comment="defconf: accept ipsec ESP" protocol=\
    ipsec-esp
add action=accept chain=input comment=\
    "defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=input comment=\
    "defconf: drop everything else not coming from LAN" in-interface-list=\
    !LAN
add action=accept chain=forward comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" \
    connection-state=invalid
add action=drop chain=forward comment=\
    "defconf: drop packets with bad src ipv6" src-address-list=bad_ipv6
add action=drop chain=forward comment=\
    "defconf: drop packets with bad dst ipv6" dst-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: rfc4890 drop hop-limit=1" \
    hop-limit=equal:1 protocol=icmpv6
add action=accept chain=forward comment="defconf: accept ICMPv6" protocol=\
    icmpv6
add action=accept chain=forward comment="defconf: accept HIP" protocol=139
add action=accept chain=forward comment="defconf: accept IKE" dst-port=\
    500,4500 protocol=udp
add action=accept chain=forward comment="defconf: accept ipsec AH" protocol=\
    ipsec-ah
add action=accept chain=forward comment="defconf: accept ipsec ESP" protocol=\
    ipsec-esp
add action=accept chain=forward comment=\
    "defconf: accept all that matches ipsec policy" ipsec-policy=in,ipsec
add action=drop chain=forward comment=\
    "defconf: drop everything else not coming from LAN" in-interface-list=\
    !LAN

/system clock
set time-zone-name=America/Chicago

/system identity
set name=Router-1

/system routerboard settings
set auto-upgrade=yes enter-setup-on=delete-key

/tool mac-server
set allowed-interface-list=OFFBRandLAN

/tool mac-server mac-winbox
set allowed-interface-list=OFFBRandLAN
2

Hi there,

This likely blocks your resolutions to the router as your interfaces guest, iot and pc are not in any list.

Add a rule above the drny to permit tcp and udp to port 53 in the input chain on the router.

3

@vingjfg. Thanks. As a quick test of that possibility I disabled the ipv4 firewall rules entirely (Am behind the firewall provided by the ISP on their ONT device). That does not seem to have any effect. I then added the vlans (PC, Guest & IOT) to the LAN interface list without effect. Finally added the sfp to the LAN list with out effect. This was done on the switch and the router. Am leaving the PC, Guest & IOT vlans in the LAN list for now. I do not see any downside to this and I think they should be in that list as you pointed out. The thought that the firewall was blocking traffic because the vlans were not explicitly in the LAN list was a good one but unfortunately not the solution here.

4

I'm providing a link to the verbose exports of the setup code for the devices. The addition of the vlans to the LAN interface list is not included in these although I have made that change:

Verbose versions of the router-1 and switch-1 export files

5

Hi there,

The *1 indicates you deleted the pool but not the dhcp server.

Can you check you have a dns server defined for these networks?

6

@vingjfg Missed the missing pool. :wink: That was for the default 192.168.88.0 subnet. Deletion was accidental. To be honest, I don't see the point of leaving that subnet in place once the router is setup. On the other hand I don't see any harm either in a home office. Do you have any thoughts on that?

I think the dns for the vlans is provided for by this:

/ip dns
set allow-remote-requests=yes servers=1.1.1.1,1.0.0.1

7

This is from Switch-1.

What is the purpose of the vlan interfaces other than Management? Specifically why are there vlan interfaces for vlan 10 (PC), vlan 20 (Guest), and vlan 30 (IOT)? These are what cisco would call SVI, and for a device used as a switch, you would normally have only a single vlan interface, and it would have an ip address associated with it used for management of the switch. It is almost like a host that is connected to the switching fabric, and the host allows configuration of the switch.

The only time you need more than one vlan interface is if the device is routing traffic between vlans. In your case, the inter-vlan routing should be going to the router-1 L009 over the trunk, where it will be routed to another vlan, and possibly sent back over the same trunk link.

I don't think it is the cause of the problem, because you have no ip addresses associated with the interfaces, and they aren't "connected" to the "switching block" via /interface bridge vlan (i.e. only vlan 99 has BR1 in the tagged list of ports), but they may be confusing the router because there is a connection point to the cpu created when you add vlan interfaces to the bridge.

Oh, and by the way, is there any way for someone without an Apple iCloud account to download the files you shared? If so, it wasn't obvious to me.

8

I just noticed this on Switch-1 as well

But you have ip address associated with BR1 on Switch-1 (the untagged vlan on the trunk, currently set to pvid 1 by default). So both Switch-1 and Router-1 have connected routes to 192.168.88.0/24, but because you have specified frame-types="admit-only-vlan-tagged" for sfp1, traffic in 192.168.88.0/24 between the two L009s will be blocked. I.e. if you attempt pinging 192.168.88.2 from Router-1, you will get time outs (at least that would be what I would expect).

I would remove the ip addresses from BR1 on both L009s.

9

You are right, the default is to pass either the dynamic or static servers.

10

On a bigger screen.

A few things -

  • On your switch, the bridge definition differs than the one from the router:

On the swich:

On the router:

Add the auto-mac=no and port-cost-mode=short on the switch. It is also best practice to set the admin-mac.

/interface/bridge/set [find name=BR1] auto-mac=no port-cost-mode=short
  • The ether switch ports could be set as edge=yes to avoid going through the STP blocking state.
/interface/bridge/port/set [find interface=ether1] edge=yes
/interface/bridge/port/set [find interface=ether2] edge=yes
/interface/bridge/port/set [find interface=ether3] edge=yes
/interface/bridge/port/set [find interface=ether4] edge=yes
/interface/bridge/port/set [find interface=ether5] edge=yes
/interface/bridge/port/set [find interface=ether6] edge=yes
/interface/bridge/port/set [find interface=ether7] edge=yes

These routes on your switch are probably incorrect: sfp1 is part of a bridge, has VLANs and your management network is not the native VLAN, which btw is blocked by the port configuration.

What you likely need is a default to 192.168.0.1 (your router on the management network). The route to 10.0.10.0/24 is not needed as the only way for the switch to reach that network is through its default gateway.

/ip route remove [where dst-address=0.0.0.0/0]
/ip route remove [where dst-address=10.0.10.0/0]
/ip route add dst-address=0.0.0.0/0 gateway=192.168.0.1
11

Router-1

On Switch-1

  1. does "From the PC vlan ports pings to all *.1 ips in the LAN work" include 192.168.88.1? Or did you mean only the subnets you added?
  2. "Within a vlan (eg vlan id 10) one host cannot ping another." What are the hosts involved, and do they have any host based firewall that could be blocking the ping requests? Because you are able to ping 192.168.10.1 from switch-1, it appears there is layer 2 connectivity between vlan 10 on Switch-1 and vlan 10 on Router-1. Windows will block pings when the interface is set to public mode.
12

Sorry for being slow to get back.

@Buckeye
After a long read through this post RouterOS bridge mysteries explained I understand why the non-management vlan interfaces are not needed on the switch. Fixed.

The duplicate ip addresses on the bridge ports was a major error. Have fixed the duplication. But I have this vague idea that the ip address is needed for routing. Your reply suggests this is not correct.

@vingjfg
I will set the admin-mac and set auto-mac=no. Also will set the ether switch ports as edge=yes. I was playing with the routes in an effort to solve some of the problems. Didn’t work. But with changes per you and @Buckeye plus fixes to issues mentioned below things are now working as expected.

I had some some fundamental problems with my setup that had nothing to do with RouterOS. They were simple engineering stupidity and should have been addressed first. Always test your test bed before assuming it works as expected!

  • I am using a pair of Windows laptops and failed to verify that Windows Defender firewall rules were appropriate to the testing setup. Pings on "public networks" were NOT allowed!

  • The sfp ports on the RB5009 and the L009 are not identical and I am not using Mikrotik sfp modules. The mismatch caused problems with traffic via the trunk setup between the sfp ports. The interconnect was at times not seen and at other times provided erratic data transfer.
    The Mikrotik recommended solution:

    Since RouterOS v7.12

    /interface
    ethernet set sfp-sfpplus1 auto-negotiation=no speed=1G-baseX `

    Older RouterOS `

    /interface
    ethernet set sfp-sfpplus1 auto-negotiation=no speed=1Gbps
    full-duplex=yes`

Again, thanks to each of you.

13

Join the club. I think anyone that has been working with networks has made the error of assuming something works like they think it does. And it can lead to making breaking changes to other things that were working in an attempt to "fix the problem".

Ping is a useful tool, but there are times when ping will work but other things won't (e.g. fine grained firewall rules allowing or blocking, or when default pings work but there is an MTU blackhole along the path that silently drops packets that are too large). And the windows firewall is a common problem when people are introducing vlans and extra subnets, because by default, even when the windows interface is set to private, and ping request from within the subnet will work, pings from "foreign" (aka remote) networks (i.e. not directly connected) will be blocked. You can modify the windows firewall rules to be more permissive (e.g. allows other networks or ranges), and that is generally safer than turning off the firewall. How to allow pings from rfc1918 addresses.

14

I'm not sure which reply you are referring to.

What I was saying in

Two hosts connected to two different vlans on the Switch-1 device (e.g. a PC on switch-1:ether2 in vlan 10, and a PC on switch-1:ether4 in vlan 20) should be able to communicate (as long as not being blocked by a firewall). However, since these are in different vlans, no "direct" communication at layer 2 will be allowed on switch-1. The traffic will have to go to the router-1 L009 router (the default gateway for both vlans), and since the router-1 L009 does have interfaces in both vlan 10 and vlan 20, it will have connected routes to 10.0.10.0/24 and 10.0.20.0/24 networks, so it "knows" where to forward traffic.

Note that this means that all inter-vlan routing is done only by the router-1 L009, and if the router-1 L009 is disconnected or is shut off, no traffic will be allowed between vlan 10 and vlan 20, even on devices directly connected to access ports for vlan 10 and vlan 20 on switch-1. However, devices within the same vlan will be able to communicate, as they do not require the router.

This configuration (a router connected to a switch via a trunk link) is what is called a "router on a stick" because the traffic uses the same "wire" for both vlans.

15

Makes sense. Thanks.

16

I knew better. But was so focused on 'just make it work' and tired that i made just the compounding errors you described.