Router on a stick with WAN on the stick

smashy1 · November 20, 2024, 4:14am

Hello! I am attempting to configure a “router on a stick” and am at the point where I just need to ask for help.

The goal is Router → Switch → WAN device, where the WAN device is on a VLAN by itself and all traffic goes directly to the router, and the other ports of the switch are on my home network, not part of the WAN device. I currently have a working setup, but it is Router → dstnat → switch → WAN device, so the rest of the switch is not part of the home network. It is not possible to run another cable from the router to the switch - I need to use the existing link.

The target diagram is attached. Everything is working, including the VLANs, except like I said the “barn switch” is dstnat’d with only the starlink device plugged in. This is what I want to fix.

The switch is on SWOS, and I had difficulty understanding how to configure it. I reverted all the VLAN config on the switch and need to start over.

Once again, to summarize, the goal is to add a VLAN ID=5 where only the starlink interface on the switch is a member. All traffic must flow to the router to ingress/egress from the starlink device. The other ports of the switch should be on the home network (VLAN ID unspecified). I do not have a management VLAN, and I do not have a VLAN for the home network. I know these are best practices but have not implemented it - this is just a home network hobby project.

Thank you so much for any help, I have sunk many hours into this.

Here is my router config:

# 2024-11-19 22:55:09 by RouterOS 7.16.1
# software id = **ELIDED**
#
# model = RB2011iLS
# serial number = **ELIDED**
/interface bridge
add comment="main bridge" ingress-filtering=no name=local port-cost-mode=\
    short protocol-mode=none vlan-filtering=yes
/interface ethernet
set [ find default-name=ether1 ] comment="DSL Modem"
set [ find default-name=ether2 ] comment=kitchen-ap
set [ find default-name=ether3 ] comment=Curator
set [ find default-name=ether4 ] comment=zion
set [ find default-name=ether6 ] comment="Movie Room Firestick"
set [ find default-name=ether7 ] comment="Downstairs Work Ethernet"
set [ find default-name=ether8 ] comment="Living Room TV"
set [ find default-name=ether9 ] comment="**ELIDED** Work Ethernet"
set [ find default-name=ether10 ] comment=downstairs-ap
/interface vlan
add interface=local name=GUEST_VLAN vlan-id=30
add interface=local name=TDS_VLAN vlan-id=40
add disabled=yes interface=local name=WAN_VLAN vlan-id=5
add interface=local name=WORK_VLAN vlan-id=20
/interface list
add name=WAN
add name=VLAN
add name=BASE
add comment=defconf name=LAN
/interface lte apn
set [ find default=yes ] ip-type=ipv4 use-network-apn=no
/ip ipsec proposal
set [ find default=yes ] auth-algorithms=sha256,sha1 enc-algorithms=\
    aes-256-cbc,aes-128-cbc
/ip pool
add name=dhcp_pool0 ranges=192.168.1.100-192.168.1.254
add name=work-dhcp-pool ranges=192.168.2.100-192.168.2.254
add name=guest-dhcp-pool ranges=192.168.3.100-192.168.3.254
add name=tds-pool ranges=192.168.4.100-192.168.4.254
/ip dhcp-server
add address-pool=dhcp_pool0 interface=local name=dhcp1
add address-pool=work-dhcp-pool interface=WORK_VLAN lease-time=10m name=\
    work-dhcp-server
add address-pool=guest-dhcp-pool interface=GUEST_VLAN lease-time=10m name=\
    guest-dhcp-server
add address-pool=tds-pool interface=TDS_VLAN name=tds-dhcp-server
/ip smb users
set [ find default=yes ] disabled=yes
/port
set 0 name=serial0
/ppp profile
add bridge=local local-address=192.168.1.1 name=mainProfile remote-address=\
    dhcp_pool0
/routing bgp template
set default disabled=no output.network=bgp-networks
/routing ospf instance
add disabled=no name=default-v2
/routing ospf area
add disabled=yes instance=default-v2 name=backbone-v2
/interface bridge port
add bridge=local ingress-filtering=no interface=ether4 internal-path-cost=10 \
    path-cost=10
add bridge=local ingress-filtering=no interface=ether3 internal-path-cost=10 \
    path-cost=10
add bridge=local ingress-filtering=no interface=ether5 internal-path-cost=10 \
    path-cost=10
add bridge=local ingress-filtering=no interface=ether6 internal-path-cost=10 \
    path-cost=10
add bridge=local frame-types=admit-only-untagged-and-priority-tagged \
    ingress-filtering=no interface=ether7 internal-path-cost=10 path-cost=10 \
    pvid=20
add bridge=local ingress-filtering=no interface=ether8 internal-path-cost=10 \
    path-cost=10
add bridge=local frame-types=admit-only-untagged-and-priority-tagged \
    ingress-filtering=no interface=ether9 internal-path-cost=10 path-cost=10 \
    pvid=20
add bridge=local ingress-filtering=no interface=ether10 internal-path-cost=10 \
    path-cost=10
add bridge=local ingress-filtering=no interface=ether2 internal-path-cost=10 \
    path-cost=10
add bridge=local disabled=yes ingress-filtering=no interface=sfp1 \
    internal-path-cost=10 path-cost=10
/ip firewall connection tracking
set udp-timeout=10s
/ip neighbor discovery-settings
set discover-interface-list=!BASE
/ip settings
set max-neighbor-entries=8192
/ipv6 settings
set disable-ipv6=yes max-neighbor-entries=8192
/interface bridge vlan
add bridge=local tagged=local,ether2,ether10 vlan-ids=20
add bridge=local tagged=local,ether2,ether10 vlan-ids=30
# sfp1 not a bridge port
add bridge=local disabled=yes tagged=sfp1 vlan-ids=5
add bridge=local comment="backup internet" tagged=ether2,ether10,local \
    vlan-ids=40
/interface l2tp-server server
set default-profile=mainProfile enabled=yes use-ipsec=yes
/interface list member
add interface=ether1 list=WAN
add interface=WORK_VLAN list=VLAN
add interface=GUEST_VLAN list=VLAN
add interface=local list=LAN
add disabled=yes interface=WAN_VLAN list=WAN
add interface=sfp1 list=WAN
add interface=TDS_VLAN list=VLAN
/interface ovpn-server server
set auth=sha1,md5
/ip address
add address=192.168.1.1/24 interface=local network=192.168.1.0
add address=192.168.2.1/24 interface=WORK_VLAN network=192.168.2.0
add address=192.168.3.1/24 interface=GUEST_VLAN network=192.168.3.0
add address=192.168.253.1/24 disabled=yes interface=WAN_VLAN network=\
    192.168.253.0
add address=192.168.4.1/24 interface=TDS_VLAN network=192.168.4.0
/ip cloud
set update-time=no
/ip dhcp-client
add interface=ether1 use-peer-dns=no
add disabled=yes interface=WAN_VLAN use-peer-dns=no
add interface=sfp1
/ip dhcp-server lease
add address=192.168.3.40 mac-address=**ELIDED** server=\
    guest-dhcp-server
/ip dhcp-server network
add address=192.168.1.0/24 dns-server=192.168.1.9 gateway=192.168.1.1
add address=192.168.2.0/24 dns-server=8.8.8.8 gateway=192.168.2.1
add address=192.168.3.0/24 dns-server=8.8.8.8 gateway=192.168.3.1
add address=192.168.4.0/24 dns-server=8.8.8.8 gateway=192.168.4.1
/ip dns
set allow-remote-requests=yes servers=192.168.1.9
/ip dns static
add address=192.168.1.9 name=pihole type=A
/ip firewall filter
add action=accept chain=input comment="accept established,related,untracked" \
    connection-state=established,related,untracked
add action=drop chain=input comment="Drop invalid connections" \
    connection-state=invalid
add action=accept chain=input comment="Allow VLAN to connect to router" \
    disabled=yes in-interface-list=VLAN
add action=accept chain=input comment="allow ICMP to connect to router" \
    in-interface-list=LAN protocol=icmp
add action=accept chain=input comment="allow Winbox to connect to router" \
    in-interface-list=LAN port=8291 protocol=tcp src-address-list=""
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    in-interface-list=!LAN
add action=drop chain=input comment=\
    "block everything else trying to connect to router" disabled=yes \
    in-interface-list=WAN
add action=fasttrack-connection chain=forward comment=\
    "fast-track for established,related" connection-state=established,related \
    hw-offload=yes
add action=accept chain=forward comment="accept established,related" \
    connection-state=established,related,untracked
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
    ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" \
    ipsec-policy=out,ipsec
add action=accept chain=forward comment="Printer - any LAN device can access" \
    connection-state=new dst-address=192.168.1.20 in-interface-list=LAN
add action=accept chain=forward comment="VLAN Internet Access Only" \
    connection-state=new in-interface-list=VLAN out-interface-list=WAN
add action=accept chain=forward comment="Base LAN Internet Access" \
    connection-state=new out-interface-list=WAN src-address=192.168.1.0/24
add action=accept chain=forward comment="defconf: accept in ipsec policy" \
    disabled=yes ipsec-policy=in,ipsec
add action=accept chain=forward comment="Accept all that is DST NATed - this r\
    ule accepts all incoming traffic that is port forwrded" \
    connection-nat-state=dstnat connection-state=new disabled=yes
add action=accept chain=forward comment=\
    "Allow access to Curator4 from all devices" disabled=yes dst-address=\
    192.168.1.11
add action=drop chain=forward comment="drop all other forward chains - VLANs c\
    annot access each other - THIS RULE BREAKS REMOTEDESKTOP\? (fixed with dst\
    nat rule)" connection-state=""
add action=drop chain=forward comment=\
    "drop access to clients behind NAT from WAN" connection-nat-state=!dstnat \
    connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="Configure outbound NAT" disabled=\
    yes out-interface=ether1
add action=masquerade chain=srcnat out-interface=sfp1
add action=netmap chain=dstnat comment="Remote Desktop on 3389" disabled=yes \
    dst-port=3389 protocol=tcp to-addresses=192.168.1.10 to-ports=3389
add action=netmap chain=dstnat disabled=yes dst-port=54545 protocol=tcp \
    to-addresses=192.168.1.10 to-ports=3389
add action=dst-nat chain=dstnat connection-limit=100,32 disabled=yes \
    dst-limit=1,5,dst-address/1m40s dst-port=80 fragment=no \
    in-interface-list=WAN ingress-priority=0 limit=1,5:packet protocol=tcp \
    to-addresses=192.168.1.11 to-ports=80
add action=masquerade chain=srcnat disabled=yes dst-address=192.168.1.11 \
    dst-port=80,443 protocol=tcp src-port=""
add action=netmap chain=dstnat disabled=yes dst-address=192.168.1.11 \
    dst-port=443 protocol=tcp to-addresses=192.168.1.11 to-ports=443
add action=dst-nat chain=dstnat comment="NGINX port forwarding" disabled=yes \
    dst-port=80 in-interface-list=WAN protocol=tcp to-addresses=192.168.1.11 \
    to-ports=80
add action=dst-nat chain=dstnat comment="NGINX port forwarding" disabled=yes \
    dst-port=80 in-interface-list=WAN protocol=udp src-port="" to-addresses=\
    192.168.1.11 to-ports=80
add action=dst-nat chain=dstnat comment="Redirect DNS to pi-hole" disabled=\
    yes dst-address=!192.168.1.9 dst-port=53 protocol=udp src-address=\
    !192.168.1.9 to-addresses=192.168.1.9
add action=dst-nat chain=dstnat disabled=yes dst-address=!192.168.1.9 \
    dst-port=53 protocol=tcp src-address=!192.168.1.9 to-addresses=\
    192.168.1.9
add action=masquerade chain=srcnat comment="These masquerade rules are apparen\
    tly needed, but they change the source IP in pihole to mikrotik" \
    disabled=yes dst-address=192.168.1.9 dst-port=53 protocol=udp \
    src-address=192.168.1.0/24
add action=masquerade chain=srcnat disabled=yes dst-address=192.168.1.9 \
    dst-port=53 protocol=tcp src-address=192.168.1.0/24
add action=masquerade chain=srcnat disabled=yes dst-address=192.168.1.9 \
    dst-port=53 protocol=udp src-address=192.168.2.0/24
add action=masquerade chain=srcnat disabled=yes dst-address=192.168.1.9 \
    dst-port=53 protocol=tcp src-address=192.168.2.0/24
add action=masquerade chain=srcnat disabled=yes dst-address=192.168.1.9 \
    dst-port=53 protocol=udp src-address=192.168.3.0/24
add action=masquerade chain=srcnat disabled=yes dst-address=192.168.1.9 \
    dst-port=53 protocol=tcp src-address=192.168.3.0/24
/ip hotspot profile
set [ find default=yes ] html-directory=flash/hotspot
/ip ipsec profile
set [ find default=yes ] dpd-interval=2m dpd-maximum-failures=5
/ip smb shares
set [ find default=yes ] directory=/pub
/ip ssh
set strong-crypto=yes
/ppp secret
add disabled=yes name=emulcahy-vpn profile=mainProfile
/routing bfd configuration
add disabled=no interfaces=all min-rx=200ms min-tx=200ms multiplier=5
/system clock
set time-zone-name=**ELIDED**
/system note
set show-at-login=no
/system package update
set channel=development
/tool bandwidth-server
set enabled=no
/tool mac-server
set allowed-interface-list=none

anav · November 21, 2024, 2:07am

Thats like asking to use a rowboat to sail to the moon.
The switch is not a router but a very simple managed switch.

Your router should be setup to to vlan filtering with one bridge…
Decide on either a separate management LAN or perhaps use one of the VLANs as a trusted vlan for this purpseose let say your work vlan!!

Change your main home network from no vlan to another VLAN10.
Now you will have to build the switch for ROS but if it was a hex as a switch it would look like.

/interface bridge
add ingress-filtering=no name=bridgeswitch port-cost-mode=short vlan-filtering=yes
/interface vlan
add interface=bridgeswitch  name=work-VLAN vlan-id=10
/interface bridge port
add bridge=bridgeswitch ingress-filtering=yes frame-types=admit-only-vlan-tagged interface=sfp1 comment="trunk to router"
add bridge=bridgeswitch ingress-filtering=yesframe-types=admit-only-vlan-tagged  interface=ether1  comment="WAN2 on vlan5"
add bridge=bridgeswitch ingress-filtering=yesframe-types=admit-only-vlan-tagged interface=ether2 comment="trunk  to barn AP"
add bridge=bridgeswitch ingress-filtering=yesframe-types=admit-only-untagged-and-priority-tagged interface=ether3  pvid=20  comment="local admin access to switch" 
add bridge=bridgeswitch ingress-filtering=yesframe-types=admit-only-untagged-and-priority-tagged interface=ether4  pvid=10  comment="To  cam5"
add bridge=bridgeswitch ingress-filtering=yesframe-types=admit-only-untagged-and-priority-tagged interface=ether5 pvid=10   comment="To  cam6"
/interface bridge vlan
add bridge=bridgeswitch tagged=bridgeswitch,sfp1,ether2   untagged=ether3  vlan-ids=20  comment="work trusted vlan"
add bridge=bridgeswitch tagged=sfp1,ether2  untagged=ether4,ether5 vlan-ids=10  { assuming barn AP  has  home users }
add bridge=bridgeswitch tagged=sfp1,ether1  vlan-id=5  comment="WAN2 input"
/ip address
add address=192.168.2.X/24 interface=work-VLAN network=192.168.2.0

Will have to think about it for the SWOS scenario

smashy1 · November 21, 2024, 2:37am

Thats like asking to use a rowboat to sail to the moon.

Hah, thanks for putting it in perspective. I don’t have the experience to know if I just need a different approach. I guess if I upgraded the switch to a router, even the same RB2011 model, it would make things quite a bit simpler. Perhaps that is the right choice.

anav · November 21, 2024, 3:03am

Now for the RB device…
/interface bridge
add comment=“main bridge” ingress-filtering=no name=local port-cost-mode=
short protocol-mode=none vlan-filtering=yes
/interface vlan
add interface=local name=HOME_VLAN vlan-id=10

/interface list
add name=WAN
add name=VLAN
add name=BASE

/ip dhcp-server
add address-pool=dhcp_pool0 interface=HOME_VLAN name=dhcp1

/interface bridge port
add bridge=local ingress-filtering=yes frame-types=admit-only-vlan-tagged interface=ether2 comment=“trunk to kitchen AP”
add bridge=local ingress-filtering=yes frame-types=admit-priority-tagged-and-untagged interface=ether3 pvid=10
add bridge=local ingress-filtering=yes frame-types=admit-priority-tagged-and-untagged interface=ether4 pvid=10
add bridge=local ingress-filtering=yes frame-types=admit-priority-tagged-and-untagged interface=ether5 pvid=10
add bridge=local ingress-filtering=yes frame-types=admit-priority-tagged-and-untagged interface=ether6 pvid=10
add bridge=local ingress-filtering=yes frame-types=admit-priority-tagged-and-untagged interface=ether8 pvid=10
add bridge=local ingress-filtering=yes frame-types=admit-priority-tagged-and-untagged interface=ether7 pvid=20
add bridge=local ingress-filtering=yes frame-types=admit-priority-tagged-and-untagged interface=ether9 pvid=20
add bridge=local ingress-filtering=yes frame-types=admit-only-vlan-tagged interface=ether10 comment=“trunk to downstairs ap/switch”
add bridge=local ingress-filtering=yes frame-types=admit-only-vlan-tagged interface=sfp1 comment=“trunk to barn switch”

/ip neighbor discovery-settings
set discover-interface-list=BASE

/interface bridge vlan
add bridge=local tagged=local,sfp1 vlan-ids=5 comment=“wan2 input from barn”
add bridge=local tagged=local,ether2,ether10,sfp1 untagged=ether3,ether4,ether5,ether6,ether8 vlan-ids=10
add bridge=local tagged=local,ether2,ether10,sfp1 untagged=ether7,ether9 vlan-ids=20
add bridge=local tagged=local,ether2,ether10 vlan-ids=30 { assuming downstairs AP may also serve guests, if not remove ether10 }

/interface list member
add interface=ether1 list=WAN
add interface=WAN_VLAN list=WAN
add interface=WORK_VLAN list=VLAN
add interface=GUEST_VLAN list=VLAN
add interface=HOME_VLAN list=VLAN
add interface=TDS_VLAN list=VLAN
add interface=WORK_VLAN list=BASE

/ip address
add address=192.168.1.1/24 interface=HOME_VLANnetwork=192.168.1.

/ip dhcp-server network
add address=192.168.1.0/24 dns-server=192.168.1.9 gateway=192.168.1.1
add address=192.168.2.0/24 dns-server=192.168.2.1 gateway=192.168.2.1
add address=192.168.3.0/24 dns-server=192.168.3.1 gateway=192.168.3.1
add address=192.168.4.0/24 dns-server=192.168.4.1 gateway=192.168.4.1

/ip dns
set allow-remote-requests=yes servers=8.8.8.8

/ip firewall filter
add action=accept chain=input comment=“accept established,related,untracked”
connection-state=established,related,untracked
add action=drop chain=input comment=“Drop invalid connections”
connection-state=invalid
add action=accept chain=input comment=“allow ICMP to connect to router”
in-interface-list=LAN protocol=icmp
add action=accept chain=input comment=“Allow Work VLAN full access” interface=VLAN_WORK src-address=192.168.2.0/24
add action=accept chain=input comment=“users to services” in-interface-list=VLAN dst-port=53 protocol=udp
add action=accept chain=input comment=“users to services” in-interface-list=VLAN dst-port=53 protocol=tcp
add action=drop chain=input comment=“Drop all else”
++++++++++++++++++++++++++++++++++++++++++++++++++++++
add action=fasttrack-connection chain=forward comment=
“fast-track for established,related” connection-state=established,related
hw-offload=yes
add action=accept chain=forward comment=“accept established,related”
connection-state=established,related,untracked
add action=drop forward=input comment=“Drop invalid connections”
connection-state=invalid
add action=accept chain=forward comment=“internet access” in-interface-list=VLAN out-interface-list=WAN
add action=accept chain=forward comment=“Shared printer” in-interface-list=VLAN dst-address=192.168.1.20
add action=accept chain=forward comment=“port forwarding” connection-nat-state=dstnat
add action=accept chain=forward comment=“shared curator4” in-interface-list=VLAN dst-address=192.168.1.14 disabled=yes
add action=drop chain=forward comment=“Drop all else”
/ip firewall nat
add action=masquerade chain=srcnat comment=“hairpin nat rule for pi server” dst-address=192.168.1.0/24 src-address=192.168.1.0/24
add action=masquerade chain=srcnat out-interface-list=WAN
add action=dst-nat chain=dstnat in-interface=HOME_VLAN src-address-list=!Excluded dst-port=53 protocol=udp to-address=192.168.1.9
add action=dst-nat chain=dstnat in-interface=HOME_VLAN src-address-list=!Excluded dst-port=53 protocol=tcp to-address=192.168.1.9

WHERE
/ip firewall address-list
add address=192.168.1.9 list=Excluded
(+ any other address in 192.168.1.0/24 subnet you wish excluded from PI for DNS.)

/tool mac-server mac-winbox
set allowed-interface-list=BASE

++++++++++++++++++++++++++++++++++++
Things that need to be cleared UP.

Your WAN2 situation. Is it coming out of the ISP device as tagged on vlan 5 or simply untagged frames coming into the port.
Is WAN2 coming out of ISP device as a fixed private IP, a dynamic private IP, or dynamic Public IP???

Answers will help sort out if one is to use IP DHCP client or IP address for WAN2 at the ROUTER, and the proper Setting at the switch ( either tagged port for ISP expecting tagged frames, or access port expecting untagged frames which would vlan ID 5 added upon entry at the port ).

WEIRD netmap and other rules that look out of place.
DSTNAT rules have to be rationalized, and relooked after.

anav · November 21, 2024, 3:04am

Not necessarily, the swos should work fine for this need when setup properly, the more important thing is to configure the router the RB, to a full vlan state.
ALso need the info on the WAN2 side of the house.

smashy1 · November 21, 2024, 3:43am

Your WAN2 situation. Is it coming out of the ISP device as tagged on vlan 5 or simply untagged frames coming into the port.

Is WAN2 coming out of ISP device as a fixed private IP, a dynamic private IP, or dynamic Public IP???

For both the starlink connected to the switch, and DSL connected to RB port 1, The WAN is untagged frames coming into the port. Both are dynamic public IP.

The TDS_VLAN in my config is an incomplete attempt to have a backup WAN connection. But most likely, this will be dropped once I am comfortable with starlink, so not important.

WEIRD netmap and other rules that look out of place.

Yes, these are all disabled. Was trying port forwarding but did not implement it. I think just ignore it for this.

Thank you for your input so far! I will be working through this very soon.

anav · November 21, 2024, 3:50am

Okay so basically on the router one uses IP DCHP client
/ip dhcp-client
add interface=ether1 use-peer-dns=no
add interface=WAN_VLAN use-peer-dns=no

and remove IP address for WAN_VLAN.

anav · November 21, 2024, 4:03am

As for SWOS barn switch. Keep in mind, the trusted vlan is 20 and thus all smart devices should get an IP address on this vlan, preferably by taking mac address and asssiging a static DHCP lease on the RB router.

Port - VLAN MODE / VLAN RECEIVE / VLAN ID
SFP1 - strict / allow only tagged frames / default (1)
Ether2 -strict / allow only tagged frames / default (1)
ether1 - strict / allow only untagged frames / vlan 5
ether3/4/5 strict / allow only untagged frames / assigned access port VLAN

VLAN membership
vlan1 → sfp1, ether2
vlan5 → sfp1, ether1
vlan10 -->sfp1, ether2,3,4,5 ( assumes barn AP will provide only home wifi)
vlan20 → sfp1,ether2 ( barn AP needs van20 to get its IP address on trusted subnet )