Beating my head on the wall as a mikrotik newb

ok so the configuration im having to use to get my crs326 to talk to my rb5009 seems way off and ive tried what some have suggested to get it working correctly but no joy

using router os 7.19.0 on both units and currently no wan as i wanted to get the lan all working before i unnplugged my isp to switch to the new hardware from the tp link stuff im running now

like this i can get comunication to and from everywhere

like this i get nothing

here is the crs export with it working

/interface bridge
add name=bridge vlan-filtering=yes
/interface vlan
add comment=IOT interface=bridge name=IOT vlan-id=30
add comment=MGMT interface=bridge name=MGMT vlan-id=99
add comment=Servers interface=bridge name=Servers vlan-id=20
add comment=Trusted interface=bridge name=Trusted vlan-id=10
add comment=WWW interface=bridge name=WWW vlan-id=80
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge frame-types=admit-only-vlan-tagged interface=sfp-sfpplus24 pvid=666
add bridge=bridge interface=sfp-sfpplus23
add bridge=bridge interface=sfp-sfpplus21
add bridge=bridge interface=sfp-sfpplus19
add bridge=bridge interface=sfp-sfpplus17 pvid=10
add bridge=bridge interface=ether1
add bridge=bridge interface=sfp-sfpplus15 pvid=10
add bridge=bridge interface=sfp-sfpplus13 pvid=10
/ip neighbor discovery-settings
set discover-interface-list=!dynamic
/ipv6 settings
set disable-ipv6=yes
/interface bridge vlan
add bridge=bridge comment=Trusted tagged=sfp-sfpplus24,bridge untagged=sfp-sfpplus15,sfp-sfpplus17,sfp-sfpplus13 vlan-ids=10
add bridge=bridge comment=Servers tagged=sfp-sfpplus24,bridge vlan-ids=20
add bridge=bridge comment=IOT tagged=sfp-sfpplus24,bridge vlan-ids=30
add bridge=bridge comment=WWW tagged=sfp-sfpplus24,bridge vlan-ids=80
add bridge=bridge comment=MGMT tagged=sfp-sfpplus24,bridge vlan-ids=99
/ip address
add address=10.10.1.2 interface=ether1 network=10.10.1.2
add address=10.10.10.2/24 comment=Trusted interface=Trusted network=10.10.10.0
add address=10.10.20.2/24 comment=Servers interface=Servers network=10.10.20.0
add address=10.10.30.2/24 comment=IOT interface=IOT network=10.10.30.0
add address=10.10.80.2/24 comment=WWW interface=WWW network=10.10.80.0
add address=10.10.99.2/24 comment=MGMT interface=MGMT network=10.10.99.0
/ip route
add disabled=no dst-address=10.10.20.0/24 gateway=10.10.20.1 routing-table=main suppress-hw-offload=no
add disabled=no distance=1 dst-address=10.10.30.0/24 gateway=10.10.30.1 routing-table=main scope=30 suppress-hw-offload=no target-scope=10
add disabled=no distance=1 dst-address=10.10.80.0/24 gateway=10.10.80.1 routing-table=main scope=30 suppress-hw-offload=no target-scope=10
add disabled=no dst-address=10.10.10.0/24 gateway=10.10.10.1 routing-table=main suppress-hw-offload=no
add disabled=no dst-address=10.10.99.0/24 gateway=10.10.99.1 routing-table=main suppress-hw-offload=no
/system identity
set name="10G Switch"
/system routerboard settings
set enter-setup-on=delete-key

here is the rb5009 export with it working

/interface bridge
add name=Bridge vlan-filtering=yes
/interface vlan
add comment=IOT interface=Bridge name=IOT vlan-id=30
add comment=MGMT interface=Bridge name=MGMT vlan-id=99
add comment=Servers interface=Bridge name=Servers vlan-id=20
add comment=Trusted interface=Bridge name=Trusted vlan-id=10
add comment=WWW interface=Bridge name=WWW vlan-id=80
/ip pool
add name=Trusted-dhcp-pool ranges=10.10.10.100-10.10.10.254
add name=IOT-dhcp-pool ranges=10.10.30.50-10.10.30.254
/ip dhcp-server
add address-pool=Trusted-dhcp-pool interface=Trusted lease-time=8h name=\
    Trusted
add address-pool=IOT-dhcp-pool interface=IOT lease-time=8h name=dhcp1
/interface bridge port
add bridge=Bridge interface=ether1
add bridge=Bridge frame-types=admit-only-vlan-tagged interface=sfp-sfpplus1 \
    pvid=666
/ip neighbor discovery-settings
set discover-interface-list=!dynamic
/ipv6 settings
set disable-ipv6=yes
/interface bridge vlan
add bridge=Bridge comment=Trusted tagged=Bridge,sfp-sfpplus1 vlan-ids=10
add bridge=Bridge comment=Servers tagged=sfp-sfpplus1,Bridge vlan-ids=20
add bridge=Bridge comment=IOT tagged=sfp-sfpplus1,Bridge vlan-ids=30
add bridge=Bridge comment=WWW tagged=sfp-sfpplus1,Bridge vlan-ids=80
add bridge=Bridge comment=MGMT tagged=Bridge,sfp-sfpplus1 vlan-ids=99
/ip address
add address=10.10.1.1/24 interface=Bridge network=10.10.1.0
add address=10.10.10.1/24 comment=Trusted interface=Trusted network=10.10.10.0
add address=10.10.20.1/24 comment=Servers interface=Servers network=10.10.20.0
add address=10.10.30.1/24 comment=IOT interface=IOT network=10.10.30.0
add address=10.10.80.1/24 comment=WWW interface=WWW network=10.10.80.0
add address=10.10.99.1/24 comment=MGMT interface=MGMT network=10.10.99.0
/ip dhcp-server network
add address=10.10.10.0/24 gateway=10.10.10.1
add address=10.10.30.0/24 gateway=10.10.30.1
/ip firewall filter
add action=accept chain=forward comment="trusted > servers" in-interface=\
    Trusted out-interface=Servers
add action=accept chain=forward comment="trusted > IOT" in-interface=Trusted \
    out-interface=IOT
add action=accept chain=forward comment="trusted > WWW" in-interface=Trusted \
    out-interface=WWW
add action=accept chain=forward comment="servers > trusted" in-interface=\
    Servers out-interface=Trusted
add action=accept chain=forward comment="MGMT > trusted" in-interface=MGMT \
    out-interface=Trusted
add action=accept chain=forward comment="MGMT > servers" in-interface=MGMT \
    out-interface=Servers
add action=accept chain=forward comment="MGMT > IOT" in-interface=MGMT \
    out-interface=IOT
add action=accept chain=forward comment="MGMT > WWW" in-interface=MGMT \
    out-interface=WWW
add action=drop chain=forward comment="servers > MGMT" in-interface=Servers \
    out-interface=MGMT
add action=drop chain=forward comment="IOT > trusted" in-interface=IOT \
    out-interface=Trusted
add action=drop chain=forward comment="IOT > servers" in-interface=IOT \
    out-interface=Servers
add action=drop chain=forward comment="IOT > WWW" in-interface=IOT \
    out-interface=WWW
add action=drop chain=forward comment="IOT > MGMT" in-interface=IOT \
    out-interface=MGMT
add action=drop chain=forward comment="WWW > trusted" in-interface=WWW \
    out-interface=Trusted
add action=drop chain=forward comment="WWW > servers" in-interface=WWW \
    out-interface=Servers
add action=drop chain=forward comment="WWW > IOT" in-interface=WWW \
    out-interface=IOT
add action=drop chain=forward comment="WWW > MGMT" in-interface=WWW \
    out-interface=MGMT
/ip route
add disabled=no dst-address=10.10.10.0/24 gateway=10.10.10.2 routing-table=\
    main suppress-hw-offload=no
add disabled=no dst-address=10.10.20.0/24 gateway=10.10.20.2 routing-table=\
    main suppress-hw-offload=no
add disabled=no dst-address=10.10.30.0/24 gateway=10.10.30.2 routing-table=\
    main suppress-hw-offload=no
add disabled=no dst-address=10.10.80.0/24 gateway=10.10.80.2 routing-table=\
    main suppress-hw-offload=no
add disabled=no distance=1 dst-address=10.10.99.0/24 gateway=10.10.99.2 \
    routing-table=main scope=30 suppress-hw-offload=no target-scope=10
add disabled=no dst-address=10.10.20.0/24 gateway=10.10.20.1 routing-table=\
    main suppress-hw-offload=no
/system identity
set name=Router

im being told there should be no need for interface vlan , ip addresses, or routes on the switch but like i said when ever i disable that stuff ( or delete it) i get no route to the rb5009 from the crs326

On the RB5009:

  • I see sfpplus1 with pvid=666 but no vlan definition for
  • normally you do not assign an IP to the bridge, but vlans themselves only
  • ether1 is part of the bridge, but no vlan setting for?
  • you set bridge vlan sfppplus1 to all vlanids, but then still set the port to 666 only. This does not make sense. Shouldn't be this for a trunk port for multiple vlanids?
/interface bridge port
add bridge=Bridge frame-types=admit-only-vlan-tagged interface=sfp-sfpplus1 \
    pvid=1

You don't say how the switch and RB5009 are connected to each other? Which port?

Firewalls, I don't understand them. All get access to all, back and forth, and from WAN as well to all? Why is that needed?

Missing lots of IP Addresses for vlans?

No Firewall input chain? You like hackers / hacking?

A few comments. I would make backups and exports of your current configs on the RB5009 and the CRS326 (backup so you can easily revert, export so you can easily read, and copy parts from).

It's always a good ideas to do both of these before making any changes, because if you don't it will be much harder to determine what the differences are.

The most likely reason you get no connection between the CRS326 and RB5009 when vlan interfaces don't exist on the CRS326 is because there is no default route configured. And without interfaces with ip addresses applied to them, you have no "connected routes" defined on the CRS326 (when there are no vlan interfaces).

If you do a /ip/route print on your devices, you will see only the connected routes that exist for the vlan interfaces. But no 0.0.0.0/0 default route.

The only vlan interface you should have on the CRS326 is the vlan99 MGMT vlan. But you need a default route (via the RB5009's 10.10.99.1 ip address) so that any ip address in in the MGMT subnet will be routed to the RB5009 to deal with.

You are using the CPU in the CRS as a host when doing the ping from the CRS326 itself. If you want ping to work from the switch itself, you will need as least one ip address on the switch (usually the management ip address) and a gateway defined for the interface on the RB5009 in the same subnet (probably 10.10.99.1 in vlan99, your management vlan). The vlan interface vlan99 on the CRS326 will need an i address in 10.10.99.0/24 that is different than 10.10.99.1 (which is used by the RB5009)

If you had a CRS326 bridge port configured as an access port for a vlan, and the device connected got its ip address from the dhcp server on the RB5009, then you should be able to ping the RB5009 interface on the same vlan from the host connected to the vlan access port, through the CRS326, i.e. the L2 connection should work whether you have an default gateway configured on the CRS or not, since L3 on the CRS326 isn't involved here, it would just be switching.

My recommendation would be to reset the configuration with the default config. Then you can let your current TP-Link router act as the virtual ISP for your RB5009. The default config on the RB5009 will use ether1 as the "WAN" interface and it will get its ip address via DHCP (from your TP-Link router). In the default config, all the other ports will be added to the bridge and the bridge interface with be the LAN connection to the router. It will create a LAN with 192.168.88.0/24 as the network, and the bridge itself will have ip address 192.168.88.1 This will then have a safe "default firewall" protecting the LAN from the WAN side. So you will be able to have internet access (and access to the devices in your TP-link (LAN, which is the RB5009's WAN) from the RB5009 LAN devices. But you should be blocked from accessing the RB5009 LAN (or the RB5009 itself) from the TP-link LAN. This is how you can verify the firewall on the RB5009 is working before you switch the ether1 interface to your real ISP connection.

As long as your TP-Link hasn't used 192.168.88.0/24 for its LAN, (tp-link LAN interface defaults are 192.168.0.1/24 ,192.168.1.1/24 or 192.168.68.1/24), so if you are using the defaults there, they should no conflict with either the MikroTik default or the 10.x.x.x addresses you have used).

Remove a port from the bridge (while you are learning, this is the easiest way to prevent yourself from being locked out when making changes to the bridge, for example turning on vlan-aware mode (vlan-filtering=yest) See Once and for all COMPLETE Offbridge Port setup for how to do this.

You will need to add this interface to the LAN list so you will have access to the RB5009 from the port.

Later you should create a MGMT interface list and add vlan99 and the "emergency access" port to the MGMT list, and then limit access to the MGMT interfaces. But leave this until you get things working without the firewall to limit things. i.e. after you add your vlan interfaces on the RB5009 add the vlan interfaces to the LAN list at first. This will then allow things to route between subnets.

In the default config, vlan-filtering is off, but the L3 bridge interface (that has the ip address 192.168.88.1) is preconfigured to connect to vlan1 once vlan-filtering is turned on, and all the bridge ports are also in vlan 1 (pvid=1 is default, but you won't see in export unless you export verbose), you can see this once vlan-filtering is turned on if you use the commands:
/interface/bridge print
/interface/bridge/port print
/interface/bridge/vlan print

I would also suggest configuring at least one access port (using a bridge port with pvid specified) for each vlan on the RB5009.

I don't understand what the WWW vlan80 is for. Is that supposed to be for your ISP connection?

Once you get vlans working, then start to work on your firewall. And you need to understand how the stateful firewall works, so you can create rules that allow the trusted vlan to create new connections to
other vlans, and allow return traffic for the connection to return, while blocking new connections from the other vlan. I.e. you may want to the trusted vlan to connect to an IoT device, and allow the return traffic, while still preventing new connections from the IoT device to the RB5009 or to devices on the trusted vlan.

Here's a link to the last saved copy of anav's Firewall setup thread on web.archive.org

The DEFACTO DEFAULT FIREWALL Setup

Your comments about backups is spot on with how I usually operate but I did no backups of this yet as I didn't have a working config so why backup anything that's already broken right?

The vlan80 is for the things I'm hosting that will be accessable from the web

I have dual isp so my tplink was left up and running on 1 while I was fighting with mikrotik to set up what will replace the tp link hardware

All in all the 0.0.0.0/0 route was what I was missing and I finally found it earlier today before your reply .. I'm not sure how I missed it . I'll blame that my brain was melting from the learning curve of mikrotik lol

But now it's sorted I've got inter VLAN connection was able to setup wan got a few ports forwarded for some basic firewall rulling in place and then had to leave for my weeks work feeling pretty good about it after the last 3 weekends of banging my head on the wall .

All in all i am pretty impressed with mikrotik and winbox and router is now that I'm not hating it because I had a skill issue lol

Thanks for taking the time to explain it out because if I hadn't found the answer your reply would have gotten me to it for sure

1 Like

So I was going off some reading and vids I watched and some of the choices didn't make sense to me either but after 4 attempts at what made sense to me i figured I'd just follow some of the vids to the letter

So your absolutely right there is some weird stuff in there …

But on a better note I got it figured out and was able to clean up the stuff that didn't make sense

Follow up with full export of both router and switch, for a sanity check? Remove serials and what you think private is.

I'm gone for the week but I will on Friday when. I get home