Community discussions

MikroTik App
 
Dulcow
just joined
Topic Author
Posts: 9
Joined: Thu Nov 30, 2023 12:13 pm

CRS317 + CRS328 - InterVLAN routing with L3HW

Sun Mar 24, 2024 10:15 am

Hi there,

I'm trying to understand the setup described https://help.mikrotik.com/docs/display/ ... with+Bonds as I want to achieve similar with my gear: CRS328-24P-4S+RM, CRS317-1G-16S+RM and RB5009UG+S+IN. I'm running 2.5Gbe Internet fibre connection using the SFP+ port of the RB5009. As per the example, I have set a bond between CRS328 and CRS317 on 2 ports, and that works just fine.

A few things that I want to achieve:
- Most of my 10G gear will be connected to CRS317
- Leverage CRS317 for Inter-VLAN access/routing
- Use the RB5009 only for the NAT with Internet

There are a few things that I don't understand.

1) What is going on with ether1? CRS317 also has an ether1 port which isn't mentioned in the configuration. Why is it set to "Admit all"? I would have expected to get only untagged traffic and use the ether1 port a way to get on the management network.

2) What is the best practise here? Is it to restrict access to VLAN 99 from a few hosts only or to have a dedicated port to access management network?

3) For the VLANs on the bridge, only VLAN 99 also has "bridge" on the tagged list. The others don't (VLAN 10, 20 and 30). Why is that? Is that preventing VLANs to "talk" to each other by default?

4) If I want to use the CRS317 as a core switch to do the inter-VLAN routing, should I consider it as the "Router' in the example and playing the L3 hardware features?

Thanks!

D.
Last edited by Dulcow on Sun Mar 24, 2024 5:08 pm, edited 2 times in total.
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11646
Joined: Thu Mar 03, 2016 10:23 pm

Re: VLAN switching and routing with bonds

Sun Mar 24, 2024 10:39 am

1) Default setting is frame-types=admit-all ... so if it's not changed explicitly according to needs, it'll remain that way.

2) Do as you see fit. IMO access to management VLAN should be as restricted as possible but also depends on particular use case.

3) Bridge is (also) interface which allows ROS to communicate with nerworks passing bridge the switch-like entity. To enable it, bridge port has to be made member of appropriate VLAN(s). See this explanation of different bridge personalities.

4) If you want to use CRS317 as router, then you definitely want to go L3HW way. Without L3HW this device has shitty routing performance.
 
Dulcow
just joined
Topic Author
Posts: 9
Joined: Thu Nov 30, 2023 12:13 pm

Re: VLAN switching and routing with bonds

Sun Mar 24, 2024 5:13 pm

1) Default setting is frame-types=admit-all ... so if it's not changed explicitly according to needs, it'll remain that way.

2) Do as you see fit. IMO access to management VLAN should be as restricted as possible but also depends on particular use case.

3) Bridge is (also) interface which allows ROS to communicate with nerworks passing bridge the switch-like entity. To enable it, bridge port has to be made member of appropriate VLAN(s). See this explanation of different bridge personalities.

4) If you want to use CRS317 as router, then you definitely want to go L3HW way. Without L3HW this device has shitty routing performance.

Thanks for the nudge, it does help indeed ;-)

For now, I can ping 192.168.99.2 and 192.168.99.3 (management interface of both switches) when my PC is connected in the LAN VLAN (30) on CRS328 (for instance port ether17)

For some reasons though, I cannot access 192.168.10.210 which is connected on sfpplus1 (CRS317). I have 192.168.30.2 as the gateway on my client (PC).

Here are the full configurations (looks pretty much like the example I initially linked):
# 1970-01-02 03:18:01 by RouterOS 7.14.1
# software id = IZBV-VVB6
#
# model = CRS317-1G-16S+
# serial number = 
/interface bridge
add frame-types=admit-only-vlan-tagged name=bridge vlan-filtering=yes
/interface vlan
add interface=bridge name="vlan10 [DMZ]" vlan-id=10
add interface=bridge name="vlan20 [IOT]" vlan-id=20
add interface=bridge name="vlan30 [LAN]" vlan-id=30
add interface=bridge name="vlan99 [ADM]" vlan-id=99
/interface bonding
add lacp-rate=1sec mode=802.3ad name=bond_15-16 slaves=sfp-sfpplus15,sfp-sfpplus16 transmit-hash-policy=layer-2-and-3
/ip hotspot profile
set [ find default=yes ] html-directory=hotspot
/ip smb users
set [ find default=yes ] disabled=yes
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus1 pvid=10
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus2 pvid=10
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus3 pvid=10
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus4 pvid=10
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus5 pvid=20
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus6 pvid=20
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus7 pvid=20
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus8 pvid=20
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus9 pvid=30
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus10 pvid=30
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus11 pvid=30
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus12 pvid=30
add bridge=bridge frame-types=admit-only-vlan-tagged interface=bond_15-16
/ip firewall connection tracking
set udp-timeout=10s
/interface bridge vlan
add bridge=bridge tagged=bridge,bond_15-16 untagged=sfp-sfpplus1 vlan-ids=10
add bridge=bridge tagged=bridge,bond_15-16 vlan-ids=20
add bridge=bridge tagged=bridge,bond_15-16 untagged=sfp-sfpplus9 vlan-ids=30
add bridge=bridge tagged=bridge,bond_15-16 vlan-ids=99
/ip address
add address=192.168.10.2/24 interface="vlan10 [DMZ]" network=192.168.10.0
add address=192.168.20.2/24 interface="vlan20 [IOT]" network=192.168.20.0
add address=192.168.30.2/24 interface="vlan30 [LAN]" network=192.168.30.0
add address=192.168.99.2/24 interface="vlan99 [ADM]" network=192.168.99.0
/ip smb shares
set [ find default=yes ] directory=/flash/pub
/system note
set show-at-login=no
/system routerboard settings
set boot-os=router-os

# 1970-01-02 01:38:25 by RouterOS 7.14.1
# software id = H7ZT-96GQ
#
# model = CRS328-24P-4S+
# serial number = 
/interface bridge
add frame-types=admit-only-vlan-tagged name=bridge vlan-filtering=yes
/interface vlan
add interface=bridge name="vlan99 [ADM]" vlan-id=99
/interface bonding
add lacp-rate=1sec mode=802.3ad name=bond_3-4 slaves=sfp-sfpplus3,sfp-sfpplus4 transmit-hash-policy=layer-2-and-3
/ip hotspot profile
set [ find default=yes ] html-directory=hotspot
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether1 pvid=10
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether2 pvid=10
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether3 pvid=10
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether4 pvid=10
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether5 pvid=10
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether6 pvid=10
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether7 pvid=10
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether9 pvid=20
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether10 pvid=20
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether11 pvid=20
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether12 pvid=20
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether13 pvid=20
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether14 pvid=20
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether15 pvid=20
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether17 pvid=30
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether18 pvid=30
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether19 pvid=30
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether20 pvid=30
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether21 pvid=30
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether22 pvid=30
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether23 pvid=30
add bridge=bridge frame-types=admit-only-vlan-tagged interface=bond_3-4
/ip neighbor discovery-settings
set discover-interface-list=!dynamic
/interface bridge vlan
add bridge=bridge tagged=bond_3-4 vlan-ids=10
add bridge=bridge tagged=bond_3-4 vlan-ids=20
add bridge=bridge tagged=bond_3-4 vlan-ids=30
add bridge=bridge tagged=bridge,bond_3-4 vlan-ids=99
/ip address
add address=192.168.99.3/24 interface="vlan99 [ADM]" network=192.168.99.0
/ip route
add gateway=192.168.99.2
/system note
set show-at-login=no
/system routerboard settings
set boot-os=router-os
Last edited by Dulcow on Sun Mar 24, 2024 9:14 pm, edited 2 times in total.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 19404
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: CRS317 + CRS328 - InterVLAN routing with L3HW

Sun Mar 24, 2024 7:21 pm

I disagree with MKX, cannot recommend, in general, a switch for routing.
The CRS317, can route up to about 400Mbps but thats it, so depending upon your internet connectivity????????
The RB50009 is designed to router and would be my first choice and then feed one of the switches with the SFP+ port.
 
tangent
Forum Guru
Forum Guru
Posts: 1404
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: CRS317 + CRS328 - InterVLAN routing with L3HW

Sun Mar 24, 2024 8:26 pm

The CRS317, can route up to about 400Mbps but thats it

You're talking software routing. @mkx is talking L3HW routing, where everything gets offloaded to the switch chip. The CRS317 is one of the handful devices in MT's lineup that can do this well, but even then, it isn't capable of much of what you can do in software routing.

If you need any of that, then yes, the device lands hard on its poorly-padded processing posterior.
 
Dulcow
just joined
Topic Author
Posts: 9
Joined: Thu Nov 30, 2023 12:13 pm

Re: CRS317 + CRS328 - InterVLAN routing with L3HW

Sun Mar 24, 2024 9:14 pm

The CRS317, can route up to about 400Mbps but thats it

You're talking software routing. @mkx is talking L3HW routing, where everything gets offloaded to the switch chip. The CRS317 is one of the handful devices in MT's lineup that can do this well, but even then, it isn't capable of much of what you can do in software routing.

If you need any of that, then yes, the device lands hard on its poorly-padded processing posterior.

That was my understanding of this kind of product, it can do near wire-speed routing with some limitations. I wanted to see if my small setup could fit those requirements or if I was in need of something bigger (like a proper CCR unit).

Does any of you would know why inter-VLAN routing does not work from scratch (not talking about L3HW here)? All my VLANs are onto the same bridge, ports are tagged as they should. Still, I can reach both server and PC from the CRS317 but I cannot ping server to PC through CRS317 and vice-versa.

I checked with my RB5009 currently being used for my Internet access, and I don't see any particular differences. I remember very well that by default, all VLANs were talking to each other the moment they were added to the bridge.

Thanks,

D.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 19404
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: CRS317 + CRS328 - InterVLAN routing with L3HW

Sun Mar 24, 2024 9:16 pm

As for your setups. If 317 is a router, where are the router settings?
Make up your mind!
If the RB5009 is just for internet then you need to have double nat and make up all the subnets on the switch.
The switch will get a private IP as its WAN ( private IP from 5009 default subnet I guess )

If the plan is to have the 5009 create all the vlans then the only vlan you need to identify ( not create subnet ) is the management vlans of which the two switches will get their own IP address from.
In that case I can have a look.......... Assuming a standard switch setup then,

1. Get rid of frame types on bridge itself, use frame types on /interface bridge ports
2. Use off bridge port to do all configuration, makes life easy when bridge burps.
3. Add ingress filtering to /bridge ports
4. Assuming the BOND, 15,16 are ties to the CRS354 but where is the port from 5009 ??????????? sigh......................
will use etheport 14 for that........... will use 10.100.1000 gig port for off bridge access.
5. Only tag bridge for management vlan at /interface bridge vlans.
# 1970-01-02 03:18:01 by RouterOS 7.14.1
# model = CRS317-1G-16S
/interface bridge
add name=bridge vlan-filtering=yes
/interface ethernet
set [ find default-name=ether1 ] name=off-bridge-access
/interface list
add name=MGMT
/interface vlan
add interface=bridge name="vlan10 [DMZ]" vlan-id=10
add interface=bridge name="vlan20 [IOT]" vlan-id=20
add interface=bridge name="vlan30 [LAN]" vlan-id=30
add interface=bridge name="vlan99 [ADM]" vlan-id=99
/interface bonding
add lacp-rate=1sec mode=802.3ad name=bond_15-16 slaves=sfp-sfpplus15,sfp-sfpplus16 transmit-hash-policy=layer-2-and-3
/interface bridge port
add bridge=bridge ingress-filtering=yes  frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus1 pvid=10
add bridge=bridge ingress-filtering=yes  frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus2 pvid=10
add bridge=bridge ingress-filtering=yes frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus3 pvid=10
add bridge=bridge ingress-filtering=yes  frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus4 pvid=10
add bridge=bridge ingress-filtering=yes  frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus5 pvid=20
add bridge=bridge ingress-filtering=yes  frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus6 pvid=20
add bridge=bridge ingress-filtering=yes  frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus7 pvid=20
add bridge=bridge ingress-filtering=yes frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus8 pvid=20
add bridge=bridge ingress-filtering=yes frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus9 pvid=30
add bridge=bridge ingress-filtering=yes  frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus10 pvid=30
add bridge=bridge ingress-filtering=yes  frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus11 pvid=30
add bridge=bridge ingress-filtering=yes  frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus12 pvid=30
add bridge=bridge ingress-filtering=yes frame-types=admit-only-vlan-tagged interface=sfp-sfpplus14  comment="Trunk To RB5009"
add bridge=bridge ingress-filtering=yes frame-types=admit-only-vlan-tagged interface=bond_15-16  comment="Trunk to CRS354"
/ip firewall connection tracking
set udp-timeout=10s
/interface bridge vlan
add bridge=bridge tagged=sfp-sfpplus14,bond_15-16  untagged=sfp-sfpplus1,sfp-sfpplus2,sfp-sfpplus,3sfp-sfpplus4  vlan-ids=10
add bridge=bridge tagged=sfp-sfpplus14,bond_15-16  untagged=sfp-sfpplus5,sfp-sfpplus6,sfp-sfpplus7,sfp-sfpplus8  vlan-ids=20
add bridge=bridge tagged=sfp-sfpplus14,bond_15-16  untagged=sfp-sfpplus9,sfp-sfpplus10,sfp-sfpplus11,sfp-sfpplus12  vlan-ids=30
add bridge=bridge tagged=bridge,sfp-sfpplus14,bond_15-16 vlan-ids=99
/ip neighbor discovery-settings
set discover-interface-list=MGMT
/interface list members
add interface="vlan99 [ADM]"  list=MGMT
add interface=off-bridge-access list=MGMT
/ip address
add address=192.168.99.2/24 interface="vlan99 [ADM]" network=192.168.99.0
add address=192.168.55.1/24 inteface=off-bridge-access  network=192.168.55.0
/ip dns
set allow-remote-requests=yes servers=192.168.99.1
/ip route
add dst-address=0.0.0.0/0 gateway=192.168.99.1 routing-table=main
/ip smb shares
set [ find default=yes ] directory=/flash/pub
/tool mac-server
set allowed-interface-list=NONE
/tool mac-server mac-winbox
set allowed-interface-list=MGMT
Last edited by anav on Sun Mar 24, 2024 9:41 pm, edited 2 times in total.
 
tangent
Forum Guru
Forum Guru
Posts: 1404
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: CRS317 + CRS328 - InterVLAN routing with L3HW

Sun Mar 24, 2024 9:21 pm

Fair question, @anav. The first rule on the page I linked to says "l3-hw-offloading=yes". Where's that in your config, @Dulcow?
 
Dulcow
just joined
Topic Author
Posts: 9
Joined: Thu Nov 30, 2023 12:13 pm

Re: CRS317 + CRS328 - InterVLAN routing with L3HW

Sun Mar 24, 2024 9:24 pm

As for your setups. If 317 is a router, where are the router settings?
Make up your mind!

AFAIK, the bridge on CRS317 has an IP in all VLANs which should be enough to enabled inter-VLAN routing (I'm not talking about performance here, just to trying to make it work first).

In this page (CRS317 also used but not as router), it is never mentioned any L3HW offloading => https://help.mikrotik.com/docs/display/ ... with+Bonds

Are there things we have to do in hardware that we cannot do in software on Mikrotik?

Thanks,

D.
 
tangent
Forum Guru
Forum Guru
Posts: 1404
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: CRS317 + CRS328 - InterVLAN routing with L3HW

Sun Mar 24, 2024 9:28 pm

Are there things…we cannot do in software on Mikrotik?

Go fast. 😛

That's it, as far as I'm aware.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 19404
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: CRS317 + CRS328 - InterVLAN routing with L3HW

Sun Mar 24, 2024 9:44 pm

I have no idea what you are trying to do................... but the setup I gave you should enable HW offloading on the bridge, its done. Intervlan routing is something that is foreign to me. Either the unit is a switch or a router in terms of setup............. it can or cannot have hw offloading.

Do not confuse yourself, there is no need to identify any vlans other than management vlan on the switch and that is because the switch needs its IP address from that subnet.

The only difference in the setup I would provide for the 354 is that you have an incoming BOND (taking up two SPF+ ports ) as trunk and maybe no other trunk ports.
Plus the IP address will likely by 192.168.99.3/24..................... also should have an off bridge access for this switch as well.
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11646
Joined: Thu Mar 03, 2016 10:23 pm

Re: CRS317 + CRS328 - InterVLAN routing with L3HW

Mon Mar 25, 2024 8:26 am

Are your LAN devices (in all VLANs) set up to use CRS317 as gateway?
 
Dulcow
just joined
Topic Author
Posts: 9
Joined: Thu Nov 30, 2023 12:13 pm

Re: CRS317 + CRS328 - InterVLAN routing with L3HW

Mon Mar 25, 2024 2:59 pm

@anav, thanks for the detailed answer ;-)

1) Frame types on the bridge is to forbid the default VLAN ID =1 traffic if I understood everything right. It is part of the MikroTik VLAN tutorial and I have been using this on my current RB5009 setup.

2) I learnt this one the hard way when I was setting up the RB5009 initially (I had to wipe it and reconfigure it...)

3) Ingress filtering is checked by default and does not appear in the configuration file

Image

4) My point was to setup inter-VLAN routing first before adding the NAT and the rest. I found what my issue was on Windows side and now, inter-VLAN routing works fine (all VLANs can access all VLANs, I will lock that up later on). I changed all VLANs and IPs to make sure there are no overlap with any of my currently used setup. I will change configuration before putting everything live.

5) I have untagged ports on CRS317 as well (future 10G devices), I need to tag those ports and have some IP addresses set if I want to be able to route between VLANs.

To give a bit more context and background about my setup: I'm running a little homelab at home, mainly built around NUCs right now but I will retire those to get devices that can support 10G NIC. My current Ceph cluster is on 2.5Gbe and consumer drives, it works but it is far from being optimal.

"An image is worth a thousand words"

Image
Image

Here is what I'm trying to achieve right now:
Image

I managed to get the L3HW offloading working on both switches. Just need to remember to disable/reenable L3HW to get modifications taken in (I think the rules are "copied" over to the switch chip when you do that).

I'm now connected to the Internet via the RB5009 (sfp-sfpplus13 on CRS317). I have set the MTUs and I will now try to test with iperf3 to see how much I get performance wise.
# 2024-03-25 13:21:39 by RouterOS 7.14.1
# software id = IZBV-VVB6
#
# model = CRS317-1G-16S+
# serial number = 
/interface bridge
add frame-types=admit-only-vlan-tagged name=bridge vlan-filtering=yes
/interface ethernet
set [ find default-name=sfp-sfpplus1 ] l2mtu=10218 mtu=10218
set [ find default-name=sfp-sfpplus2 ] l2mtu=10218 mtu=10218
set [ find default-name=sfp-sfpplus3 ] l2mtu=10218 mtu=10218
set [ find default-name=sfp-sfpplus4 ] l2mtu=10218 mtu=10218
set [ find default-name=sfp-sfpplus5 ] l2mtu=10218 mtu=10218
set [ find default-name=sfp-sfpplus6 ] l2mtu=10218 mtu=10218
set [ find default-name=sfp-sfpplus7 ] l2mtu=10218 mtu=10218
set [ find default-name=sfp-sfpplus8 ] l2mtu=10218 mtu=10218
set [ find default-name=sfp-sfpplus10 ] l2mtu=10218 mtu=10218
set [ find default-name=sfp-sfpplus11 ] l2mtu=10218 mtu=10218
set [ find default-name=sfp-sfpplus12 ] l2mtu=10218 mtu=10218
set [ find default-name=sfp-sfpplus14 ] l2mtu=10218 mtu=10218
set [ find default-name=sfp-sfpplus15 ] l2mtu=10218 mtu=10218
set [ find default-name=sfp-sfpplus16 ] l2mtu=10218 mtu=10218
/interface vlan
add interface=bridge name=vlan110 vlan-id=110
add interface=bridge name=vlan120 vlan-id=120
add interface=bridge name=vlan130 vlan-id=130
add interface=bridge name=vlan199 vlan-id=199
/interface bonding
add lacp-rate=1sec mode=802.3ad name=bond_15-16 slaves=sfp-sfpplus15,sfp-sfpplus16 transmit-hash-policy=layer-2-and-3
/interface ethernet switch
set 0 l3-hw-offloading=yes
/interface ethernet switch port
set 12 l3-hw-offloading=no
/ip hotspot profile
set [ find default=yes ] html-directory=hotspot
/ip smb users
set [ find default=yes ] disabled=yes
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus1 pvid=110
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus2 pvid=110
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus3 pvid=110
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus4 pvid=110
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus5 pvid=120
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus6 pvid=120
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus7 pvid=120
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus8 pvid=120
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus9 pvid=130
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus10 pvid=130
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus11 pvid=130
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=sfp-sfpplus12 pvid=130
add bridge=bridge frame-types=admit-only-vlan-tagged interface=bond_15-16
/ip firewall connection tracking
set udp-timeout=10s
/interface bridge vlan
add bridge=bridge tagged=bridge,bond_15-16 untagged=sfp-sfpplus4 vlan-ids=110
add bridge=bridge tagged=bridge,bond_15-16 vlan-ids=120
add bridge=bridge tagged=bridge,bond_15-16 untagged=sfp-sfpplus9,sfp-sfpplus12 vlan-ids=130
add bridge=bridge tagged=bridge,bond_15-16 vlan-ids=199
/ip address
add address=192.168.110.2/24 interface=vlan110 network=192.168.110.0
add address=192.168.120.2/24 interface=vlan120 network=192.168.120.0
add address=192.168.130.2/24 interface=vlan130 network=192.168.130.0
add address=192.168.199.2/24 interface=vlan199 network=192.168.199.0
add address=192.168.30.254/24 interface=sfp-sfpplus13 network=192.168.30.0
/ip dns
set servers=192.168.10.101
/ip firewall filter
add action=fasttrack-connection chain=forward connection-state=established,related hw-offload=yes
add action=accept chain=forward connection-state=established,related
/ip firewall nat
add action=masquerade chain=srcnat out-interface=sfp-sfpplus13
/ip route
add gateway=192.168.30.1
/ip smb shares
set [ find default=yes ] directory=/flash/pub
/system clock
set time-zone-name=Europe/Paris
/system note
set show-at-login=no
/system routerboard settings
set boot-os=router-os

# 2024-03-25 13:58:10 by RouterOS 7.14.1
# software id = H7ZT-96GQ
#
# model = CRS328-24P-4S+
# serial number = 
/interface bridge
add frame-types=admit-only-vlan-tagged name=bridge vlan-filtering=yes
/interface ethernet
set [ find default-name=sfp-sfpplus1 ] l2mtu=10218 mtu=10218
set [ find default-name=sfp-sfpplus2 ] l2mtu=10218 mtu=10218
set [ find default-name=sfp-sfpplus3 ] l2mtu=10218 mtu=10218
set [ find default-name=sfp-sfpplus4 ] l2mtu=10218 mtu=10218
/interface vlan
add interface=bridge name=vlan199 vlan-id=199
/interface bonding
add lacp-rate=1sec mode=802.3ad name=bond_3-4 slaves=sfp-sfpplus3,sfp-sfpplus4 transmit-hash-policy=layer-2-and-3
/interface ethernet switch
set 0 l3-hw-offloading=yes
/ip hotspot profile
set [ find default=yes ] html-directory=hotspot
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether1 pvid=110
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether2 pvid=110
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether3 pvid=110
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether4 pvid=110
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether5 pvid=110
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether6 pvid=110
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether7 pvid=110
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether9 pvid=120
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether10 pvid=120
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether11 pvid=120
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether12 pvid=120
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether13 pvid=120
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether14 pvid=120
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether15 pvid=120
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether17 pvid=130
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether18 pvid=130
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether19 pvid=130
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether20 pvid=130
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether21 pvid=130
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether22 pvid=130
add bridge=bridge frame-types=admit-only-untagged-and-priority-tagged interface=ether23 pvid=130
add bridge=bridge frame-types=admit-only-vlan-tagged interface=bond_3-4 internal-path-cost=10 path-cost=10
/ip neighbor discovery-settings
set discover-interface-list=!dynamic
/interface bridge vlan
add bridge=bridge tagged=bond_3-4 vlan-ids=110
add bridge=bridge tagged=bond_3-4 vlan-ids=120
add bridge=bridge tagged=bond_3-4 vlan-ids=130
add bridge=bridge tagged=bridge,bond_3-4 vlan-ids=199
/ip address
add address=192.168.199.3/24 interface=vlan199 network=192.168.199.0
/ip dns
set servers=192.168.10.101
/ip route
add gateway=192.168.199.2
/system clock
set time-zone-name=Europe/Paris
/system note
set show-at-login=no
/system routerboard settings
set boot-os=router-os
Your point is something that I did not realise: double nat. Do you have any ideas how I could prevent that? I could remove the NAT from RB5009 and treat it as firewall/bridge, no?

Thanks,

D.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 19404
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: CRS317 + CRS328 - InterVLAN routing with L3HW

Mon Mar 25, 2024 3:30 pm

You have bad info and are not completing the config as provided on the second bridge device.
neighbours discovery, bridge itself still has frame types, IP DNS server wrong etc etc.

The first one acting as a router is a complete abomination mixing non CRS3 and CRS3 switch setup types and MKX politely, via a question, pointed out how your router setup was woefully inadequate.
You are barking up a wrong tree here.
Use the 5009 for routing and DHCP and use the switches for switching is my advice.
I am moving on.
 
Dulcow
just joined
Topic Author
Posts: 9
Joined: Thu Nov 30, 2023 12:13 pm

Re: CRS317 + CRS328 - InterVLAN routing with L3HW

Mon Mar 25, 2024 3:47 pm

You have bad info and are not completing the config as provided on the second bridge device.
neighbours discovery, bridge itself still has frame types, IP DNS server wrong etc etc.

The first one acting as a router is a complete abomination mixing non CRS3 and CRS3 switch setup types and MKX politely, via a question, pointed out how your router setup was woefully inadequate.
You are barking up a wrong tree here.
Use the 5009 for routing and DHCP and use the switches for switching is my advice.
I am moving on.

I came here to seek help and see if/how people are using those products. I don't know much about MikroTik nor network configurations, homelab is a way for me to learn. And to be honest, I don't feel that I'm one barking right now...

I will go and look for help somewhere else.

D.
 

Who is online

Users browsing this forum: Majestic-12 [Bot] and 31 guests