Community discussions

MikroTik App
 
DonZalmrol
just joined
Topic Author
Posts: 9
Joined: Thu Mar 19, 2020 9:00 pm

How to set up a trunk between Miktrotik and Brocade switch and other devices with LACP trunks

Thu Mar 19, 2020 9:22 pm

Hello all,

My first time using Miktrotik and posting here, I've purchased a CRS317-1G-16S+RM switch and I'm starting to set it up to work with my Brocade ICX6450-48P and servers and a NAS, all configured in LACP with 10GB SFP+. I've been trying to get it set up on my own, but I believe I'm overlooking something. Seems I can't get my VLANs to go from my ICX to the CRS, the ICX already is configured properly in LACP with the VLANs trunked. How do I know this? The LACP trunk is in use for my primary server and that works flawlessy for the past year.

So how can I implement the following scenario?
Network.png
Things I've tried:
  • Created the vlans under the interfaces and bridge tabs
  • Created a new bridge with vlan filtering enabled
  • Created bonds and set them up using 802.3ad which is supported by all my devices
  • Under bridge vlans added the interfaces and bonds as tagged and allowed as a test vlans 1-4096
I believe I'm not that far from a working a scenario, but I'm missing something, googling it gives me different answers then what I'm looking for.

My config is back to defaults with only the bonds still added, if anyone could help out with one of the trunks (e.g. ICX <-> CRS) that would get me started for the other devices as my topology is "basic" in a certain sense.

Many thanks in advance!
You do not have the required permissions to view the files attached to this post.
 
DonZalmrol
just joined
Topic Author
Posts: 9
Joined: Thu Mar 19, 2020 9:00 pm

Re: How to set up a trunk between Miktrotik and Brocade switch and other devices with LACP trunks

Fri Mar 20, 2020 9:01 am

Update: Using this guide to see I can get further, though it seems similar to what I tried already.

viewtopic.php?f=13&t=143620
 
 
DonZalmrol
just joined
Topic Author
Posts: 9
Joined: Thu Mar 19, 2020 9:00 pm

Re: How to set up a trunk between Miktrotik and Brocade switch and other devices with LACP trunks

Sun Mar 22, 2020 3:45 pm

I've followed the guide but I can't seem to get it up and running correctly.

Have the following test set up:
bonds:
[admin@MikroTik] /interface bonding> print
Flags: X - disabled, R - running
0 name="bond_3-4" mtu=1500 mac-address=C4:AD:34:56:A3:B4 arp=enabled arp-timeout=auto
slaves=sfp-sfpplus3,sfp-sfpplus4 mode=802.3ad primary=none link-monitoring=mii arp-interval=100ms
arp-ip-targets="" mii-interval=100ms down-delay=0ms up-delay=0ms lacp-rate=30secs
transmit-hash-policy=layer-2 min-links=0

1 name="bond_5-6" mtu=1500 mac-address=C4:AD:34:56:A3:B6 arp=enabled arp-timeout=auto
slaves=sfp-sfpplus5,sfp-sfpplus6 mode=802.3ad primary=none link-monitoring=mii arp-interval=100ms
arp-ip-targets="" mii-interval=100ms down-delay=0ms up-delay=0ms lacp-rate=30secs
transmit-hash-policy=layer-2 min-links=0

These two bonds are used for: bond_3-4 for connecting my Mikrotik to my Brocade switch, bond_5-6 for connecting my Mikrotik to my server.

Bridge:
[admin@MikroTik] /interface bridge> print
Flags: X - disabled, R - running
0 R name="bridge" mtu=auto actual-mtu=1500 l2mtu=1592 arp=enabled arp-timeout=auto mac-address=C4:AD:34:56:A3:B4
protocol-mode=rstp fast-forward=yes igmp-snooping=no auto-mac=yes ageing-time=5m priority=0x8000
max-message-age=20s forward-delay=15s transmit-hold-count=6 vlan-filtering=no dhcp-snooping=no

Tried setting is to vlan-filtering=yes which results in no ether1 (management port) webgui access.
vlan-filtering=no results in working webgui access...

Bridge ports:
[admin@MikroTik] /interface bridge port> print
Flags: X - disabled, I - inactive, D - dynamic, H - hw-offload
# INTERFACE BRIDGE HW PVID PRIORITY PATH-COST INTERNAL-PATH-COST HORIZON
0 I H sfp-sfpplus1 bridge yes 100 0x80 10 10 none
1 I H sfp-sfpplus2 bridge yes 5 0x80 10 10 none
2 I bond_3-4 bridge yes 1 0x80 10 10 none
3 I bond_5-6 bridge yes 1 0x80 10 10 none
4 H ether1 bridge yes 30 0x80 10 10 none

ports are set and configured with hw-offload and pvids (as in the guide)

Bridge vlan:
[admin@MikroTik] /interface bridge vlan> print
Flags: X - disabled, D - dynamic
# BRIDGE VLAN-IDS CURRENT-TAGGED CURRENT-UNTAGGED
0 bridge 1
5
10
20
30
100
150
200

I have no clue at the moment, any tips would be greatly appreciated!
 
tdw
Member Candidate
Member Candidate
Posts: 279
Joined: Sat May 05, 2018 11:55 am

Re: How to set up a trunk between Miktrotik and Brocade switch and other devices with LACP trunks

Sun Mar 22, 2020 5:03 pm

The output of /export hide-sensitive would let us see what you have currently got configured, none of the PVID or bridge VLAN settings have any effect unless vlan-filtering=yes
 
DonZalmrol
just joined
Topic Author
Posts: 9
Joined: Thu Mar 19, 2020 9:00 pm

Re: How to set up a trunk between Miktrotik and Brocade switch and other devices with LACP trunks

Sun Mar 22, 2020 6:53 pm

Hi, output is (very basic = test setup):

[admin@MikroTik] > export hide-sensitive
# jan/02/1970 03:42:10 by RouterOS 6.46.4
# software id = xxxxxxxx
#
# model = CRS317-1G-16S+
# serial number = xxxxxxxx
/interface bridge
add name=bridge
/interface ethernet
set [ find default-name=sfp-sfpplus4 ] mac-address=C4:AD:34:56:A3:B4
set [ find default-name=sfp-sfpplus6 ] mac-address=C4:AD:34:56:A3:B6
/interface bonding
add mode=802.3ad name=bond_3-4 slaves=sfp-sfpplus3,sfp-sfpplus4
add mode=802.3ad name=bond_5-6 slaves=sfp-sfpplus5,sfp-sfpplus6
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/interface bridge port
add bridge=bridge interface=sfp-sfpplus1 pvid=100
add bridge=bridge interface=sfp-sfpplus2 pvid=5
add bridge=bridge interface=bond_3-4
add bridge=bridge interface=bond_5-6
add bridge=bridge interface=ether1 pvid=30
/interface bridge vlan
add bridge=bridge tagged=bridge vlan-ids=1,5,10,20,30,100,150,200
/ip address
add address=10.0.90.5/24 interface=ether1 network=10.0.90.0
/ip route
add distance=1 gateway=10.0.90.1
/system routerboard settings
set boot-os=router-os
Last edited by DonZalmrol on Mon Mar 23, 2020 7:17 pm, edited 2 times in total.
 
tdw
Member Candidate
Member Candidate
Posts: 279
Joined: Sat May 05, 2018 11:55 am

Re: How to set up a trunk between Miktrotik and Brocade switch and other devices with LACP trunks

Sun Mar 22, 2020 10:51 pm

A couple of obvious errors...

The IP address should be attached to the bridge, not member interfaces (in this case ether1)

You have only configured VLAN IDs on the bridge itself, not any of the member interfaces. The wiki documentation is slightly out of date, untagged membership is configured dynamically from the pvid= values in /interface bridge port. I'm also not a fan of specifying multiple VLAN IDs in /interface bridge vlan and there is mention of it not playing well with untagged memberships.

The following changes will pass VLANs 5,10,20,30,100,150,200 and untagged traffic between the two bonds; makes VLAN 100 available untagged on sfp-sfpplus1, VLAN 5 available untagged on sfp-sfpplus2 and management access via VLAN 30.
/interface bridge
add name=bridge pvid=30
....
/interface bridge vlan
add bridge=bridge tagged=bond_3-4,bond_5-6 vlan-ids=5
add bridge=bridge tagged=bond_3-4,bond_5-6 vlan-ids=10
add bridge=bridge tagged=bond_3-4,bond_5-6 vlan-ids=20
add bridge=bridge tagged=bond_3-4,bond_5-6 vlan-ids=30
add bridge=bridge tagged=bond_3-4,bond_5-6 vlan-ids=100
add bridge=bridge tagged=bond_3-4,bond_5-6 vlan-ids=150
add bridge=bridge tagged=bond_3-4,bond_5-6 vlan-ids=200
/ip address
add address=10.0.90.5/24 interface=bridge network=10.0.90.0
....

Others may prefer not changing the bridge PVID and adding a VLAN to the bridge instead - a more consistent approach if you need the CRS CPU to have access to multiple VLANs, but unnecessary for a single one.

It may be best to use one of the spare ports and MAC winbox access so you don't cut yourself off whilst making changes.
 
DonZalmrol
just joined
Topic Author
Posts: 9
Joined: Thu Mar 19, 2020 9:00 pm

Re: How to set up a trunk between Miktrotik and Brocade switch and other devices with LACP trunks

Mon Mar 23, 2020 10:36 pm

So step further, the webinterface is working again, the trunks still aren't.
Believe I'm missing something else.

[admin@MikroTik] > export hide-sensitive
# mar/23/2020 18:27:40 by RouterOS 6.46.4
# software id = XXXXXX
#
# model = CRS317-1G-16S+
# serial number = XXXXXX
/interface bridge
add name=bridge pvid=30 vlan-filtering=yes
/interface ethernet
set [ find default-name=sfp-sfpplus4 ] mac-address=C4:AD:34:56:A3:B4
set [ find default-name=sfp-sfpplus6 ] mac-address=C4:AD:34:56:A3:B6
/interface bonding
add mode=802.3ad name=bond_3-4 slaves=sfp-sfpplus3,sfp-sfpplus4
add mode=802.3ad name=bond_5-6 slaves=sfp-sfpplus5,sfp-sfpplus6
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile
set [ find default=yes ] html-directory=flash/hotspot
/interface bridge port
add bridge=bridge interface=ether1 pvid=30
add bridge=bridge interface=sfp-sfpplus1 pvid=100
add bridge=bridge interface=sfp-sfpplus2 pvid=5
add bridge=bridge interface=bond_3-4
add bridge=bridge interface=bond_5-6
/interface bridge vlan
add bridge=bridge tagged=bond_3-4,bond_5-6 vlan-ids=5
add bridge=bridge tagged=bond_3-4,bond_5-6 vlan-ids=10
add bridge=bridge tagged=bond_3-4,bond_5-6 vlan-ids=20
add bridge=bridge tagged=bond_3-4,bond_5-6 vlan-ids=30
add bridge=bridge tagged=bond_3-4,bond_5-6 vlan-ids=100
add bridge=bridge tagged=bond_3-4,bond_5-6 vlan-ids=150
add bridge=bridge tagged=bond_3-4,bond_5-6 vlan-ids=200
/ip address
add address=10.0.90.5/24 interface=bridge network=10.0.90.0
/ip dns
set servers=10.0.90.10,10.0.90.50
/ip route
add distance=1 gateway=10.0.90.1
/system clock
set time-zone-name=Europe/Brussels
/system ntp client
set enabled=yes primary-ntp=10.0.90.1 secondary-ntp=10.0.90.10
/system routerboard settings
set boot-os=router-os
 
DonZalmrol
just joined
Topic Author
Posts: 9
Joined: Thu Mar 19, 2020 9:00 pm

Re: How to set up a trunk between Miktrotik and Brocade switch and other devices with LACP trunks

Wed Mar 25, 2020 9:41 am

No one that sees the issue? :)
 
tdw
Member Candidate
Member Candidate
Posts: 279
Joined: Sat May 05, 2018 11:55 am

Re: How to set up a trunk between Miktrotik and Brocade switch and other devices with LACP trunks

Wed Mar 25, 2020 2:45 pm

Nothing obvious. What does the output of /interface bridge vlan print and /interface bonding print give?
 
DonZalmrol
just joined
Topic Author
Posts: 9
Joined: Thu Mar 19, 2020 9:00 pm

Re: How to set up a trunk between Miktrotik and Brocade switch and other devices with LACP trunks

Wed Mar 25, 2020 3:25 pm

This is the output below.

[admin@MikroTik] > interface bridge vlan print
Flags: X - disabled, D - dynamic
# BRIDGE VLAN-IDS CURRENT-TAGGED CURRENT-UNTAGGED
0 bridge 5
1 bridge 10
2 bridge 20
3 bridge 30 bridge
MGMT
4 bridge 100
5 bridge 150
6 bridge 200
7 D bridge 1 bond_3-4

[admin@MikroTik] > interface bonding print
Flags: X - disabled, R - running
0 name="bond_3-4" mtu=1500 mac-address=C4:AD:34:56:A3:B4 arp=enabled arp-timeout=auto
slaves=sfp-sfpplus3,sfp-sfpplus4 mode=802.3ad primary=none link-monitoring=mii arp-interval=100ms
arp-ip-targets="" mii-interval=100ms down-delay=0ms up-delay=0ms lacp-rate=30secs
transmit-hash-policy=layer-2-and-3 min-links=0

1 name="bond_5-6" mtu=1500 mac-address=C4:AD:34:56:A3:B6 arp=enabled arp-timeout=auto
slaves=sfp-sfpplus5,sfp-sfpplus6 mode=802.3ad primary=none link-monitoring=mii arp-interval=100ms
arp-ip-targets="" mii-interval=100ms down-delay=0ms up-delay=0ms lacp-rate=30secs
transmit-hash-policy=layer-2-and-3 min-links=0
 
tdw
Member Candidate
Member Candidate
Posts: 279
Joined: Sat May 05, 2018 11:55 am

Re: How to set up a trunk between Miktrotik and Brocade switch and other devices with LACP trunks

Wed Mar 25, 2020 3:55 pm

Hmm, the bonding interfaces do not appear to be running (no R after the index number), and only the bridge itself appearing in the bridge current tagged vlans.

What does /interface bridge vlan print detail (provides addition detail) and /interface bridge port print give?
 
DonZalmrol
just joined
Topic Author
Posts: 9
Joined: Thu Mar 19, 2020 9:00 pm

Re: How to set up a trunk between Miktrotik and Brocade switch and other devices with LACP trunks

Wed Mar 25, 2020 5:21 pm

It's down as there is nothing connected at the moment :)
My working environment is directly connected to the 2x10GB SPF+ ports on my Brocade ICX6450.

Once the test config works it will be used as drawn in my initial post.

For the testing I do unplug the DAC cables and insert them into the bonds_5-6, bond_3-4 are used for the link between the Brocade and the Mikrotik.

Requested outputs:
[admin@MikroTik] > interface bridge vlan print detail
Flags: X - disabled, D - dynamic
0 bridge=bridge vlan-ids=5 tagged=bond_3-4,bond_5-6,bond_7-8,bond_9-10,bond_11-12,bond_13-14,bond_15-16 untagged="" current-tagged="" current-untagged=""

1 bridge=bridge vlan-ids=10 tagged=bond_3-4,bond_5-6,bond_7-8,bond_9-10,bond_11-12,bond_13-14,bond_15-16 untagged="" current-tagged="" current-untagged=""

2 bridge=bridge vlan-ids=20 tagged=bond_3-4,bond_5-6,bond_7-8,bond_9-10,bond_11-12,bond_13-14,bond_15-16 untagged="" current-tagged="" current-untagged=""

3 bridge=bridge vlan-ids=30 tagged=bond_3-4,bond_5-6,bond_7-8,bond_9-10,bond_11-12,bond_13-14,bond_15-16 untagged="" current-tagged="" current-untagged=bridge,MGMT

4 bridge=bridge vlan-ids=100 tagged=bond_3-4,bond_5-6,bond_7-8,bond_9-10,bond_11-12,bond_13-14,bond_15-16 untagged="" current-tagged="" current-untagged=""

5 bridge=bridge vlan-ids=150 tagged=bond_3-4,bond_5-6,bond_7-8,bond_9-10,bond_11-12,bond_13-14,bond_15-16 untagged="" current-tagged="" current-untagged=""

6 bridge=bridge vlan-ids=200 tagged=bond_3-4,bond_5-6,bond_7-8,bond_9-10,bond_11-12,bond_13-14,bond_15-16 untagged="" current-tagged="" current-untagged=""

7 D bridge=bridge vlan-ids=1 tagged="" untagged="" current-tagged="" current-untagged=bond_3-4



[admin@MikroTik] > interface bridge port print
Flags: X - disabled, I - inactive, D - dynamic, H - hw-offload
# INTERFACE BRIDGE HW PVID PRIORITY PATH-COST INTERNAL-PATH-COST HORIZON
0 H MGMT bridge yes 30 0x80 10 10 none
1 I H sfp-sfpplus1 bridge yes 100 0x80 10 10 none
2 I H sfp-sfpplus2 bridge yes 5 0x80 10 10 none
3 I bond_3-4 bridge yes 1 0x80 10 10 none
4 I bond_5-6 bridge yes 1 0x80 10 10 none
 
tdw
Member Candidate
Member Candidate
Posts: 279
Joined: Sat May 05, 2018 11:55 am

Re: How to set up a trunk between Miktrotik and Brocade switch and other devices with LACP trunks

Wed Mar 25, 2020 6:14 pm

What is the MGMT interface, it doesn't appear in the previous config export?

That aside, the bridge port print line "3 I bond_3-4 bridge yes 1 0x80 10 10 none" shows bond3_4 is inactive, and also does not have hardware offloading which is odd as 802.3ad bonding interfaces can be hardware offloaded. What does the device on the other end of the bond_3-4 connection report the lin status to be?

The bridge vlan print shows which VLANs have been configured (tagged= & untagged=), and which are operational (current-tagged= & current-untagged=) - inactive interfaces are not added to the current entries.

Also, only bond_3-4 & bond_5-6 have been added to the bridge, so despite bond_7-8, bond_9-10, bond_11-12, bond_13-14 & bond_15-16 having bridge VLAN configurations they will not work as they are not members of the bridge.
 
DonZalmrol
just joined
Topic Author
Posts: 9
Joined: Thu Mar 19, 2020 9:00 pm

Re: How to set up a trunk between Miktrotik and Brocade switch and other devices with LACP trunks

Wed Mar 25, 2020 6:32 pm

I was working further on the switch and MGMT is the renamed ehter1 interface. Bonds_7-8 through Bonds_15-16 are still in progress.
I can remove these to make it make clean for you?

Seems HW offloading is enabled on the bonds.
2020-03-25 17_34_40-Window.png
You do not have the required permissions to view the files attached to this post.
 
tdw
Member Candidate
Member Candidate
Posts: 279
Joined: Sat May 05, 2018 11:55 am

Re: How to set up a trunk between Miktrotik and Brocade switch and other devices with LACP trunks

Wed Mar 25, 2020 6:58 pm

No that's all fine. It is odd that hw=yes but no H flag - possibly the way inactive bonding interfaces work.

The VLAN configuration all looks fine, finding out why the bonding interface is inactive is the next thing to do, what does /interface print show?

Who is online

Users browsing this forum: MSN [Bot] and 87 guests