Community discussions

 
inatec
just joined
Topic Author
Posts: 13
Joined: Wed May 02, 2018 3:47 pm

[OS 6.43.12] 4x CRS317-1G-16S: MSTP problems between two (link up / down)

Mon Mar 04, 2019 11:32 am

Hello,

we have problems between two switches and MSTP. The interfaces between them goes up and down every few seconds and sw-core-lan-04 is isolated for round about 2 seconds. The sw-core-lan-03 shows than: root-bridge: yes
  • Some base infos:
  • 4 x CRS317-1G-16S
  • All with RouterOS firmware 6.43.12
  • We use SFP+ modules from Ubiquity and Allnet
  • Bridge VLAN core-lan
  • Bridge VLAN management
Bildschirmfoto von 2019-03-04 09-40-29.png
  • sw-core-lan-01

sw-core-lan-01] >  /interface bridge print
Flags: X - disabled, R - running 
 0 R ;;; defconf
     name="core-lan" mtu=auto actual-mtu=1500 l2mtu=1592 arp=enabled arp-timeout=auto 
     mac-address=74:4D:28:3D:38:01 protocol-mode=mstp fast-forward=yes igmp-snooping=no auto-mac=no 
     admin-mac=74:4D:28:3D:38:01 ageing-time=5m priority=0x1000 max-message-age=20s forward-delay=15s 
     transmit-hold-count=6 region-name="QHA07" region-revision=1 max-hops=20 vlan-filtering=yes ether-type=0x8100 
     pvid=1 frame-types=admit-all ingress-filtering=no dhcp-snooping=no 

 1 R name="management" mtu=auto actual-mtu=1500 l2mtu=1592 arp=enabled arp-timeout=auto 
     mac-address=74:4D:28:3D:38:01 protocol-mode=rstp fast-forward=yes igmp-snooping=no auto-mac=yes 
     ageing-time=5m priority=0x8000 max-message-age=20s forward-delay=15s transmit-hold-count=6 vlan-filtering=no 
     dhcp-snooping=no

sw-core-lan-01] > /interface print detail
     14  RS ;;; sw-core-03-lan-p14
       name="sfp-sfpplus14" default-name="sfp-sfpplus14" type="ether" mtu=1500 actual-mtu=1500 l2mtu=1592 
       max-l2mtu=10218 mac-address=74:4D:28:3D:37:FE last-link-down-time=feb/28/2019 22:50:19 
       last-link-up-time=feb/28/2019 22:50:20 link-downs=1 

15  RS ;;;sw-core-lan-04-p16
       name="sfp-sfpplus15" default-name="sfp-sfpplus15" type="ether" mtu=1500 actual-mtu=1500 l2mtu=1592 
       max-l2mtu=10218 mac-address=74:4D:28:3D:37:FF last-link-up-time=feb/26/2019 17:18:01 link-downs=0 

16  RS ;;;sw-core-lan-02-p15
       name="sfp-sfpplus16" default-name="sfp-sfpplus16" type="ether" mtu=1500 actual-mtu=1500 l2mtu=1592 
       max-l2mtu=10218 mac-address=74:4D:28:3D:38:00 last-link-up-time=feb/26/2019 17:17:29 link-downs=0 
sw-core-lan-01] >  /interface bridge monitor core-lan  
                       ;;; defconf
                    state: enabled
      current-mac-address: 74:4D:28:3D:38:01
              root-bridge: yes
           root-bridge-id: 0x1000.74:4D:28:3D:38:01
  regional-root-bridge-id: 0x1000.74:4D:28:3D:38:01
           root-path-cost: 0
                root-port: none
               port-count: 12
    designated-port-count: 11

  • sw-core-lan-02

sw-core-lan-02] > /interface bridge print 
Flags: X - disabled, R - running 
 0 R ;;; defconf
     name="core-lan" mtu=auto actual-mtu=1500 l2mtu=1592 arp=enabled arp-timeout=auto mac-address=CC:2D:E0:A3:5B:33 
     protocol-mode=mstp fast-forward=yes igmp-snooping=no auto-mac=no admin-mac=CC:2D:E0:A3:5B:33 ageing-time=5m 
     priority=0x2000 max-message-age=20s forward-delay=15s transmit-hold-count=6 region-name="QHA07" 
     region-revision=1 max-hops=20 vlan-filtering=yes ether-type=0x8100 pvid=1 frame-types=admit-all 
     ingress-filtering=no dhcp-snooping=no 

 1 R name="management" mtu=auto actual-mtu=1500 l2mtu=1592 arp=enabled arp-timeout=auto mac-address=CC:2D:E0:A3:5B:33 
     protocol-mode=none fast-forward=no igmp-snooping=no auto-mac=yes ageing-time=5m vlan-filtering=no 
     dhcp-snooping=no
sw-core-lan-02] > /interface print detail

     14  RS ;;; sw-core-lan-04-p14
       name="sfp-sfpplus14" default-name="sfp-sfpplus14" type="ether" mtu=1500 actual-mtu=1500 l2mtu=1592 
       max-l2mtu=10218 mac-address=CC:2D:E0:A3:5B:30 last-link-up-time=feb/26/2019 17:18:01 link-downs=0 

15  RS ;;; sw-core-lan-01-p16
       name="sfp-sfpplus15" default-name="sfp-sfpplus15" type="ether" mtu=1500 actual-mtu=1500 l2mtu=1592 
       max-l2mtu=10218 mac-address=CC:2D:E0:A3:5B:31 last-link-up-time=feb/26/2019 17:17:29 link-downs=0 

16  RS ;;; sw-core-lan-03-p15
       name="sfp-sfpplus16" default-name="sfp-sfpplus16" type="ether" mtu=1500 actual-mtu=1500 l2mtu=1592 
       max-l2mtu=10218 mac-address=CC:2D:E0:A3:5B:32 last-link-down-time=mar/04/2019 08:38:59 
       last-link-up-time=mar/04/2019 08:39:00 link-downs=33 
sw-core-lan-02] >  /interface bridge monitor core-lan  
                       ;;; defconf
                    state: enabled
      current-mac-address: CC:2D:E0:A3:5B:33
              root-bridge: no
           root-bridge-id: 0x1000.74:4D:28:3D:38:01
  regional-root-bridge-id: 0x1000.74:4D:28:3D:38:01
           root-path-cost: 0
                root-port: sfp-sfpplus15
               port-count: 12
    designated-port-count: 11
        mst-config-digest: 36a394ba96d689fdbdc3264fea68bdb6
        
  • sw-core-lan-03

sw-core-lan-03] >  /interface bridge print
Flags: X - disabled, R - running
 0 R ;;; defconf
     name="core-lan" mtu=auto actual-mtu=1500 l2mtu=1592 arp=enabled arp-timeout=auto mac-address=CC:2D:E0:A3:5A:DE
     protocol-mode=mstp fast-forward=yes igmp-snooping=no auto-mac=no admin-mac=CC:2D:E0:A3:5A:DE ageing-time=5m
     priority=0x3000 max-message-age=20s forward-delay=15s transmit-hold-count=6 region-name="QHA07"
     region-revision=1 max-hops=20 vlan-filtering=yes ether-type=0x8100 pvid=1 frame-types=admit-all
     ingress-filtering=no dhcp-snooping=no

 1 R name="management" mtu=auto actual-mtu=1500 l2mtu=1592 arp=enabled arp-timeout=auto mac-address=CC:2D:E0:A3:5A:DE
     protocol-mode=rstp fast-forward=yes igmp-snooping=no auto-mac=yes ageing-time=5m priority=0x8000
     max-message-age=20s forward-delay=15s transmit-hold-count=6 vlan-filtering=no dhcp-snooping=no
     
sw-core-lan-03] > /interface print detail
14  RS ;;; qh-a07-sw-core-01-p14
       name="sfp-sfpplus14" default-name="sfp-sfpplus14" type="ether" mtu=1500 actual-mtu=1500 l2mtu=1592
       max-l2mtu=10218 mac-address=CC:2D:E0:A3:5A:DB last-link-down-time=feb/28/2019 23:46:52
       last-link-up-time=feb/28/2019 23:46:53 link-downs=5

15  RS ;;; qh-a07-sw-core-02-p16
       name="sfp-sfpplus15" default-name="sfp-sfpplus15" type="ether" mtu=1500 actual-mtu=1500 l2mtu=1592
       max-l2mtu=10218 mac-address=CC:2D:E0:A3:5A:DC last-link-down-time=mar/04/2019 08:39:10
       last-link-up-time=mar/04/2019 08:39:11 link-downs=27

16  RS ;;; qh-a07-sw-core-04-p15
       name="sfp-sfpplus16" default-name="sfp-sfpplus16" type="ether" mtu=1500 actual-mtu=1500 l2mtu=1592
       max-l2mtu=10218 mac-address=CC:2D:E0:A3:5A:DD last-link-down-time=mar/04/2019 10:21:05
       last-link-up-time=mar/04/2019 10:21:06 link-downs=3164

sw-core-lan-03] > /interface bridge monitor core-lan
                       ;;; defconf
                    state: enabled
      current-mac-address: CC:2D:E0:A3:5A:DE
              root-bridge: no
           root-bridge-id: 0x1000.74:4D:28:3D:38:01
  regional-root-bridge-id: 0x1000.74:4D:28:3D:38:01
           root-path-cost: 0
                root-port: sfp-sfpplus14
               port-count: 13
    designated-port-count: 10
        mst-config-digest: 36a394ba96d689fdbdc3264fea68bdb6
        

  • sw-core-lan-04

sw-core-lan-04] > /interface bridge print
Flags: X - disabled, R - running
 0 R ;;; defconf
     name="core-lan" mtu=auto actual-mtu=1500 l2mtu=1592 arp=enabled arp-timeout=auto mac-address=74:4D:28:39:01:85
     protocol-mode=mstp fast-forward=yes igmp-snooping=no auto-mac=no admin-mac=74:4D:28:39:01:85 ageing-time=5m
     priority=0x4000 max-message-age=20s forward-delay=15s transmit-hold-count=6 region-name="QHA07"
     region-revision=1 max-hops=20 vlan-filtering=yes ether-type=0x8100 pvid=1 frame-types=admit-all
     ingress-filtering=no dhcp-snooping=no

 1 R name="management" mtu=auto actual-mtu=1500 l2mtu=1592 arp=enabled arp-timeout=auto mac-address=74:4D:28:39:01:85
     protocol-mode=none fast-forward=yes igmp-snooping=no auto-mac=yes ageing-time=5m vlan-filtering=no
     dhcp-snooping=no
sw-core-lan-04] > /interface ethernet print  detail 

14 RS ;;; qh-a07-sw-core-lan-02-p14
      name="sfp-sfpplus14" default-name="sfp-sfpplus14" mtu=1500 l2mtu=1592 mac-address=74:4D:28:39:01:82 orig-mac-address=74:4D:28:39:01:82 arp=enabled arp-timeout=auto loop-protect=default loop-protect-status=off                       
      loop-protect-send-interval=5s loop-protect-disable-time=5m auto-negotiation=yes advertise=10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full full-duplex=yes tx-flow-control=off rx-flow-control=off speed=10Gbps             
      bandwidth=unlimited/unlimited switch=switch1

15 RS ;;; qh-a07-sw-core-lan-03-p16
      name="sfp-sfpplus15" default-name="sfp-sfpplus15" mtu=1500 l2mtu=1592 mac-address=74:4D:28:39:01:83 orig-mac-address=74:4D:28:39:01:83 arp=enabled arp-timeout=auto loop-protect=default loop-protect-status=off                       
      loop-protect-send-interval=5s loop-protect-disable-time=5m auto-negotiation=yes advertise=10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full full-duplex=yes tx-flow-control=off rx-flow-control=off speed=10Gbps             
      bandwidth=unlimited/unlimited switch=switch1

16 RS ;;; qh-a07-sw-core-lan-01-p15
      name="sfp-sfpplus16" default-name="sfp-sfpplus16" mtu=1500 l2mtu=1592 mac-address=74:4D:28:39:01:84 orig-mac-address=74:4D:28:39:01:84 arp=enabled arp-timeout=auto loop-protect=default loop-protect-status=off                       
      loop-protect-send-interval=5s loop-protect-disable-time=5m auto-negotiation=yes advertise=10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full full-duplex=yes tx-flow-control=off rx-flow-control=off speed=10Gbps             
      bandwidth=unlimited/unlimited switch=switch1


sw-core-lan-04] > /interface bridge monitor core-lan
                       ;;; defconf
                    state: enabled
      current-mac-address: 74:4D:28:39:01:85
              root-bridge: no
           root-bridge-id: 0x1000.74:4D:28:3D:38:01
  regional-root-bridge-id: 0x1000.74:4D:28:3D:38:01
           root-path-cost: 0
                root-port: sfp-sfpplus16
               port-count: 13
    designated-port-count: 10
        mst-config-digest: 36a394ba96d689fdbdc3264fea68bdb6
In our log, we have:
...
Mar  4 10:00:52 sw-core-lan-03 sfp-sfpplus16 link up (speed 10G, full duplex)
Mar  4 10:01:22 sw-core-lan-04 sfp-sfpplus15 link down
Mar  4 10:01:23 sw-core-lan-04 sfp-sfpplus15 link up (speed 10G, full duplex)
Mar  4 10:01:26 sw-core-lan-03 sfp-sfpplus16 link down
Mar  4 10:01:27 sw-core-lan-03 sfp-sfpplus16 link up (speed 10G, full duplex)
Mar  4 10:01:42 sw-core-lan-04 sfp-sfpplus15 link down
Mar  4 10:01:43 sw-core-lan-04 sfp-sfpplus15 link up (speed 10G, full duplex)
Mar  4 10:01:48 sw-core-lan-03 sfp-sfpplus16 link down
Mar  4 10:01:49 sw-core-lan-03 sfp-sfpplus16 link up (speed 10G, full duplex)
Mar  4 10:01:53 sw-core-lan-03 sfp-sfpplus16 link down
Mar  4 10:01:54 sw-core-lan-03 sfp-sfpplus16 link up (speed 10G, full duplex)
Mar  4 10:02:20 sw-core-lan-03 sfp-sfpplus16 link down
Mar  4 10:02:21 sw-core-lan-03 sfp-sfpplus16 link up (speed 10G, full duplex)
Mar  4 10:02:24 sw-core-lan-04 sfp-sfpplus15 link down
Mar  4 10:02:26 sw-core-lan-04 sfp-sfpplus15 link up (speed 10G, full duplex)
...
Maybe, we have a configuration error, or one of the SFTPs are bad.

Any suggestions ?
You do not have the required permissions to view the files attached to this post.
 
AVA
just joined
Posts: 21
Joined: Mon Jun 18, 2018 6:19 pm

Re: [OS 6.43.12] 4x CRS317-1G-16S: MSTP problems between two (link up / down)

Tue Mar 19, 2019 4:43 pm

Hi,

looks like a link problem, not an mstp error. We've observed on 10gbe links between 2 CRS317, that you have to use autonegotiation for stable links. With static settings 10gbe fullduplex we had link flappings like you.
 
User avatar
omega-00
Forum Guru
Forum Guru
Posts: 1166
Joined: Sat Jun 06, 2009 4:54 am
Location: Australia
Contact:

Re: [OS 6.43.12] 4x CRS317-1G-16S: MSTP problems between two (link up / down)

Sat Mar 23, 2019 3:31 am

I've just observed the exact same issue going from v6.42.10 -> v6.43.13 on a CCR1036

SFP+2 interface with a DAC cable connected to a CRS317 saw the link showing up and down according to the remote side while on the CCR it showed up the whole time.

Ended up downgrading to v6.42.12 and the issue went away.
brightwifi.com | mikrotik-routeros.com | MTCNA,MTCWE.MTCTCE | Give karma where due

Who is online

Users browsing this forum: No registered users and 48 guests