Community discussions

 
rualark
just joined
Topic Author
Posts: 10
Joined: Sun Nov 11, 2018 8:28 pm

Unstable 10Gbit connection of Mikrotik CSS326-24G-2S+RM with Mikrotik S+RJ10 SFP

Sun Nov 18, 2018 1:25 pm

I use one ASUS XG-C100C network card in each of two PCs that are connected to Mikrotik CSS326-24G-2S+RM (each via Mikrotik S+RJ10 SFP).

One PC is very close (2 meter Cat5E cable), the other is far (15 meter Cat5E cable). I get brilliant 10Gbit speed on both links. Everything works fine until I even slightly touch SFP+ module or RJ-45 connector. If not touched, links work brilliant at least for a month without problems (have not tested longer yet). If touched, link immediately goed down and then random interruptions start to occur. Next interruption can happen in several minutes or in several hours - unpredictable.

I use latest RouterOS and firmware. The only thing that seems to help - I try to insert connectors with SFP+ modules more strongly (firmly) into the SFP+ ports, but this seems strange and unstable, because even a slight touch can break connection again.

What am I doing wrong? Any suggestions?

Mikrotik CSS326-24G-2S+RM config:
 # nov/18/2018 14:20:22 by RouterOS 6.43.4
    # software id = SUYY-DNX3
    #
    # model = CRS326-24G-2S+
    # serial number = 763C08FD32BB
    /interface bridge
    add admin-mac=CC:2D:E0:D3:EC:44 auto-mac=no comment=defconf name=bridge
    /interface ethernet
    set [ find default-name=ether1 ] speed=100Mbps
    set [ find default-name=ether2 ] speed=100Mbps
    set [ find default-name=ether3 ] speed=100Mbps
    set [ find default-name=ether4 ] speed=100Mbps
    set [ find default-name=ether5 ] speed=100Mbps
    set [ find default-name=ether6 ] speed=100Mbps
    set [ find default-name=ether7 ] speed=100Mbps
    set [ find default-name=ether8 ] speed=100Mbps
    set [ find default-name=ether9 ] speed=100Mbps
    set [ find default-name=ether10 ] speed=100Mbps
    set [ find default-name=ether11 ] speed=100Mbps
    set [ find default-name=ether12 ] speed=100Mbps
    set [ find default-name=ether13 ] speed=100Mbps
    set [ find default-name=ether14 ] speed=100Mbps
    set [ find default-name=ether15 ] speed=100Mbps
    set [ find default-name=ether16 ] speed=100Mbps
    set [ find default-name=ether17 ] speed=100Mbps
    set [ find default-name=ether18 ] speed=100Mbps
    set [ find default-name=ether19 ] speed=100Mbps
    set [ find default-name=ether20 ] speed=100Mbps
    set [ find default-name=ether21 ] speed=100Mbps
    set [ find default-name=ether22 ] speed=100Mbps
    set [ find default-name=ether23 ] speed=100Mbps
    set [ find default-name=ether24 ] speed=100Mbps
    set [ find default-name=sfp-sfpplus1 ] advertise=10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full,2500M-full,5000M-full,10000M-full rx-flow-control=auto tx-flow-control=auto
    set [ find default-name=sfp-sfpplus2 ] speed=10Gbps
    /interface list
    add name=WAN
    add name=LAN
    /interface bridge port
    add bridge=bridge interface=ether1
    add bridge=bridge interface=ether2
    add bridge=bridge interface=ether3
    add bridge=bridge interface=ether4
    add bridge=bridge interface=ether5
    add bridge=bridge interface=ether6
    add bridge=bridge interface=ether7
    add bridge=bridge interface=ether8
    add bridge=bridge interface=ether9
    add bridge=bridge interface=ether10
    add bridge=bridge interface=ether11
    add bridge=bridge interface=ether12
    add bridge=bridge interface=ether13
    add bridge=bridge interface=ether14
    add bridge=bridge interface=ether15
    add bridge=bridge interface=ether16
    add bridge=bridge interface=ether17
    add bridge=bridge interface=ether18
    add bridge=bridge interface=ether19
    add bridge=bridge interface=ether20
    add bridge=bridge interface=ether21
    add bridge=bridge interface=ether22
    add bridge=bridge interface=ether23
    add bridge=bridge interface=ether24
    add bridge=bridge interface=sfp-sfpplus1
    add bridge=bridge interface=sfp-sfpplus2
    /interface list member
    add interface=ether1 list=WAN
    add interface=ether2 list=LAN
    add interface=ether3 list=LAN
    add interface=ether4 list=LAN
    add interface=ether5 list=LAN
    add interface=ether6 list=LAN
    add interface=ether7 list=LAN
    add interface=ether8 list=LAN
    add interface=ether9 list=LAN
    add interface=ether10 list=LAN
    add interface=ether11 list=LAN
    add interface=ether12 list=LAN
    add interface=ether13 list=LAN
    add interface=ether14 list=LAN
    add interface=ether15 list=LAN
    add interface=ether16 list=LAN
    add interface=ether17 list=LAN
    add interface=ether18 list=LAN
    add interface=ether19 list=LAN
    add interface=ether20 list=LAN
    add interface=ether21 list=LAN
    add interface=ether22 list=LAN
    add interface=ether23 list=LAN
    add interface=ether24 list=LAN
    add interface=sfp-sfpplus1 list=LAN
    add interface=sfp-sfpplus2 list=LAN
    /ip address
    add address=192.168.9.9/8 comment=defconf interface=ether2 network=192.0.0.0
    /ip cloud
    set ddns-enabled=yes
    /ip dns
    set servers=8.8.8.8
    /ip route
    add distance=1 gateway=192.168.9.1
    /system clock
    set time-zone-name=Europe/Moscow
    /system leds
    set 0 leds=user-led type=off
    add interface=ether1 leds=ether1-led type=interface-activity
    /system logging
    add action=remote disabled=yes topics=error,warning,critical,info
    /system routerboard settings
    set boot-os=router-os silent-boot=no
2018-11-20 23.31.55.jpg
2018-11-20 23.32.24.jpg
2018-11-20 23.33.01.jpg
You do not have the required permissions to view the files attached to this post.
Last edited by rualark on Tue Nov 20, 2018 10:42 pm, edited 1 time in total.
 
User avatar
sebastia
Forum Guru
Forum Guru
Posts: 1795
Joined: Tue Oct 12, 2010 3:23 am
Location: Antwerp, BE

Re: Unstable 10Gbit connection of Mikrotik CSS326-24G-2S+RM with Mikrotik S+RJ10 SFP

Sun Nov 18, 2018 2:20 pm

Maybe a shielding problem? Have you tried other cables? Also, what do the stats on these interfaces tell you: error packets and / or dropped packets?
 
patrick7
Member Candidate
Member Candidate
Posts: 298
Joined: Sat Jul 20, 2013 2:40 pm

Re: Unstable 10Gbit connection of Mikrotik CSS326-24G-2S+RM with Mikrotik S+RJ10 SFP

Sun Nov 18, 2018 3:57 pm

Why is your IP on ether2? It should be on the bridge. And /8 mask is... a bad idea.
 
mkx
Forum Guru
Forum Guru
Posts: 3223
Joined: Thu Mar 03, 2016 10:23 pm

Re: Unstable 10Gbit connection of Mikrotik CSS326-24G-2S+RM with Mikrotik S+RJ10 SFP

Sun Nov 18, 2018 4:49 pm

Why is your IP on ether2? It should be on the bridge. And /8 mask is... a bad idea.
This doesn't really relate with OP's problem.

But if we pursue non-relevant bits and pieces, then it's not only bad idea, it's plain wrong. 192.0.0.0/8 covers also regular routed subnets. Another thing, which is weird, is to have ether1 part of interface list WAN... all interfaces should be members of same interface list since they are members of same L2 network (via bridge membership).
BR,
Metod
 
patrick7
Member Candidate
Member Candidate
Posts: 298
Joined: Sat Jul 20, 2013 2:40 pm

Re: Unstable 10Gbit connection of Mikrotik CSS326-24G-2S+RM with Mikrotik S+RJ10 SFP

Sun Nov 18, 2018 7:53 pm

At least the IP on ether2 instead of bridge may be a problem :-)
 
User avatar
yngndrw
just joined
Posts: 17
Joined: Sun Oct 27, 2013 12:26 am

Re: Unstable 10Gbit connection of Mikrotik CSS326-24G-2S+RM with Mikrotik S+RJ10 SFP

Mon Nov 19, 2018 1:25 am

I use one ASUS XG-C100C network card in each of two PCs that are connected to Mikrotik CSS326-24G-2S+RM (each via Mikrotik S+RJ10 SFP).

One PC is very close (2 meter Cat5E cable), the other is far (15 meter Cat5E cable). I get brilliant 10Gbit speed on both links. Everything works fine until I even slightly touch SFP+ module or RJ-45 connector. If not touched, links work brilliant at least for a month without problems (have not tested longer yet). If touched, link immediately goed down and then random interruptions start to occur. Next interruption can happen in several minutes or in several hours - unpredictable.

I use latest RouterOS and firmware. The only thing that seems to help - I try to insert connectors with SFP+ modules more strongly (firmly) into the SFP+ ports, but this seems strange and unstable, because even a slight touch can break connection again.

What am I doing wrong? Any suggestions?
Apologies if I'm stating the obvious, but have the modules actually locked into the SFP ports ? They should click when you push them in and should require the gold lever to be opened in order to remove them. Also do your ethernet cables have rubber boots on could be preventing them from locking into the modules properly ?

Does the connection drop when you touch the middle of the long cable, or just near the SFP modules ?

I know that CAT5e is technically supported for 10GbE, but I'd personally switch to CAT6 or CAT6A cables. (Well personally I'd use SFP+ network cards with a direct attached copper cable for the short one and a fibre for the long one, but you have the 10GbE gear now.)
 
rualark
just joined
Topic Author
Posts: 10
Joined: Sun Nov 11, 2018 8:28 pm

Re: Unstable 10Gbit connection of Mikrotik CSS326-24G-2S+RM with Mikrotik S+RJ10 SFP

Tue Nov 20, 2018 9:24 am

Maybe a shielding problem? Have you tried other cables? Also, what do the stats on these interfaces tell you: error packets and / or dropped packets?
My cable is unshielded. Should I use shilded or not inside a flat?
 
rualark
just joined
Topic Author
Posts: 10
Joined: Sun Nov 11, 2018 8:28 pm

Re: Unstable 10Gbit connection of Mikrotik CSS326-24G-2S+RM with Mikrotik S+RJ10 SFP

Tue Nov 20, 2018 9:27 am

Why is your IP on ether2? It should be on the bridge. And /8 mask is... a bad idea.
This doesn't really relate with OP's problem.

But if we pursue non-relevant bits and pieces, then it's not only bad idea, it's plain wrong. 192.0.0.0/8 covers also regular routed subnets. Another thing, which is weird, is to have ether1 part of interface list WAN... all interfaces should be members of same interface list since they are members of same L2 network (via bridge membership).
Thanks a lot for auditing my config! I agree. I fixed all three issues. This did not fix the problem with the link:
# nov/20/2018 10:26:43 by RouterOS 6.43.4
# software id = SUYY-DNX3
#
# model = CRS326-24G-2S+
# serial number = 763C08FD32BB
/interface bridge
add admin-mac=CC:2D:E0:D3:EC:44 auto-mac=no comment=defconf name=bridge
/interface ethernet
set [ find default-name=ether1 ] speed=100Mbps
set [ find default-name=ether2 ] speed=100Mbps
set [ find default-name=ether3 ] speed=100Mbps
set [ find default-name=ether4 ] speed=100Mbps
set [ find default-name=ether5 ] speed=100Mbps
set [ find default-name=ether6 ] speed=100Mbps
set [ find default-name=ether7 ] speed=100Mbps
set [ find default-name=ether8 ] speed=100Mbps
set [ find default-name=ether9 ] speed=100Mbps
set [ find default-name=ether10 ] speed=100Mbps
set [ find default-name=ether11 ] speed=100Mbps
set [ find default-name=ether12 ] speed=100Mbps
set [ find default-name=ether13 ] speed=100Mbps
set [ find default-name=ether14 ] speed=100Mbps
set [ find default-name=ether15 ] speed=100Mbps
set [ find default-name=ether16 ] speed=100Mbps
set [ find default-name=ether17 ] speed=100Mbps
set [ find default-name=ether18 ] speed=100Mbps
set [ find default-name=ether19 ] speed=100Mbps
set [ find default-name=ether20 ] speed=100Mbps
set [ find default-name=ether21 ] speed=100Mbps
set [ find default-name=ether22 ] speed=100Mbps
set [ find default-name=ether23 ] speed=100Mbps
set [ find default-name=ether24 ] speed=100Mbps
set [ find default-name=sfp-sfpplus1 ] advertise=10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full,2500M-full,5000M-full,10000M-full rx-flow-control=auto tx-flow-control=auto
set [ find default-name=sfp-sfpplus2 ] speed=10Gbps
/interface list
add name=WAN
add name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile
set [ find default=yes ] html-directory=flash/hotspot
/snmp community
set [ find default=yes ] name=yifan
/system logging action
set 3 remote=192.168.9.7
/interface bridge port
add bridge=bridge interface=ether1
add bridge=bridge interface=ether2
add bridge=bridge interface=ether3
add bridge=bridge interface=ether4
add bridge=bridge interface=ether5
add bridge=bridge interface=ether6
add bridge=bridge interface=ether7
add bridge=bridge interface=ether8
add bridge=bridge interface=ether9
add bridge=bridge interface=ether10
add bridge=bridge interface=ether11
add bridge=bridge interface=ether12
add bridge=bridge interface=ether13
add bridge=bridge interface=ether14
add bridge=bridge interface=ether15
add bridge=bridge interface=ether16
add bridge=bridge interface=ether17
add bridge=bridge interface=ether18
add bridge=bridge interface=ether19
add bridge=bridge interface=ether20
add bridge=bridge interface=ether21
add bridge=bridge interface=ether22
add bridge=bridge interface=ether23
add bridge=bridge interface=ether24
add bridge=bridge interface=sfp-sfpplus1
add bridge=bridge interface=sfp-sfpplus2
/interface list member
add interface=ether1 list=LAN
add interface=ether2 list=LAN
add interface=ether3 list=LAN
add interface=ether4 list=LAN
add interface=ether5 list=LAN
add interface=ether6 list=LAN
add interface=ether7 list=LAN
add interface=ether8 list=LAN
add interface=ether9 list=LAN
add interface=ether10 list=LAN
add interface=ether11 list=LAN
add interface=ether12 list=LAN
add interface=ether13 list=LAN
add interface=ether14 list=LAN
add interface=ether15 list=LAN
add interface=ether16 list=LAN
add interface=ether17 list=LAN
add interface=ether18 list=LAN
add interface=ether19 list=LAN
add interface=ether20 list=LAN
add interface=ether21 list=LAN
add interface=ether22 list=LAN
add interface=ether23 list=LAN
add interface=ether24 list=LAN
add interface=sfp-sfpplus1 list=LAN
add interface=sfp-sfpplus2 list=LAN
/ip address
add address=192.168.9.9/24 comment=defconf interface=bridge network=192.168.9.0
/ip cloud
set ddns-enabled=yes
/ip dns
set servers=8.8.8.8
/ip route
add distance=1 gateway=192.168.9.1
/snmp
set contact=MikroTik enabled=yes location=Alchemistry
/system clock
set time-zone-name=Europe/Moscow
/system leds
set 0 leds=user-led type=off
add interface=ether1 leds=ether1-led type=interface-activity
/system logging
add action=remote disabled=yes topics=error,warning,critical,info
/system routerboard settings
set boot-os=router-os silent-boot=no
By the way, what is the most convenient way to get full router config into the clipboard?
 
rualark
just joined
Topic Author
Posts: 10
Joined: Sun Nov 11, 2018 8:28 pm

Re: Unstable 10Gbit connection of Mikrotik CSS326-24G-2S+RM with Mikrotik S+RJ10 SFP

Tue Nov 20, 2018 10:40 pm

Apologies if I'm stating the obvious, but have the modules actually locked into the SFP ports ? They should click when you push them in and should require the gold lever to be opened in order to remove them. Also do your ethernet cables have rubber boots on could be preventing them from locking into the modules properly ?

Does the connection drop when you touch the middle of the long cable, or just near the SFP modules ?

I know that CAT5e is technically supported for 10GbE, but I'd personally switch to CAT6 or CAT6A cables. (Well personally I'd use SFP+ network cards with a direct attached copper cable for the short one and a fibre for the long one, but you have the 10GbE gear now.)
I cannot say that they click, but there is something close to click when I insert them. Gold lever is required to be opened in order to remove them. There are no rubber boots.

Connection drops when I touch near SFP modules.

I added photos to the post.

I will check with Cat6 later.
 
Paternot
Long time Member
Long time Member
Posts: 607
Joined: Thu Jun 02, 2016 4:01 am
Location: Niterói / Brazil

Re: Unstable 10Gbit connection of Mikrotik CSS326-24G-2S+RM with Mikrotik S+RJ10 SFP  [SOLVED]

Wed Nov 21, 2018 4:29 am

You should use Cat6. Take a look at the SFP+ module brochure:

https://i.mt.lv/cdn/rb_files/sfp_splusr ... 150322.pdf

It clearly says that Cat6 (or Cat7) is needed to achieve 10Gb - and even then, only up to 30 meters.
 
User avatar
yngndrw
just joined
Posts: 17
Joined: Sun Oct 27, 2013 12:26 am

Re: Unstable 10Gbit connection of Mikrotik CSS326-24G-2S+RM with Mikrotik S+RJ10 SFP

Wed Nov 21, 2018 11:47 am

According to that, it should be fine at 5Gb - It might also be worth setting that link speed manually with CAT5e.
 
rualark
just joined
Topic Author
Posts: 10
Joined: Sun Nov 11, 2018 8:28 pm

Re: Unstable 10Gbit connection of Mikrotik CSS326-24G-2S+RM with Mikrotik S+RJ10 SFP

Sat Nov 24, 2018 10:57 am

According to that, it should be fine at 5Gb - It might also be worth setting that link speed manually with CAT5e.
Mikrotik S+RJ10 does not support manual setting of link speed - please see the manual: https://wiki.mikrotik.com/wiki/MikroTik ... e#S.2BRJ10
 
rualark
just joined
Topic Author
Posts: 10
Joined: Sun Nov 11, 2018 8:28 pm

Re: Unstable 10Gbit connection of Mikrotik CSS326-24G-2S+RM with Mikrotik S+RJ10 SFP

Sat Nov 24, 2018 10:58 am

You should use Cat6. Take a look at the SFP+ module brochure:

https://i.mt.lv/cdn/rb_files/sfp_splusr ... 150322.pdf

It clearly says that Cat6 (or Cat7) is needed to achieve 10Gb - and even then, only up to 30 meters.
It seems that the issue was fixed by replacing cable with Category 6A patch cord. I will do more testing later.
You can see replaced cable on the front. I will replace the second cable later:
You do not have the required permissions to view the files attached to this post.

Who is online

Users browsing this forum: No registered users and 71 guests