Community discussions

MikroTik App
 
User avatar
lapsio
Long time Member
Long time Member
Topic Author
Posts: 514
Joined: Wed Feb 24, 2016 5:19 pm

CRS305 10G -> 1G over 30% packet loss

Wed Dec 29, 2021 2:23 am

I'm having issues with CRS305 switch connected to CRS317. My full traffic chain goes like this:

laptop ---1G-->
CRS305 ---10G-->
CRS317 ---10G-->
CRS317 ---10G-->
CRS1009

When I'm doing small pings everything is fine but when I do ping -i .2 -s 1300 or higher packet loss starts to get out of hand like 20-30%. It results in terribly slow loading for some websites, especially LAN services. When I skip CRS305 and connect directly to CRS317 OR use "auto-negotiation=no speed=1Gbps" between CRS305 and CRS317 then everything is fine (also when I skip CRS305 completely).

So it seems that CRS305 has some issues with handling 1G / 10G forwarding that doesn't seem to be an issue for CRS317 nor CRS326 (used in other part of network). Is it known problem for this small device? Are there known solutions? I'm using this switch in quite straightforward fashion:
/interface bridge
add name=br-hardware protocol-mode=none vlan-filtering=yes
/interface ethernet
set [ find default-name=ether1 ] name=ether1-rescue
set [ find default-name=sfp-sfpplus1 ] auto-negotiation=no l2mtu=9112 mtu=9000
set [ find default-name=sfp-sfpplus2 ] auto-negotiation=no l2mtu=9112 mtu=9000
set [ find default-name=sfp-sfpplus3 ] auto-negotiation=no l2mtu=9112 mtu=9000
set [ find default-name=sfp-sfpplus4 ] l2mtu=9112 mtu=9000
/interface vlan
add interface=br-hardware name=vlan10-ccr vlan-id=1030
/interface bridge port
add bridge=br-hardware frame-types=admit-only-untagged-and-priority-tagged ingress-filtering=yes interface=sfp-sfpplus1 pvid=410
add bridge=br-hardware frame-types=admit-only-untagged-and-priority-tagged ingress-filtering=yes interface=sfp-sfpplus2 pvid=411
add bridge=br-hardware frame-types=admit-only-untagged-and-priority-tagged ingress-filtering=yes interface=sfp-sfpplus3 pvid=412
add bridge=br-hardware frame-types=admit-only-vlan-tagged ingress-filtering=yes interface=sfp-sfpplus4 pvid=5
/interface bridge vlan
add bridge=br-hardware tagged=br-hardware,sfp-sfpplus4 vlan-ids=1030
add bridge=br-hardware tagged=sfp-sfpplus4 untagged=sfp-sfpplus1 vlan-ids=410
add bridge=br-hardware tagged=sfp-sfpplus4 untagged=sfp-sfpplus2 vlan-ids=411
add bridge=br-hardware tagged=sfp-sfpplus4 untagged=sfp-sfpplus3 vlan-ids=412
that's all. I don't use queues or anything like that on any of those CRS devices and bigger ones seem to work just fine.

MTU on interface doesn't seem to affect problem, I'm getting the same packet loss no matter what MTU I set on laptop (for 1300 packets). However of course the higher packet size is, the bigger loss. With 8k packets (when MTU is set to 9k) I'm getting like 50% packet loss.
 
User avatar
lapsio
Long time Member
Long time Member
Topic Author
Posts: 514
Joined: Wed Feb 24, 2016 5:19 pm

Re: CRS305 10G -> 1G over 30% packet loss

Wed Dec 29, 2021 3:00 am

I tried enabling flow-control but no use.
 
tangent
Forum Guru
Forum Guru
Posts: 1333
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: CRS305 10G -> 1G over 30% packet loss

Wed Dec 29, 2021 3:57 am

when I do ping -i .2 -s 1300 or higher packet loss starts to get out of hand

5 Ethernet frames per second is nothing. If your network can't sustain that, you have serious problems.

l2mtu=9112 mtu=9000

Jumbo frames aren't often all that effective, even when they're fully supported end-to-end. Unless you're dealing with massive flows on the LAN between compatible devices, I'd return that to standard Ethernet MTUs.

I tried enabling flow-control but no use.

Ethernet flow control is almost never the proper solution. It is at best a band-aid on a real problem, and you're better off finding and fixing that.

I can't tell what that "real problem" is from what you've provided. It feels like incompatible 10G SFP+ modules, but that's just a guess.
 
User avatar
lapsio
Long time Member
Long time Member
Topic Author
Posts: 514
Joined: Wed Feb 24, 2016 5:19 pm

Re: CRS305 10G -> 1G over 30% packet loss

Wed Dec 29, 2021 5:02 am

I can't tell what that "real problem" is from what you've provided. It feels like incompatible 10G SFP+ modules, but that's just a guess.
I'm using jumbo frames mostly to allow for 10G with CCR1009 without fasttrack. It's mostly useful for backups. I have not noticed any side effects. It's end-to-end 9k jumbo terminated on CCR1009 in this case. However like I said switching to 1500 doesn't solve the issue.

I'm using SFP-10GSR-85 from FS.COM and it may indeed be an issue with sfp modules. I have two kinds of FS.COM modules "F5" and generic ones and it seems that issue has been significantly reduced when I put "F5" ones between CRS305 and CRS317. Weird. Okay I guess I'll try to order more of those "F5" compatible ones. Too bad they don't seem to be making them anymore in form I used to get before. New ones are not detected by F5 hardware as genuine anymore so idk if they're gonna work... meh.

Though I'm still getting ~10% packet loss. It's just not 50% like before and LAN services seem to be pretty usable now. I would prefer to have less packet loss tho. I mean realistically around 10% in LAN is terrible especially considering that all devices in fact are in the same room lol...
 
tangent
Forum Guru
Forum Guru
Posts: 1333
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: CRS305 10G -> 1G over 30% packet loss

Thu Dec 30, 2021 12:35 am

It feels like incompatible 10G SFP+ modules, but that's just a guess.

Another possibility has occurred to me: incorrect optical fiber type. Fiber that works fine at 1G might well fail to transmit data reliably at 10G over the distance you're using. OM1 drops from 275m for 1000BASE-SX to 33m for 10GBASE-SR, for example. Depending on the SFP+ module specs, you might not even get that much distance. Cable damage from being stepped on or pulled tighter than its radius limits might give even less distance without failing outright.

Now, if you tell me you're using brand-new OM4 for everything, then I say, "Never mind." :)
 
User avatar
lapsio
Long time Member
Long time Member
Topic Author
Posts: 514
Joined: Wed Feb 24, 2016 5:19 pm

Re: CRS305 10G -> 1G over 30% packet loss

Thu Dec 30, 2021 5:39 am

Now, if you tell me you're using brand-new OM4 for everything, then I say, "Never mind." :)
Well I actually am using brand new OM4 3m uniboot patchcables for everything :D
That said I checked different patchcord with the same results (I suspected that while it sure isn't damaged, maybe somewhere it bends too much for this fibre but unfortunately it wasn't the case. With shortest path connection between devices it behaves the same way.

Who is online

Users browsing this forum: Fogga and 16 guests