Community discussions

MikroTik App
 
forteller
just joined
Topic Author
Posts: 14
Joined: Tue Jun 13, 2023 9:58 am

RB5009 2,5Gbe problems

Tue Aug 29, 2023 10:17 am

Hello,
I've read a lot of threads here and on reddit regarding RB5009 2,5Gbe port related issues, but most of them got resolved around ROS7.9 and I haven't found anyone with setup similar to mine.
I have a dumb 2,5Gbe switch connected to RB5009's 2,5Gbe port. Apart from RB5009, NAS and PC are connected to this switch which allows me to transfer files between PC and NAS quickly as well as to uzilize 2Gbe downlink from my ISP which is provided using SFP module. That portion of the network works flawlessly and I can utilize full internet speed on both NAS and PC as well as transfer files at >1Gbe speeds between them (HDDs in NAS are limiting factor).

Problem starts with two Xiaomis AX3600 which are flashed with openWRT and wired directly to 1Gbe ports of my RB5009. And this one is really strange. If I connect anything to AX3600 using ethernet cable - no problems there. I can see 1gig troughput from NAS and internet downlink. But as soon as I switch to WiFi that no longer is the case - I still can do speedtest @ 1Gig speed (AX3600's are capable of 1,3-1,7Gbe wireless while copying from two sources to overcome 1Gbe link limit), but transfer from NAS is limited to ~400Mbps. As soon as I remove 2,5Gbe advertisement on RB5009's eth1 port, transfer speed goes up to 1Gbe, which can be observed on this screen recording:
https://youtu.be/5qoR7-OQsM0

The only clue I got is that 400Mbps is the value I get while running iperf3 between my wirelessly connected laptop and NAS with 1 thread. Increasing thread count with 1Gbe enforced increases troughput to 1Gbe gradually, but when 2,5Gbe is enabled it does close to nothing when it comes to transfer speed. Do note that I have SMB Multichannel enabled on my NAS which also uses multiple threads to copy files. It's like having 2,5Gbe enabled on eth1 somehow breaks the ability to use multithreaded transfers and that would explain why only wifi is affected as using the cable I can saturate 1Gbe / 2,5Gbe links using just one thread.

To me it looks like I discovered some bug in ROS, but I cannot figure out how to troubleshoot it further. I used two 2,5Gbe switches based on different platforms (Realtek and Broadcom): TP-Link TL-SH1005 and QNAP QSW-1105T-5T. I have RB5009UPR+S+IN running latest ROS stable - 7.11.2. I also attached simple diagram to help visualize my network setup. If some configuration files are needed, I will happily provide them.
You do not have the required permissions to view the files attached to this post.
 
forteller
just joined
Topic Author
Posts: 14
Joined: Tue Jun 13, 2023 9:58 am

Re: RB5009 2,5Gbe problems

Wed Sep 13, 2023 9:38 pm

Bump, anyone?
 
holvoetn
Forum Guru
Forum Guru
Posts: 5072
Joined: Tue Apr 13, 2021 2:14 am
Location: Belgium

Re: RB5009 2,5Gbe problems

Wed Sep 13, 2023 9:42 pm

Maybe because nobody has a clue?

Already contacted support with accompanying supout.rif file from your router ?
 
hooyao
newbie
Posts: 37
Joined: Mon Feb 20, 2017 6:11 pm

Re: RB5009 2,5Gbe problems

Mon Sep 18, 2023 6:05 am

I have the similar issues observed on Unifi U6 Pros, which has 1Gb eth. My NAS is wired to CRS326 via 10G DAC, and the Unifi APs are connected to CRS326 via CAT6 cables. It can get 1Gbps from any 1Gb eth device in the LAN, but I can only get 400Mbps download speed from my 10G NAS to any wireless devices via Unifi U6 Pro, but directly running iperf on Unifi U6 Pro via ssh, everything was fine. I guess there is something wrong with the Qualcomm IPQ5018 wlan.
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11280
Joined: Thu Mar 03, 2016 10:23 pm

Re: RB5009 2,5Gbe problems

Mon Sep 18, 2023 8:29 am

There's always a problem when passing between zones with different speeds ... in particular when traffic is passing from faster towards slower speed. The device which spans the two zones has to buffer data and different devices do it with various success. MT seems to be among the worst and it seems that device type is not the biggest factor. Working implementation of flow control should help a lot, but MT default seems to be "off" (check /interface/ethernet, settings rx-flow-control and tx-flow-control are per port), but it should be enabled on both link partners and in particular on the faster side.
 
hooyao
newbie
Posts: 37
Joined: Mon Feb 20, 2017 6:11 pm

Re: RB5009 2,5Gbe problems

Mon Sep 18, 2023 3:41 pm

There's always a problem when passing between zones with different speeds ... in particular when traffic is passing from faster towards slower speed. The device which spans the two zones has to buffer data and different devices do it with various success. MT seems to be among the worst and it seems that device type is not the biggest factor. Working implementation of flow control should help a lot, but MT default seems to be "off" (check /interface/ethernet, settings rx-flow-control and tx-flow-control are per port), but it should be enabled on both link partners and in particular on the faster side.
Thank you, enabling flow-control on the interface does help.

Just a kind reminder, you need to reboot the switch before it takes effort.
 
User avatar
jbl42
Member Candidate
Member Candidate
Posts: 214
Joined: Sun Jun 21, 2020 12:58 pm

Re: RB5009 2,5Gbe problems

Mon Sep 18, 2023 7:10 pm

Buffering towards slower interfaces only helps for short bursts. For constant streams like iperf or large file transfers, even large buffers only can help for a short period. If there is more data arriving than possible to output on the egress port, buffers will overflow.
Only flow control can help, as it allows to build back pressure towards the sending device which will stop sending data until there is space in the buffer again.

The question is why MT disables flow control by default, especially on devices like RB5009 having 1, 2.5 and 10G ports on the same switch. It does not hurt if activated while not required.But it hurts the other way round like you have experienced.
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11280
Joined: Thu Mar 03, 2016 10:23 pm

Re: RB5009 2,5Gbe problems

Mon Sep 18, 2023 11:05 pm

The question is why MT disables flow control by default...

I can't talk for Mikrotik ... but I seem to remember forum threads where users were complaining about subtle problems which went away after disabling flow control. Draw your own conclusions ...
 
forteller
just joined
Topic Author
Posts: 14
Joined: Tue Jun 13, 2023 9:58 am

Re: RB5009 2,5Gbe problems

Tue Sep 19, 2023 8:50 pm

Hello,
Thank you for your replies. I issued the support ticket few days after posting here but after first response, which didn't helped, support went silent. Hence my bump here. Unfortunately, enabling flow control for every port of my RB5009 only made it worse - now switching to 1Gig mode doesn't help to bring the transfer speed back. I think I will have to live with 1Gig downlink until support for rtl8156 is added and maybe issue will go away when I will be using usb nic connected directly to RB5009.
 
R1CH
Forum Guru
Forum Guru
Posts: 1097
Joined: Sun Oct 01, 2006 11:44 pm

Re: RB5009 2,5Gbe problems

Wed Sep 20, 2023 6:40 pm

Flow control for backpressure is not recommended these days, it's better to let packets drop so that the upper level protocols know to slow down. It's especially bad when you have an uplink port that's transmitting to an end device - backpressure from the end device will cause flow control to be sent to the uplink port, pausing transmissions to every other device!
 
User avatar
jbl42
Member Candidate
Member Candidate
Posts: 214
Joined: Sun Jun 21, 2020 12:58 pm

Re: RB5009 2,5Gbe problems

Wed Sep 20, 2023 8:37 pm

Maybe I'm missing something, but flow control does not change the fact that packets are dropped. It just changes where they are dropped. Without flow control packets are dropped at the receiving end because the RX buffer overruns. With flow control in case of backpressure packets are dropped at the sending device because the TX buffer overruns.
A local TX buffer overrun is easier to detect for the sending device compared to lost packets dropped at the receiving end.

A TCP stack can reduce the window size in case of TX buffer overrun of the underlying Ethernet adapter without having to wait for detection of missing ACKs due to packets being dropped at the receiving end.
 
forteller
just joined
Topic Author
Posts: 14
Joined: Tue Jun 13, 2023 9:58 am

Re: RB5009 2,5Gbe problems

Mon Sep 25, 2023 3:05 pm

I received a response from support that they have been able to reproduce this issue and passed it to engineering team without ETA unfortunately.

So this is a bug, not misconfiguration after all.
 
forteller
just joined
Topic Author
Posts: 14
Joined: Tue Jun 13, 2023 9:58 am

Re: RB5009 2,5Gbe problems

Wed Nov 29, 2023 11:53 pm

Problem still occurs on v7.12.
 
forteller
just joined
Topic Author
Posts: 14
Joined: Tue Jun 13, 2023 9:58 am

Re: RB5009 2,5Gbe problems

Sat Jan 20, 2024 12:07 pm

Still no solution to this problem.
 
ToTheCLI
Frequent Visitor
Frequent Visitor
Posts: 82
Joined: Mon Jan 04, 2016 3:54 am

Re: RB5009 2,5Gbe problems

Mon Jan 29, 2024 5:56 pm

I find it strange that this has not been fixed yet, is it a hardware issue?
 
forteller
just joined
Topic Author
Posts: 14
Joined: Tue Jun 13, 2023 9:58 am

Re: RB5009 2,5Gbe problems

Fri Feb 02, 2024 7:15 am

I confirmed that issue occurs even without the switch. If I have NAS device connected directly to RB5009, download speeds on laptop get halved as soon as I enable 2,5Gbit link. But the problem is visible only when moving data in one direction - when I upload files from laptop to NAS, I get full gigabit speeds.

Lack of any response from Mikrotik support is disappointing. I reported this issue half a year ago, they said that they reproduced it, but became silent since. If that's the hardware issue, then just admit it. Otherwise it looks like devices are being sold without any customer service. I received better support for some cheap Chinese products than the RB5009...

Who is online

Users browsing this forum: galp, GoogleOther [Bot] and 15 guests