Community discussions

MikroTik App
  4. Other topics
  5. SwOS

CRS309-1G-8S+IN Doesn't negotiate 10Gbps, only 5.

Post Reply
 
gogofc
just joined
Topic Author
Posts: 2
Joined: Mon Sep 18, 2023 1:08 am

CRS309-1G-8S+IN Doesn't negotiate 10Gbps, only 5.

Thu Nov 23, 2023 3:00 pm

How do these CRS*s play with other vendor equipment in your experience?

Here's the setup we have.

Cat8 Cables.

SwitchOS or RouterOS in bridge mode give the same results.

CRS309-1G-8S+IN with S+RJ10 Revision 2

1. Supermicro. Ubuntu. Marvell AQtion AQN-107 10 GbE NIC.
2. Desktop. Windows 10. Two 10Gbps NICs. One Gigabyte and one Marvel.
3. Laptop. Windows 11. Realtek 2.5Gbps NIC.

RouterOS reports 10G on all ports, that's false.
Windows machines do report 10G and 2.5G on NICs.
Ubuntu reports 5Gbps NIC saying partner advertised only 5G.

iperf3 gives results that are all over the place.

Supermicro --> Desktop == 4.5Gbps. Desktop --> Supermicro == 93Mbsp. (When using ISP router its stable at 1Gbps upload and download).
Supermicro --> Laptop == 1.7Gbps. Laptop --> Supermicro == 2.3Gbps.
Laptop --> Desktop == 2.3Gbps. Desktop --> Laptop 500Mbps.

Editing MTU sizes does nothing. Well it does make it worse in some cases.

All drivers updated from Microsoft to Realtek and Marvel.
Driver on Ubuntu I didn't check what that is, it's Ubuntu 22.04 with kernel 5.15.

Did not try to turn off autonegotiation on Ubuntu NIC yet.

Sometimes the Link to desktop is paused as seen from SwOS.

What I am doing wrong or is this Mikrotik broken? Or is the all other 4 NICs broken but Mikrotik is fine?

Ethtool on Ubuntu/TrueNAS.
Code: Select all
user@ubuntu:~$ sudo ethtool enp132s0
Settings for enp132s0:
	Supported ports: [ TP ]
	Supported link modes:   10baseT/Full
	                        100baseT/Full
	                        1000baseT/Full
	                        10000baseT/Full
	                        2500baseT/Full
	                        5000baseT/Full
	Supported pause frame use: Symmetric Receive-only
	Supports auto-negotiation: Yes
	Supported FEC modes: Not reported
	Advertised link modes:  10baseT/Full
	                        100baseT/Full
	                        1000baseT/Full
	                        10000baseT/Full
	                        2500baseT/Full
	                        5000baseT/Full
	Advertised pause frame use: No
	Advertised auto-negotiation: Yes
	Advertised FEC modes: Not reported
	Link partner advertised link modes:  10baseT/Half 10baseT/Full
	                                     100baseT/Half 100baseT/Full
	                                     1000baseT/Half 1000baseT/Full
	                                     2500baseT/Full
	                                     5000baseT/Full
	Link partner advertised pause frame use: No
	Link partner advertised auto-negotiation: No
	Link partner advertised FEC modes: Not reported
	Speed: 5000Mb/s
	Duplex: Full
	Auto-negotiation: on
	Port: Twisted Pair
	PHYAD: 0
	Transceiver: internal
	MDI-X: Unknown
	Supports Wake-on: pg
	Wake-on: d
        Current message level: 0x00000005 (5)
                               drv link
	Link detected: yes


iperf3 client running on Laptop connected to Desktop.
Download 478Mbps
Upload 2.09Gbps
Code: Select all
iperf3.exe -c 192.168.1.75 -R
Connecting to host 192.168.1.75, port 5201
Reverse mode, remote host 192.168.1.75 is sending
[  4] local 192.168.1.23 port 55064 connected to 192.168.1.75 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  42.2 MBytes   354 Mbits/sec
[  4]   1.00-2.00   sec  69.8 MBytes   583 Mbits/sec
[  4]   2.00-3.01   sec  64.0 MBytes   536 Mbits/sec
[  4]   3.01-4.01   sec  54.0 MBytes   452 Mbits/sec
[  4]   4.01-5.00   sec  64.1 MBytes   540 Mbits/sec
[  4]   5.00-6.01   sec  60.1 MBytes   501 Mbits/sec
[  4]   6.01-7.01   sec  61.6 MBytes   517 Mbits/sec
[  4]   7.01-8.00   sec  63.4 MBytes   538 Mbits/sec
[  4]   8.00-9.01   sec  49.5 MBytes   411 Mbits/sec
[  4]   9.01-10.00  sec  41.4 MBytes   351 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec   570 MBytes   478 Mbits/sec                  sender
[  4]   0.00-10.00  sec   570 MBytes   478 Mbits/sec                  receiver

iperf Done.

C:\Users\m\Documents\Supermicro-stuff\iperf-3.1.3-win64>iperf3.exe -c 192.168.1.75
Connecting to host 192.168.1.75, port 5201
[  4] local 192.168.1.23 port 55106 connected to 192.168.1.75 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   246 MBytes  2.06 Gbits/sec
[  4]   1.00-2.00   sec   247 MBytes  2.07 Gbits/sec
[  4]   2.00-3.00   sec   252 MBytes  2.12 Gbits/sec
[  4]   3.00-4.00   sec   253 MBytes  2.12 Gbits/sec
[  4]   4.00-5.00   sec   249 MBytes  2.09 Gbits/sec
[  4]   5.00-6.00   sec   251 MBytes  2.10 Gbits/sec
[  4]   6.00-7.00   sec   250 MBytes  2.10 Gbits/sec
[  4]   7.00-8.00   sec   250 MBytes  2.09 Gbits/sec
[  4]   8.00-9.00   sec   249 MBytes  2.09 Gbits/sec
[  4]   9.00-10.00  sec   250 MBytes  2.10 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  2.44 GBytes  2.09 Gbits/sec                  sender
[  4]   0.00-10.00  sec  2.44 GBytes  2.09 Gbits/sec                  receiver

iperf Done.


iperf client on Desktop goint to Supermicro
Upload 88Mbps
Download 4.74Gbps.
Code: Select all
C:\Users\M\Downloads\iperf-3.1.3-win64>iperf3.exe -c 192.168.1.110
Connecting to host 192.168.1.110, port 5201
[  4] local 192.168.1.75 port 65401 connected to 192.168.1.110 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.01   sec  11.1 MBytes  92.3 Mbits/sec
[  4]   1.01-2.00   sec  10.4 MBytes  87.9 Mbits/sec
[  4]   2.00-3.00   sec  8.75 MBytes  73.2 Mbits/sec
[  4]   3.00-4.00   sec  11.1 MBytes  93.5 Mbits/sec
[  4]   4.00-5.00   sec  11.4 MBytes  95.2 Mbits/sec
[  4]   5.00-6.01   sec  11.5 MBytes  96.3 Mbits/sec
[  4]   6.01-7.01   sec  8.12 MBytes  68.2 Mbits/sec
[  4]   7.01-8.00   sec  11.1 MBytes  93.6 Mbits/sec
[  4]   8.00-9.01   sec  11.1 MBytes  92.3 Mbits/sec
[  4]   9.01-10.01  sec  11.0 MBytes  93.1 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.01  sec   106 MBytes  88.6 Mbits/sec                  sender
[  4]   0.00-10.01  sec   105 MBytes  88.4 Mbits/sec                  receiver

iperf Done.

C:\Users\M\Downloads\iperf-3.1.3-win64>iperf3.exe -c 192.168.1.110 -R
Connecting to host 192.168.1.110, port 5201
Reverse mode, remote host 192.168.1.110 is sending
[  4] local 192.168.1.75 port 65414 connected to 192.168.1.110 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   563 MBytes  4.73 Gbits/sec
[  4]   1.00-2.00   sec   565 MBytes  4.74 Gbits/sec
[  4]   2.00-3.00   sec   565 MBytes  4.74 Gbits/sec
[  4]   3.00-4.00   sec   566 MBytes  4.75 Gbits/sec
[  4]   4.00-5.00   sec   566 MBytes  4.74 Gbits/sec
[  4]   5.00-6.00   sec   564 MBytes  4.74 Gbits/sec
[  4]   6.00-7.00   sec   565 MBytes  4.74 Gbits/sec
[  4]   7.00-8.00   sec   566 MBytes  4.74 Gbits/sec
[  4]   8.00-9.00   sec   566 MBytes  4.75 Gbits/sec
[  4]   9.00-10.00  sec   566 MBytes  4.75 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  5.52 GBytes  4.74 Gbits/sec    0             sender
[  4]   0.00-10.00  sec  5.52 GBytes  4.74 Gbits/sec                  receiver

iperf Done.

Link paused
linkpaused.png
You do not have the required permissions to view the files attached to this post.
Top
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11646
Joined: Thu Mar 03, 2016 10:23 pm

Re: CRS309-1G-8S+IN Doesn't negotiate 10Gbps, only 5.

Thu Nov 23, 2023 10:58 pm

What are exact cable lengths used? Operation at 10Gbps requires significant amount of energy and nkt many RJ45 SFP+ modules are capable of transmitting at needed power. And cable category doesn't affect this much.

Which SFP+ modules are you using? Support for different SFP modules in MT devices is far from ideal, some modules work much better than others.

Differences in link speeds on different legs of computer-to-computer connection are known to cause problems (buffer overflows, causing significant drops in end2end speed), MT seems to be more affected than average vendors. Might be that you'd see better overall performance if all links would be running at same (lower) link rate (e.g. 2.5Gbps)
Top
 
gogofc
just joined
Topic Author
Posts: 2
Joined: Mon Sep 18, 2023 1:08 am

Re: CRS309-1G-8S+IN Doesn't negotiate 10Gbps, only 5.

Thu Nov 23, 2023 11:03 pm

Mikrotik ones S+RJ10. Docs say they can use up to 30 Meters of cable which is about 100 feet, and I don't think it's longer than that. Probably half that size.

I don't know what else to think, but we will check the cables next and make sure that's not it.
Screenshot from 2023-11-23 11-31-11.png
You do not have the required permissions to view the files attached to this post.
Last edited by gogofc on Thu Nov 23, 2023 11:05 pm, edited 1 time in total.
Top
 
User avatar
mkx
Forum Guru
Forum Guru
Posts: 11646
Joined: Thu Mar 03, 2016 10:23 pm

Re: CRS309-1G-8S+IN Doesn't negotiate 10Gbps, only 5.

Fri Nov 24, 2023 12:05 am

Mikrotik ones S+RJ10. Docs say they can use up to 30 Meters of cable ...

Another caveat: due to required high Tx power, these modules tend to run quite hot (MT's own seems to be one of hottest) ... if cooling is not adequate (with most passively cooled devices, such as CRS309-1G-8S+IN, this is the case), then modules may overheat and throttle down Tx power (and consequently data rate). Mikrotik has a guidance on that. If I'm decoding screenshots of your config correctly, you're already following the guidance ... but that might not be enough.
Top
 
gogofc
just joined
Topic Author
Posts: 2
Joined: Mon Sep 18, 2023 1:08 am

Re: CRS309-1G-8S+IN Doesn't negotiate 10Gbps, only 5.

Fri Nov 24, 2023 9:40 am

Yeah they're kind of spaced out, but also not really being used since I can't get it to work :)
Top
Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests
  4. Other topics
  5. SwOS