Fiber Mall 10G SFP+ to RJ45 for Cisco SFP-10G-T-80 Compatible, 10GBASE-T SFP+ Copper RJ-45 80m Transceiver Module
Can find on Amazon. They are the 80m capable Broadcom chips. No drop outs running up 3 floors on Cat5e with zero issues at full 10Gbps.
I’ll test a bit more on the 1Gbps ports. I’ve only done that with a cascaded switch down to 1Gbps.
Oh do you have flow control on? That’s huge for me on mixed speed networks. Without it I see a massive reduction in speeds.
Thanks, I found it.
Flow control is off by default on all the 5009 ports. On 7.2cr4 “on” or “off” makes no difference, so I left it “as is” in all the tests. Probably will need to retest.
Yes, I run this command after each reboot - “/interface ethernet switch set switch1 cpu-flow-control=yes” - but Rx and Tx flow control for each port of 5009 gives no difference in speed.
I have 48 ports 1G switch connected with 4ports LACP to 5009 (1’ shielded cat 6a cables), 2 servers with 2 1G LACP ports, All LAN communications are as expected, but only internal link in 5009 between ether1 and other 1G ports are limited to 1G total (like connection on 2.5 is actually 1G , or communication speed from ether2,3,4,5,6,7,8 to ether1 is limited to 1G)
Tech Support sucks. Looks like hardware is not fixable. Time to sell this piece of *** on Ebay and never, ever buy anything from mikrotik!
Sell it, sell it!.
Did you get official reply from Tech support ?
Yes, I opened the case, but they pretend something else is wrong. Like LACP not giving more than 1G regardless 4 ports used for it. OK! So, I use ether1 as a regular port, not WAN port, and directly connect 3 computers to ether2, 3 and 4. And computer with proven working 2.5G port connected to ether1. All the same. Port ether1 is nor 2.5G port. It’s 1G port. They refuse to accept their hardware is not tested well and is actually an alfa version. I was downloading movie files from ether1 connected computer to three others at the same time, Combined speed is less than 45MB (360Mb/s), when 2.5G removed from ether1 advertisement - speed goes up to 113MB(904Mb/s). And my HD is M.2 Samsung 980 PRO SSD 2TB PCIe NVMe Gen 4.
So, I am done with this ***
So much easier to deal with American company, I am back to EdgeOS.
(PS they removed 5009 from Amazon…)
Running 7.2rc5 my 10G hosts are doing well, 2.3Gbps bidirectional on my 2Gbps over-provisioned AT&T link.
However, my 1G hosts are doing about 780Mbps down, and 950Mbps up. Not sure why the download is impacted. Happens on many hosts, and even with /interface/ethernet/switch/set 0 cpu-flow-control=no. May be my ISP fault (even though I can get up to 2.3Gbps), researching…
Either way a huge improvement.
UPDATE: In general, my 1G speeds are lower. 780-800-ish Mbps, and I’m pretty sure this is due to the upgrade. Uploads are not impacted.
With latest 7.2rc5 firmware on 1G hosts download speed (with 2.5G advertisement OFF) is 935Mb. And download speed on 1G hosts (with 2.5G advertisement ON) is 794Mb. I do not measure upload, because my ISP plan is: 2Gb down and only 75Mb up. Running two or three hosts at the same time gives the same combined speeds. Same cable from modem connected to PC NIC instead of RB5009 gives beautiful 2Gb download!
Ah you are seeing the same issue with 1G download that I am. Good to know I’m not alone. Working with Mikrotik on this, open a ticket and they may request info (stats counters, etc).
For me, everything fine. 2 PC on GBit of CRS326, a third one with 2.5 Gbit on 5009, CRS and 5009 connected via sfp+. Throughput about 1.8x Gbit as expected. Tested with original factory fw, about 8 weeks ago. Seems not to be a general hw problem. Also the ductus sounds like pebkac.
DNAT
[quote=madmardigan post_id=921347 time=1648203101 user_id=187199]
Are you plugging the 1G clients directly into the RB5009? When I do that the new FW is a bit better, but my typical setup has the RB5009 connected via 10G DAC to my switch, the 1G clients in there saw no noticeable improvement while the 10G clients in the same switch still got ~1400Mbps
[/quote]
My topology is still 2.5G AT&T Fiber <—> RB5009 <—10G DAC—> CRS328 <—> 1G/10G clients.
[quote=madmardigan post_id=921347 time=1648203101 user_id=187199]
Are you plugging the 1G clients directly into the RB5009? When I do that the new FW is a bit better, but my typical setup has the RB5009 connected via 10G DAC to my switch, the 1G clients in there saw no noticeable improvement while the 10G clients in the same switch still got ~1400Mbps.
[/quote]
I tried both: 1G clients directly to RB5009 1G ports (ether2,3,4) or to 1G switch connected with a LACP team of 4 ports (ether5,6,7,8). Results ~same.
And BTW, 5009 ether1 does not see modem 2.5G as 2.5G but as 1000M, but connection reported as 2.5G, full duplex on both devices.
Do you ground 5009 and how? (it is not in the rack)
Today I’ve got Wiitek SFP-10G-T. (It supports 1G,2.5G,5G,10G)
First, I connect my PC with 2.5G NIC to SFP+ at 2.5G,full. Download speed test (ether1 advertised as 1G) shows same ~900Mbs.
When I turned ether1 advertisement to 2.5G,full, my download speed drops to 95Mb/s (new firmware 7.2rc6) What was that?
Next step: SFP+ as WAN port, connection to modem as 2.5G, full. Ether1 as regular bridge port, 2.5G,full, connected to PC 2.5G NIC - download speed test shows ~2G!!!
Now, the final test: same PC connected to ether2 (should be 1G) Oops! Download speed on all 1G ports is only ~740Mb/s Tested multiple times - same result, sad…
This looks like congestion problems if download traffic ingresses with 2.5GB at ether1 and gets bridged (switched) towards the ether2-8 1GB ports.
As the bridge ingress port is faster than the egress port, the switch chip queue for the egress port overruns and packets get dropped.
This causes packet loss and issues with TCPIP windows sizes and ACKs and reduces the effective bandwith.
Has anyone checked if the traffic stats on 2.5GB ether1 towards the cable modem shows RX drops?
And for TX drops on ether2-8 towrds 1GB clients?
If there are, it could be solved by allowing bridge to produce backpressure towards the incoming download using flow control:
---------------------------------Download--------------------------------------->
CableModem<---2.5GB--->[ether1].RB5009bridge.[ether2]<---1GB--->LANclient
| |
Flow RX on off
Flow TX off on
<---------back pressure--------
This allows the RB5009 to apply back pressure towards the modem in case the 1GB output queue towards the client on ether2 is overrun by incoming 2.5GB downlad traffic on ether1.
This is how the same problem is often solved on 100GB core switches aggregating 10GB link to/from floor switches.
Unfortunately my cable modem (FAST3896U) has everything (like firewall, wifi6, router, 4 ether ports) but no setting for flow control. Love to try, but…
I have a similar problem with 1Gb devices and WiFi APs connected through the SFP+ port on the RB5009. I’m running 7.2.3. I’m using the 2.5Gb port for my cable modem and have the SFP+ connected to a CRS326-24S+2Q-RM with fiber. I got a little more speed… about 200Mbps. Playing with flow control helps a little, but ~200 is the best I’ve come up with. I forget which settings worked best… I tried “all” of them, then punted and plugged some stuff into the 1Gb ports on the RB5009. 10Gb hosts and my laptop using an Asus 2.5Gb USB adapter and an S+RJ10 get full internet speed, about 1.4Gbps. If I gimp ether1 to 1Gb the WiFi and computers connected at 1Gb work as they should, but of course that also limits the 10Gb machines and 2.5Gb laptop to 1Gb.
Disabling hardware offload in the bridge settings for the 1Gb ports seems to help, at least with my simple setup. Worth a try if you haven’t tried it already. I’m getting close to 900 from the 1Gb ports on the RB5009 from the Internet and nearly full speed locally. OTOH your Internet connection is faster than mine, so maybe that makes the problem worse?
The thing that really irks me about this problem is I can’t run a >1Gbps trunk line up to the second floor and hang a switch with 1Gb devices on it without gimping their internet speed, downgrading my service/setting ether1 to 1Gbps and gimping everything, or throwing out a router that’s only a few months old. I have two existing lines (going to different rooms), but adding more would involve a bunch of drywall work.. some of it on a ceiling.
Thank you for the tip (hardware offload) - will try it today at night.
Just curious, what is the temperature of the S+RJ10 module during the test? I bought Wiitek SFP-10G-T and It is not made by MikroTik but it’s temperature is 43C max.
Hotspot till have a problem. 
Load doesn’t actually seem to do much to it’s temperature. My S+RJ10 idles around 47-48C and only goes up to 50C whacking it with repeated local speed tests or file transfers. I’m sure I could get it a little hotter with a load generator. My setup is maybe a little out on the cool end for S+RJ10 temps outside of a datacenter. It’s all alone in a group of 8 ports in my CRS326-24S+2Q-RM, which has 3 cooling fans. The port count is overkill for me, but 8 ports wasn’t enough and I thought the 2 QSFPs were worth the price difference over the CRS317.
I thought of one more thing I should try - removing ports from the bridge on the RB5009. Maybe I’ll get to it this weekend. If that “fixes” internet speed to 1Gb hosts connected through the SFP+ port I’ll be quite happy. My target setup is the RB5009 only has 2 connections: 2.5G to the modem and 10Gb SFP+ to the CRS326. The idea here is to disable all hardware acceleration and just make the CPU do all the work. As long as it can manage a file download (aka single TCP connection) at 1.4Gb I’ll be happy.
If that doesn’t work I think I’m just going to make 2 VLANs for computers instead of one. The >1Gb VLAN would connect to the RB5009 using the SFP+ port and everything else would use a 1Gb port.
Well, I tried turning off Hardware offloading of ports in the bridge… SFP+ 2Gb down 75Mb up (cable modem), 2.5Gb NIC on ether1 - 2Gb down, 75Mb up (No difference with HW offloading on or off), on ether2 with HW offloading on - 790Mb down. HW offloading off - 800Mb+ down! BUT only first test, second one back to 790Mb down. HW offloading on again - speed goes to 800Mb+, but only first test, then goes back to 790Mb. So, no luck…