RB5009UG+S+IN SFP+ port not working with some switches on firmware up to 7.1.5

RouterOS RouterOS beta
1

I am not too sure if this is hardware issue or software but I suspect software first.

The issue is, on my newly bought RB5009 the SFP+ port does not seem to work with anything SFP I have. It appears that whenever something is connected, the port status on RB5009 side is shown as active all the time trying to Tx, but fails to negotiate speed and receives no Rx; but on the other side the connection will be shown as connected for a very short period of time (could be almost immediately to about a second) and then shown as disconnected.

I’ve tried all SFP devices I have, including:

  1. A pair of SFP (1G speed) RJ45 adapters of unknown brand
  2. A bunch of more than 5 different DAC I gathered from friends
  3. Two pairs of optical adapters of brand 10Gtek and HiFiber

All these are confirmed to be working totally fine on my RB4011iGS+RM and more than one Netgear Switches with SFP+ port.

The RB5009 comes with firmware version 7.0.5, but also tried to upgrade to 7.1rc6 (and later 7.1rc7 and 7.1 Testing) and issue is the exact same.

I’ve also tried to manually set various port speed on both sides, nothing helped.

Please let me know if there’s any more information I can provide to help this. At this moment, the issue is blocking my usage of RB5009.


UPDATE:
At this moment, I am almost sure this is an issue on Netgear MS510TXUP. I’ve able to make other switches to work with RB5009 without issue after a few mindless tries including cleaning the whole router board and DAC with electronic cleaner, a few configuration changes, and a few times of various resets.

UPDATE 2022-03-24:
I have confirmed the same issue is here with Zyxel XS1930-12HP too.

UPDATE 2022-04-19
Thanks to @xfoxtbat we have a plausible root cause:
https://forum.mikrotik.com/viewtopic.php?p=927657#p926511
It seems that the SFP+ port on RB5009 supports EEE which is not supported by the switches I found to have this issue.

SFP 1 Gbps devices in SFP+ cage (CCR2004-1G-12S+2XS) not working in 7.1x
2

Turn off Auto Negotiation and set the speed to 1G

I’ve had to do this on a CRS328 running 6.4x and on an RB5009 running 7.1RC6

3

Like mentioned in original post, did this, still the same issue.

4

For what it’s worth, I’m running an Optcore OSP10G-8503DCR on my RB5009 with 7.1rc6 and it works just fine with the default settings.

5

There was/is an issue with some DACs not working on some devices. This might be your issue.

My RB5009 and CRS328 are connecting using Cisco SFPs with fibre and both manually set to 10Gbps full duplex and autonegotiation disabled.

6

I understand it might be a compatibility issue, but all the options listed are working totally fine with RB4011. Just not on RB5009. It feels to me the compatibility is much worse on RB5009 compared to RB4011 if that’s the case.
I actually haven’t tried to manually set both side to 10Gbps (tried all sorts of lower speed though). I will give it a try.

7

With going to ROS7 you will lose some fixes that where already implemented in ROS6. The 4011 can’t reach a MTU of 1500 over PPPoE under 7 while that was fixed in 6.
Those fixes will be certainly applied also to 7 to but will take a while.

8

I have same issue, don’t have a stable solution. Just reboot the RB5009 have chance to make it work

9

Thanks for the information. At this moment, I tried to reboot about 10 times but no luck yet. Could you tell me what exactly did you do to make it work? Just so I don’t miss some details which could be critical.

10

I have a 5009, and it’s pretty fine with a Dac, a 1G SFP, and a dual speed 10G/1G SFP.
(Mostly fs.com bits, generic, cisco and hp profiles used)

Sometimes need to fiddle with the auto-negotiation on/off, 1G/10G selection, that’s about it.
Have suspicion that auto-negotiate might work better recently (maybe??).

Other thoughts

SFP Interface enabled?
Plugged in properly, and latch closed.?
Definitely says no link?
Perhaps your 5009 is broken, get your supplier to test it/swap it maybe.

Sorry, nothing more.

11

Thank you for your information!
I’ve confirmed everything you mentioned. I don’t have fs.com products but I’ve tried all I can find and that should be a good variety.
I can be certain that SFP interface are enabled. I’ve tried to factory reset both sides. Plug works fine - unless RB5009’s port is totally different from RB4011 that it seats differently. Yes definitely says no link on the other side, and on RB5009 it says connected, also says speed negotiation failed.
I will try get a replacement then.

12

I also have a similar problem with my RB5009UG+S+IN and AOC-STGN-I2S card. I have tried many configurations and so far no solution. I cannot negotiate the 10G port automatically or manually the port only gets up for a few seconds. I tried active and passive patchcords and the result is the same.

13

Hi All,

As a general update, I’m now on version 7.1.1, the issue is still there.

Also, I’ve witnessed something similar but different on RB4011:
After a few reboots, there’s once that on both RB4011 and Netgear MS510TXUP the link appears to be up but both side is receiving 100% error packets. I’ve rebooted both side once but the issue was still there, and then I rebooted a few more times and the issue is gone.

I suspect there’s a compatibility issue between RouterOS 7 and Netgear MS510TXUP.

14

Hi,
Have you ever tried the combination that leave MS510 as default and manually set RB5009 to 10Gbps full duplex and autonegotiation disabled?

15

Yes. I’ve tried to put both as default, then disabling features / use manual settings instead for everything I can think of. None helped.

At this time, I basically exhausted all possible configuration combinations available to me.

16

Not sure if this is anybody’s concern still. A bit of new development.
I’ve got a new switch, Zyxel XS1930-12HP. And this one showed basically the same issue as the MS510TXUP. It just does the cut off a few more cycles instead of just once.

I can be sure that the issue is with RB5009UG, not the other side now.

17

I have the same issue for rb5009 and xs1930-12hp. My rb5009 runs v7.2. Basically the sfp+ port on xs1930-12hp side just goes up and down all the time. I tried 10gb DAC and 10gb fiber cable. Neither works. Both work with mellanox connectx-3 nic.

Btw, I am wondering if adding a CRS305 between rb5009 and xs1930-12hp would help. But I cannot get one at the moment.

18

hi.all

I also had the same situation happen netgear MX510TXM
The response from the manufacturer was that the problem occurred when the MS510TXM continued to send packets to the MitroTik device, but the MitroTik device stopped sending data to the MS510TXM. The results of this test confirmed that it was caused by the MitroTik’s 10G SFP+ optical port supporting the EEE (Energy-Efficient Ethernet) feature, while the MS510TXM’s 10G SFP+ optical port does not support EEE (the MS510TXM’s 10G electrical port does support EEE)



Can rb5009 turn off EEE for testing spf+ available

19

OK. I ended up with using a SFP+ RJ45 on rb5009 and connecting a ethernet cable to xs1930’s RJ45. It is no ideal but works.

20

I tried the same and found that not all SFP+ RJ45 adapter works; and while some of them work, they are never stable that they disconnects quite randomly every a few hours.


This is great information! I haven’t got Netgear to help me this far. But yeah it sounds like a plausible cause and if Mikrotik could provide the option then we can confirm it. Thanks for sharing this information!