10Gtek SFP module ref ASF-GE-T auto negotiation problem with RouterOS > 7.12

melomac · Wed Mar 27, 2024 4:20 am

I am the proud owner of Mikrotik CRS309-1G-8S+IN since November 2023. That switch is fast, silent, and pleasant to use: thank you!

Currently, the device is setup like this:

port 1: 10Gtek SFP module ref ASF-10G-T (SFP-10G-SR) to Mac Studio
port 3: 10Gtek SFP module ref ASF-10G-T (SFP-10G-SR) to iMac Pro
port 5: 10Gtek SFP module ref ASF-GE-T (FTRJ8519P1BNL-PT) to PC Engine APU2C4 (pfSense appliance)
port 7: 10Gtek SFP module ref ASF-GE-T (FTRJ8519P1BNL-PT) to previous GbE unmanaged switch

The device has been operating reliably, consistantly, and silently, chilling at the typical cat rectal temperature of 38º C until today, when I decided to take a look at potential firmware updates (ex: security updates).

So I updated CRS309-1G-8S+IN, using Webfig's Quick Set "Check For Updates":

long term channel: from factory OS 6.48.6 to current v6 LTS
stable channel: from current v6 LTS to proposed v7.12 (dot something may be?)
stable channel: from proposed v7.12 to actual stable 7.14.1

It is unclear to me why the device offered v7.12 instead of v7.14.1 directly. EDIT: figured out this one:

What's new in 7.12.2 (2023-Dec-20 10:41):
(factory only release)
(...)
1. When upgrading by using "check-for-updates", all versions earlier than 7.12 will display 7.12 as the latest available version. Upgrade from v7.12 to v7.13 or later versions must be done through 7.12 in order to convert wireless packages automatically. Fresh installation with Netinstall or manual package installation works in the same manner as always.
(...)

Anyhow, and this is what's important: port 5 and 7 stopped working when device restarted with version v7.14.1

As a forum user stated Mikrotik is quick to address SFP support issues, I decided to give v7.15beta8 a try with no luck (note I had to swap cables 3 and 5 for the switch to get internet access again).

This is when I decided to I should downgrade to current RouterOS v6 version, or eventually that 7.12 something version, and this is how I lost a few hours of my life because, really, it's probably too subtle for me to downgrade that device of yours! (and I still have no clue)

Anyhow, I later realized disabling auto negotiation / forcing 10Gtek 1GB modules to operate at 1Gb full-duplex was enough to get those ports back to work.

Here you have ethernet info for SFP module with auto negotiation off:

[admin@MikroTik] > /interface ethernet monitor sfp-sfpplus5 once 
                    name: sfp-sfpplus5
                  status: link-ok
        auto-negotiation: disabled
                    rate: 1Gbps
             full-duplex: yes
         tx-flow-control: no
         rx-flow-control: no
               supported: 10M-baseT-half,10M-baseT-full,100M-baseT-half,100M-baseT-full,1G-baseT-half,1G-baseT-full,1G-baseX,2.5G-baseT,2.5G-baseX,5G-baseT,10G-baseT,10G-baseSR-LR,10G-baseCR
           sfp-supported: 1G-baseX
      sfp-module-present: yes
             sfp-rx-loss: no
            sfp-tx-fault: no
                sfp-type: SFP/SFP+/SFP28/SFP56
      sfp-connector-type: LC
     sfp-link-length-om1: 150m
     sfp-link-length-om2: 300m
         sfp-vendor-name: FINISAR CORP.
  sfp-vendor-part-number: FTRJ8519P1BNL-PT
     sfp-vendor-revision: A
       sfp-vendor-serial: 07J0PWF
  sfp-manufacturing-date: 05-05-02
          sfp-wavelength: 850nm
         eeprom-checksum: good
                  eeprom: 0000: 03 04 07 00 00 00 01 20  40 0c 05 01 0d 00 00 00  .......  @.......
                          0010: 1e 0f 00 00 46 49 4e 49  53 41 52 20 43 4f 52 50  ....FINI SAR CORP
                          0020: 2e 20 20 20 00 00 90 65  46 54 52 4a 38 35 31 39  .   ...e FTRJ8519
                          0030: 50 31 42 4e 4c 2d 50 54  41 20 20 20 03 52 00 cf  P1BNL-PT A   .R..
                          0040: 00 12 00 00 30 37 4a 30  50 57 46 20 20 20 20 20  ....07J0 PWF     
                          0050: 20 20 20 20 30 35 30 35  30 32 20 20 00 90 01 fd      0505 02  ....
                          0060: 00 00 00 00 00 00 00 00  4a 34 38 35 38 44 20 31  ........ J4858D 1
                          0070: 39 39 30 2d 34 34 31 35  df 67 d8 da 53 be 35 6d  990-4415 .g..S.5m
                          0080: 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ........ ........
                          *

And when auto negotiation is enabled, this is how the ethernet info looks like (spare 10Gtek ASF-GE-T adapter on port 6 with cable from port 7):

[admin@MikroTik] > /interface ethernet monitor sfp-sfpplus6 once
                      name: sfp-sfpplus6
                    status: no-link
          auto-negotiation: done
                 supported: 10M-baseT-half,10M-baseT-full,100M-baseT-half,100M-baseT-full,1G-baseT-half,1G-baseT-full,1G-baseX,2.5G-baseT,2.5G-baseX,5G-baseT,10G-baseT,10G-baseSR-LR,10G-baseCR
             sfp-supported: 1G-baseX
               advertising: 1G-baseX
  link-partner-advertising: 
        sfp-module-present: yes
               sfp-rx-loss: yes
              sfp-tx-fault: no
                  sfp-type: SFP/SFP+/SFP28/SFP56
        sfp-connector-type: LC
       sfp-link-length-om1: 150m
       sfp-link-length-om2: 300m
           sfp-vendor-name: FINISAR CORP.
    sfp-vendor-part-number: FTRJ8519P1BNL-PT
       sfp-vendor-revision: A
         sfp-vendor-serial: 07J0PWG
    sfp-manufacturing-date: 05-05-02
            sfp-wavelength: 850nm
           eeprom-checksum: good
                    eeprom: 0000: 03 04 07 00 00 00 01 20  40 0c 05 01 0d 00 00 00  .......  @.......
                            0010: 1e 0f 00 00 46 49 4e 49  53 41 52 20 43 4f 52 50  ....FINI SAR CORP
                            0020: 2e 20 20 20 00 00 90 65  46 54 52 4a 38 35 31 39  .   ...e FTRJ8519
                            0030: 50 31 42 4e 4c 2d 50 54  41 20 20 20 03 52 00 cf  P1BNL-PT A   .R..
                            0040: 00 12 00 00 30 37 4a 30  50 57 47 20 20 20 20 20  ....07J0 PWG     
                            0050: 20 20 20 20 30 35 30 35  30 32 20 20 00 90 01 fe      0505 02  ....
                            0060: 00 00 00 00 00 00 00 00  4a 34 38 35 38 44 20 31  ........ J4858D 1
                            0070: 39 39 30 2d 34 34 31 35  df 67 d8 da 53 be 35 6d  990-4415 .g..S.5m
                            0080: 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ........ ........
                            *

Questions

What is it we should do together in order to restore proper support including auto negotiation for 10Gtek SFP module ref ASF-GE-T (FTRJ8519P1BNL-PT) on RouterOS v7 stable branch?
What additional info do you need to reproduce the problem and test a fix?

melomac · Tue Apr 02, 2024 11:12 pm

The problem is still happening with RouterOS 7.15beta9:
As soon as I re-enable Auto Negotiation on interface sfp-sfpplus5 and sfp-sfpplus7, no link gets detected.

This wasn't the case until I updated to RouterOS > 7.12. What are the next steps for Mikrotik support to fix the problem?

rplant · Wed Apr 03, 2024 1:56 am

A couple of thoughts.

1. Plug the port into a 100M something, does it connect, and work?

2. Set auto negotiation, include 1000 base-x in the list of negotiation items.
(Or even have it as the only item)

melomac · Wed Apr 03, 2024 4:46 pm

Following up with Mikrotik support team — providing supout.rif files when the problem occurs, they quickly offered an effective alternative workaround:

It seems that the reason these modules are not working on auto-negotiation is due to incorrectly configured EEPROM.
This is indicated by the "sfp-rx-loss: yes" message observed in the SFP module monitoring.

As a workaround, we recommend disregarding sfp-rx-loss, to allow auto-negotiation function regardless. Below is a configuration example:
Code: Select all
/interface ethernet set sfp-sfpplus7 sfp-ignore-rx-los=yes

RouterOS won't detect the network cable is unplugged when auto negotiation is off:

/interface ethernet set sfp-sfpplus5 auto-negotiation=no
/interface ethernet set sfp-sfpplus7 auto-negotiation=no

RouterOS detects the network cable is unplugged when ignoring RX loss:

/interface ethernet set sfp-sfpplus5 sfp-ignore-rx-los=yes
/interface ethernet set sfp-sfpplus7 sfp-ignore-rx-los=yes

So this does look like the support team workaround is a better solution.
I'll update this thread as the problem gets addressed.

10Gtek SFP module ref ASF-GE-T auto negotiation problem with RouterOS > 7.12

10Gtek SFP module ref ASF-GE-T auto negotiation problem with RouterOS > 7.12

Re: 10Gtek SFP module ref ASF-GE-T auto negotiation problem with RouterOS > 7.12

Re: 10Gtek SFP module ref ASF-GE-T auto negotiation problem with RouterOS > 7.12

Re: 10Gtek SFP module ref ASF-GE-T auto negotiation problem with RouterOS > 7.12

Who is online