"fcs error on link", every day

Hardware MikroTik hardware general
1

Hello there,
I’m using a CCR1009 as “core” router, and a RB4011iGS+ as “edge” router.

Both are connected through their sfp-sfpplus1 interface.
I’m using two SPF+ modules: SFP-10G-BXU on one side (https://www.fs.com/fr/products/36351.html) and one SFP-10G-BXU (https://www.fs.com/fr/products/36353.html) on the other.

Problem is: I’ve to reboot the RB4011 on a daily basis because I got tons of “fcs error on link” in CCR1009 logs, every 30 seconds exactly.
On RB4011 side, I got sfp-sfpplus1 interface going down then up, again and again.

I’ve enabled the flow control on both sfp-sfpplus1 interfaces, without any changes.

Here’s the monitor output for both SFP+ interfaces:

                    name: sfp-sfpplus1
                  status: link-ok
        auto-negotiation: disabled
                    rate: 10Gbps
             full-duplex: yes
         tx-flow-control: yes
         rx-flow-control: yes
      sfp-module-present: yes
             sfp-rx-loss: no
            sfp-tx-fault: no
                sfp-type: SFP-or-SFP+
      sfp-connector-type: LC
     sfp-link-length-9um: 10000m
         sfp-vendor-name: FS
  sfp-vendor-part-number: SFP-10G-BX
       sfp-vendor-serial: F2010693887
  sfp-manufacturing-date: 19-12-04
          sfp-wavelength: 1330nm
         sfp-temperature: 64C
      sfp-supply-voltage: 3.303V
     sfp-tx-bias-current: 61mA
            sfp-tx-power: -2.21dBm
            sfp-rx-power: -2.937dBm
         eeprom-checksum: good

and

                    name: sfp-sfpplus1
                  status: link-ok
        auto-negotiation: disabled
                    rate: 10Gbps
             full-duplex: yes
         tx-flow-control: yes
         rx-flow-control: yes
      sfp-module-present: yes
             sfp-rx-loss: no
            sfp-tx-fault: no
                sfp-type: SFP-or-SFP+
      sfp-connector-type: LC
     sfp-link-length-9um: 10000m
         sfp-vendor-name: FS
  sfp-vendor-part-number: SFP-10G-BX
       sfp-vendor-serial: G2002252237
  sfp-manufacturing-date: 20-07-29
          sfp-wavelength: 1270nm
         sfp-temperature: 38C
      sfp-supply-voltage: 3.257V
     sfp-tx-bias-current: 30mA
            sfp-tx-power: -2.088dBm
            sfp-rx-power: -5.561dBm
         eeprom-checksum: good

Next step I see is to change SFP+ modules, one after the other.

Any suggestion welcomed,

Have a nice day

2

First thing I notice is that the SFP’s have different wavelengths…sure that will work?
What RouterOS version are you running?

3

I’m sure: I use mono mode fiber, so I need to have different wavelengths.
Both modules must be paired.


Oh, sorry, forgot that: I’m running the very last stable RouterOS release: 6.48.1
I setup the link few week ago, running 6.48

4

How long is the distance between that two devices?
If distance is less than 10km then i would replace sfp modules.
Also can you check fiber with otdr or any similar equipment?

5

From Link you provided those SFP+ Modules are Cisco Compatible!

6

We also have “fcs error on link” on our CCR1009s sometimes, but on the Ether-ports. Not experiencing any packetloss or issues.

7

I would say less than 20 meters: it’s a small office setup :wink:
Thanks for your advice, which confirms what I was thinking.

Cheers,
Jean Baptiste

8

Thanks for your update. Could you provide more details about any possible issue ?

As far as I can tell, modules have been perfectly detected by both Mikrotik routers.
Everything worked fine for 2 weeks or so, and then fcs errors showed up.

Cheers,
Jean Baptiste

9

Thanks for your comment.
Actually, I don’t see any error on ethernet ports, only sfp-sfpplus1 port is impacted.

Cheers,
Jean Baptiste

10

I changed both SFP+ module this morning.
Wait and see what happens now.

About testing fiber with an otdr, I don’t have such device for now, but I might consider buying one.
Do you have any model to recommend ?

Regards,
Jean Baptiste

11

Dont have any info which otdr, we always outsource company which is doing passive part (utp, fiber, etc.) :grin: