Community discussions

 
Paternot
Long time Member
Long time Member
Topic Author
Posts: 572
Joined: Thu Jun 02, 2016 4:01 am
Location: Niterói / Brazil

High Rx MAC Errors and Rx FCS Errors on CRS328-24P-4S+ and CSS326-24G-2S+

Thu Sep 13, 2018 5:55 pm

I got three switches: two CSS326-24G-2S+ and one CRS328-24P-4S+. They are still on the lab, being configured to production. Both the CSS326 are connected to the CRS328, through 10Gb fiber, using four S+85DLC03D SFP+ modules. All of them are running SwOS 2.8.

I named the CRS328 SW01, and the two CSS326 SW02 and SW03.

SW01 <-> SW02 uses SFP1
SW01 <-> SW03 uses SFP2

The images bellow show the SFP status of each switch.
sw01.png
sw02.png
sw03.png
Problem: I get high Rx MAC Errors and Rx FCS Errors, when traffic crosses the 10 Gb link. Tested with ping, iperf and iso download. These errors occur only at the fiber interfaces - the cooper ones are perfect.

Is there something I could do? Is it hardware? Software? Would future SwOS version correct this?
You do not have the required permissions to view the files attached to this post.
 
Paternot
Long time Member
Long time Member
Topic Author
Posts: 572
Joined: Thu Jun 02, 2016 4:01 am
Location: Niterói / Brazil

Re: High Rx MAC Errors and Rx FCS Errors on CRS328-24P-4S+ and CSS326-24G-2S+

Sat Sep 15, 2018 12:10 am

Now I got more time to test this. First results: looks like it is some software problem. SwOS 2.8 have it far worse than RoS 6.43. I used two Linux desktop machines to test this: banda01 and banda02.

banda01 is plugged in the sw02. banda02 is plugged in the sw03. They have to transverse sw01, through 10Gb fiber, to reach each other. The cooper interfaces got zero error. The fiber got Rx MAC Errors and Rx FCS Errors.

With RoS the errors were about 3 or 4, through the tests.
With SwOS the errors went through the roof: more than 3k.

The bulk of errors were at sw01. The other two switches got some errors - but not much. Below we can see four rounds of iperf - two with sw01 running SwOS 2.8 and two running RoS 6.43. I run one iperf session at banda01 and another at banda02.
Sw02 and Sw03 are connected through 10Gb fiber, at Sw01. All tests cross the switches.

1) Sw01 runing RoS 6.43

Round one (iperf running both directions at once)
iperf3 -t 200 -c banda02
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-200.00 sec  21.3 GBytes   915 Mbits/sec  158             sender
[  5]   0.00-200.04 sec  21.3 GBytes   915 Mbits/sec                  receiver

iperf3 -t 200 -c banda01
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-200.00 sec  21.1 GBytes   907 Mbits/sec   44             sender
[  5]   0.00-200.04 sec  21.1 GBytes   907 Mbits/sec                  receiver


Round two (iperf running both directions at once)
iperf3 -t 200 -c banda02
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-200.00 sec  21.3 GBytes   916 Mbits/sec  145             sender
[  5]   0.00-200.04 sec  21.3 GBytes   916 Mbits/sec                  receiver

iperf3 -t 200 -c banda01
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-200.00 sec  21.1 GBytes   906 Mbits/sec   31             sender
[  5]   0.00-200.05 sec  21.1 GBytes   906 Mbits/sec                  receiver



2) Sw01 runing SwOS 2.8

Round one (iperf running both directions at once)
iperf3 -t 200 -c banda02
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-200.00 sec  17.5 GBytes   752 Mbits/sec  4439             sender
[  5]   0.00-200.04 sec  17.5 GBytes   752 Mbits/sec                  receiver

iperf3 -t 200 -c banda01
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-200.00 sec  16.7 GBytes   717 Mbits/sec  7591             sender
[  5]   0.00-200.04 sec  16.7 GBytes   717 Mbits/sec                  receiver


Round two (iperf running both directions at once)
iperf3 -t 200 -c banda02
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-200.00 sec  17.6 GBytes   755 Mbits/sec  4321             sender
[  5]   0.00-200.04 sec  17.6 GBytes   755 Mbits/sec                  receiver

iperf3 -t 200 -c banda01
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-200.00 sec  16.6 GBytes   714 Mbits/sec  7637             sender
[  5]   0.00-200.04 sec  16.6 GBytes   714 Mbits/sec                  receiver
 
8cqv
Frequent Visitor
Frequent Visitor
Posts: 53
Joined: Tue Mar 06, 2018 5:25 pm

Re: High Rx MAC Errors and Rx FCS Errors on CRS328-24P-4S+ and CSS326-24G-2S+

Mon Sep 17, 2018 9:40 pm

Did support solve this for you?

I want to add a CSS326 to my CRS328, but worried about the issues you had with the trunk, CRS326 is a lot more expensive just to get ROS (and SwOS works well for a switch).

Do you have a DAC cable to try? Could it be FC specific?
 
Paternot
Long time Member
Long time Member
Topic Author
Posts: 572
Joined: Thu Jun 02, 2016 4:01 am
Location: Niterói / Brazil

Re: High Rx MAC Errors and Rx FCS Errors on CRS328-24P-4S+ and CSS326-24G-2S+

Mon Sep 17, 2018 10:08 pm

Did support solve this for you?

I want to add a CSS326 to my CRS328, but worried about the issues you had with the trunk, CRS326 is a lot more expensive just to get ROS (and SwOS works well for a switch).

Do you have a DAC cable to try? Could it be FC specific?
I didn't talked to them. Yet. Need to get some free time to do it. For now, it is working with RoS. Not what I wanted, but good enough. With two CSS326 and one CRS328 (running RoS), I got about 97% of link usage, on the 1Gb port - up and down. Sustained iperf test, running for 10 hours. With this setup I did get some errors, but nothing that bothers me. With the CRS328 running SwOS the situation was terrible: I couldn't get past 70% of link usage.

SFP1 on CRS328 (SW01)
sw01_p25.png
SFP2 on CRS328 (SW01)
sw01_p26.png
SFP1 on CSS326 (SW02)
sw02_p10.png
SFP1 on CSS326 (SW03)
sw03_p2.png
All SFP ports are configured as trunk.

All three switches have 7 VLANS: 1, 10, 200, 300, 400, 500, 600.
The two desktops used on the test were using VLAN 10.
You do not have the required permissions to view the files attached to this post.
 
8cqv
Frequent Visitor
Frequent Visitor
Posts: 53
Joined: Tue Mar 06, 2018 5:25 pm

Re: High Rx MAC Errors and Rx FCS Errors on CRS328-24P-4S+ and CSS326-24G-2S+

Mon Sep 17, 2018 11:39 pm

Just realised it works with CRS328 in ROS talking to CSS326 in SwOS that will be good enough for me.

Like the rrdtool graphs, what are you using mrtg?
Last edited by 8cqv on Tue Sep 18, 2018 9:20 am, edited 1 time in total.
 
Paternot
Long time Member
Long time Member
Topic Author
Posts: 572
Joined: Thu Jun 02, 2016 4:01 am
Location: Niterói / Brazil

Re: High Rx MAC Errors and Rx FCS Errors on CRS328-24P-4S+ and CSS326-24G-2S+

Tue Sep 18, 2018 3:42 am

Yes, MRTG
 
Paternot
Long time Member
Long time Member
Topic Author
Posts: 572
Joined: Thu Jun 02, 2016 4:01 am
Location: Niterói / Brazil

Re: High Rx MAC Errors and Rx FCS Errors on CRS328-24P-4S+ and CSS326-24G-2S+  [SOLVED]

Tue Sep 25, 2018 5:19 pm

Well, I got an answer from support (Ticket#2018092022000621). I'm hopeful and disappointed.

Hopeful because they know there is a problem, and will fix it in the future.
Disappointed that there is no ETA for this.

"Hello,

We have observed a similar behavior and will try to fix this issue in upcoming SwOS releases, but I cannot say any ETA yet. For now, we recommend using RouterOS.

Best regards,
Edgars P."
 
8cqv
Frequent Visitor
Frequent Visitor
Posts: 53
Joined: Tue Mar 06, 2018 5:25 pm

Re: High Rx MAC Errors and Rx FCS Errors on CRS328-24P-4S+ and CSS326-24G-2S+

Tue Sep 25, 2018 6:58 pm

Thanks for updating, I actually have my CSS326 on the way to run alongside 328 and replace an old netgear (Java GUI no longer supported) - will try the same test I am goign to try a DAC though not optical.
 
willtel
just joined
Posts: 1
Joined: Tue Nov 06, 2018 2:23 pm

Re: High Rx MAC Errors and Rx FCS Errors on CRS328-24P-4S+ and CSS326-24G-2S+

Tue Nov 06, 2018 2:39 pm

Someone solved this problem in the swos of crs 317RX MAC Errors And RX FCS Errors
I'm having this problem and this affects the upload of my clients

Note only one port of this error
 
8cqv
Frequent Visitor
Frequent Visitor
Posts: 53
Joined: Tue Mar 06, 2018 5:25 pm

Re: High Rx MAC Errors and Rx FCS Errors on CRS328-24P-4S+ and CSS326-24G-2S+

Fri Nov 16, 2018 11:54 am

No, support have no fix.

I was having huge random problems as it was my trunk between the switches, in the end I had to disable to port and use a 1G port for the trunk.

Not great is it :cry:

Support said to use ROS, but I don't want ROS on the 328 because its just a switch and SwOS is much quicker to make changes to vlans, etc
 
Paternot
Long time Member
Long time Member
Topic Author
Posts: 572
Joined: Thu Jun 02, 2016 4:01 am
Location: Niterói / Brazil

Re: High Rx MAC Errors and Rx FCS Errors on CRS328-24P-4S+ and CSS326-24G-2S+

Fri Nov 16, 2018 2:08 pm

Support said to use ROS, but I don't want ROS on the 328 because its just a switch and SwOS is much quicker to make changes to vlans, etc
Use RoS. I have one 328, and it solved my problems. I don't like the idea too - but it's better than no 10GiB at all.
 
ccornett
just joined
Posts: 1
Joined: Wed Jan 30, 2019 1:40 am

Re: High Rx MAC Errors and Rx FCS Errors on CRS328-24P-4S+ and CSS326-24G-2S+

Wed Jan 30, 2019 2:03 am

Did anyone get a chance to test this on the new SwOS FW 2.9?

Does the issue still persist?
 
8cqv
Frequent Visitor
Frequent Visitor
Posts: 53
Joined: Tue Mar 06, 2018 5:25 pm

Re: High Rx MAC Errors and Rx FCS Errors on CRS328-24P-4S+ and CSS326-24G-2S+

Sat Feb 02, 2019 8:31 pm

I have put 2.9 on CRS328, but I cannot get it to load on my CSS326 (using FW file in both cases). As it totally screwed my network last time I have not enabled the 10g uplink yet until both are on 2.9 but I see nothing in the release notes about the problem being fixed?

Fan's also did some weird stuff, but last 48 hours seem to now cut out when the switch is cold (overnight) and run when warm (peak daytime) as expected.
 
keefus
just joined
Posts: 9
Joined: Tue May 25, 2010 11:49 am

Re: High Rx MAC Errors and Rx FCS Errors on CRS328-24P-4S+ and CSS326-24G-2S+

Mon Feb 04, 2019 5:51 pm

Fixed in 2.9 so far :)
 
drodriguez
just joined
Posts: 1
Joined: Tue Apr 09, 2019 1:07 am

Re: High Rx MAC Errors and Rx FCS Errors on CRS328-24P-4S+ and CSS326-24G-2S+

Wed Apr 24, 2019 7:32 pm

Hello everyone, I have the same problem with a CRS317, this has ROS installed and the Rx FCS Error count keeps increasing. All the ring of my network is with CRS317 and I have the inconvenience only in some interfaces of some equipment. All have RouterOS v6.44.1. We use 10G Bi-Directional SFP + Optical Transceiver
Any idea or suggestion?
Greetings and thanks.

Who is online

Users browsing this forum: No registered users and 4 guests