CRS112 traffic slow issue, with negotiation?

8577 · Mon May 04, 2020 4:49 am

I noticed file transfers were quite a bit slower for some reason. I checked the switch interfaces and tx/rx speeds were about 10Mbps through the trunk and server ports. Was seeing about 500K transfer speed.

When I went to set the router interface (ether2 on HAP ac2) to 1Gb no negotiation, and the same on the switch sfp interface, (1G non negotiate) the router and switch no longer moved traffic - zero on stats. So I changed them back, then tried rebooting the switch and router to hard set the trunk ports to 1Gb non-negotiate, and just won't connect. I did set negotiation back on, and set to only advertise 1Gb full, on both sides. Attached shows the before and after stats, seeing about 430Mbps now, and a file transfer ~40Mbps. Is this the 'wire speed' limit for these smaller switches? I set it up with no settings in the beginning, maybe my mistake, as I thought they would all just negotiate what is best.

I may try rolling back to my original config, (with no interface settings) but so far it's acceptable. I don't know what changed, has been running great for over a month (since March been running solid) with the help of the forums here.

Host on ether1 is a Linux 18.04 server set to 1Gb full.

Config attached:

[admin@MikroTik] > export hide-sensitive
# may/03/2020 18:40:12 by RouterOS 6.44.5
# software id = QYMG-SSQH
#
# model = CRS112-8P-4S
# serial number = 9B210B746392
/interface bridge
add name=bridge protocol-mode=none
/interface ethernet
set [ find default-name=ether1 ] advertise=1000M-full rx-flow-control=auto \
    tx-flow-control=auto
set [ find default-name=ether6 ] advertise=1000M-full auto-negotiation=no
set [ find default-name=sfp9 ] advertise=1000M-full,2500M-full \
    auto-negotiation=no rx-flow-control=auto tx-flow-control=auto
set [ find default-name=sfp10 ] advertise=\
    1000M-half,1000M-full,2500M-full,5000M-full,10000M-full auto-negotiation=\
    no
set [ find default-name=sfp11 ] advertise=1000M-full,2500M-full \
    rx-flow-control=auto tx-flow-control=auto
/interface vlan
add interface=bridge name=MGMT vlan-id=99
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile
set [ find default=yes ] html-directory=flash/hotspot
/system logging action
set 3 remote=192.168.1.246
/interface bridge port
add bridge=bridge interface=ether1
add bridge=bridge interface=ether2
add bridge=bridge interface=ether3
add bridge=bridge interface=ether5
add bridge=bridge interface=ether6
add bridge=bridge interface=ether7
add bridge=bridge interface=ether8
add bridge=bridge interface=sfp9
add bridge=bridge interface=sfp10
add bridge=bridge interface=sfp11
add bridge=bridge interface=sfp12
/interface ethernet switch egress-vlan-tag
add tagged-ports=sfp10,sfp9,sfp12,sfp11 vlan-id=10
add tagged-ports=sfp10,sfp9,sfp12,sfp11 vlan-id=20
add tagged-ports=sfp10,sfp9,sfp12,sfp11 vlan-id=30
add tagged-ports=switch1-cpu,sfp10,sfp9,sfp12,sfp11 vlan-id=99
/interface ethernet switch ingress-vlan-translation
add customer-vid=0 new-customer-vid=20 ports=ether5
add customer-vid=0 new-customer-vid=20 ports=ether6
add customer-vid=0 new-customer-vid=30 ports=ether7
add customer-vid=0 new-customer-vid=30 ports=ether8
add customer-vid=0 new-customer-vid=10 ports=ether2
add customer-vid=0 new-customer-vid=20 ports=ether3
add customer-vid=0 new-customer-vid=20 ports=ether4
add customer-vid=0 new-customer-vid=10 ports=ether1
/interface ethernet switch vlan
add ports=ether1,ether2,sfp10,sfp9,sfp12,sfp11 vlan-id=10
add ports=ether3,ether4,ether5,ether6,sfp10,sfp9,sfp12,sfp11 vlan-id=20
add ports=ether7,ether8,sfp10,sfp9,sfp12,sfp11 vlan-id=30
add ports=switch1-cpu,sfp10,sfp9,sfp12,sfp11 vlan-id=99
/ip address
add address=192.168.0.2/24 interface=MGMT network=192.168.0.0
/ip dns
set servers=192.168.1.244
/ip route
add distance=1 gateway=192.168.0.1
/system clock
set time-zone-name=America/Los_Angeles
/system logging
set 0 action=remote prefix=Sw
set 1 action=remote prefix=Sw
set 2 action=remote prefix=Sw
set 3 action=remote prefix=Sw

throughput-after.png

8577 · Tue May 05, 2020 12:25 am

I wound up setting ALL interfaces to 1Gb-full, with negotiation disabled. Everthing runs fine like that, even though the wiki suggests not to disable it.

By default 'auto-negotiation' is enabled on all interfaces: https://wiki.mikrotik.com/wiki/Manual:I ... e/Ethernet

"auto-negotiation (yes | no; Default: yes)
When enabled, the interface "advertises" its maximum capabilities to achieve the best connection possible.
Note1: Auto-negotiation should not be disabled on one end only, otherwise Ethernet Interfaces may not work properly.
Note2: Gigabit Ethernet and NBASE-T Ethernet links cannot work with auto-negotiation disabled."

Anyone know why gigabit ethernet would not work with auto-negotiate disabled? I have mine all disabled, and it seems to run fine. I gather this may just be out of date information. When auto-negotiation is enabled I will eventually pick up some latency on the LAN between hosts, so setting mine all to non-negotiate, at 1G-full.

mutluit · Tue May 05, 2020 12:43 am

It seems the standardization specs were changed multiple times...:
https://en.wikipedia.org/wiki/Autonegotiation

... The debatable portions of the autonegotiation specifications were eliminated by the 1998 release of 802.3. This was later followed by the release of the Gigabit Ethernet standard IEEE 802.3ab in 1999. The new standard specified that 1000BASE-T Ethernet at 1 Gbit/s over copper wiring requires autonegotiation. Currently, most network equipment manufacturers recommend using autonegotiation on all access ports and enable it as a factory default setting.[7][8][9] ...

Tue May 05, 2020 7:07 pm

Anyone know why gigabit ethernet would not work with auto-negotiate disabled?

My understanding is that for 1G (and faster) copper links it is not only connection speed that needs to be negotiated, but also the line needs to be tested and some other TX/RX parameters then needs to be negotiated and/or tuned. That's done during the standard link negotiation procedure. You can still disable auto-negotation on 1G ports, but that should generally be used only if you intend to use that port for 100M and slower links. In case you need to lock the port speed to 1G, you should edit the list of speeds and duplex modes to be advertised, but still keep auto-negotiation enabled.

8577 · Tue May 05, 2020 7:52 pm

Anyone know why gigabit ethernet would not work with auto-negotiate disabled?
My understanding is that for 1G (and faster) copper links it is not only connection speed that needs to be negotiated, but also the line needs to be tested and some other TX/RX parameters then needs to be negotiated and/or tuned. That's done during the standard link negotiation procedure. You can still disable auto-negotation on 1G ports, but that should generally be used only if you intend to use that port for 100M and slower links. In case you need to lock the port speed to 1G, you should edit the list of speeds and duplex modes to be advertised, but still keep auto-negotiation enabled.

Seems good in theory, but doesn't tend to work great in practice for me. I would think having interfaces auto negotiate would find the optimal settings, but for some reason it does not. Will leave mine as is for now unless I find issues.

8577 · Tue May 05, 2020 8:56 pm

FWIW: I see the exact same port speed issue on my Cisco 3850's and VOIP phones, where auto-negotiation settles on 100Mb and won't initialize, so it would seem the 'negotiation' algorithm may not be perfect across vendors, or at least the same bugs carry over into different vendors. So I don't think this is specific issue to MT.

Tue May 05, 2020 10:03 pm

Check your cables.

tdw · Tue May 05, 2020 10:39 pm

My understanding is that for 1G (and faster) copper links it is not only connection speed that needs to be negotiated, but also the line needs to be tested and some other TX/RX parameters then needs to be negotiated and/or tuned. That's done during the standard link negotiation procedure. You can still disable auto-negotation on 1G ports, but that should generally be used only if you intend to use that port for 100M and slower links. In case you need to lock the port speed to 1G, you should edit the list of speeds and duplex modes to be advertised, but still keep auto-negotiation enabled.
Seems good in theory, but doesn't tend to work great in practice for me. I would think having interfaces auto negotiate would find the optimal settings, but for some reason it does not. Will leave mine as is for now unless I find issues.

Autonegotiation will pick the best speed and duplex setting available at both ends, and is mandatory for gigabit and above even if only one speed & duplex option is to be advertised. The devices perform no media testing and will pick whatever the best setting is, even if the media cannot support it - if you are using media which will not support certain speeds & duplex options (e.g. an old Cat3 cable) it is up to you to disable the unsupported options on at least one device.

Some devices will fall back to 100Mb if not all four pairs are operational, but from observation this is not universal.

8577 · Wed May 06, 2020 6:20 am

Didn't think of cables, going to try swapping a few out if the latency comes back. Good idea. I just run the cheap cat5 freebies, probably not ideal!
Thanks to all

CRS112 traffic slow issue, with negotiation?

CRS112 traffic slow issue, with negotiation?

Re: CRS112 traffic slow issue, with negotiation?

Re: CRS112 traffic slow issue, with negotiation?

Re: CRS112 traffic slow issue, with negotiation?

Re: CRS112 traffic slow issue, with negotiation?

Re: CRS112 traffic slow issue, with negotiation?

Re: CRS112 traffic slow issue, with negotiation?

Re: CRS112 traffic slow issue, with negotiation?

Re: CRS112 traffic slow issue, with negotiation?

Who is online