CRS312-4C+8XG 10Gb/s issues

Hardware MikroTik hardware general
1

Hello,

Have a CRS312-4C+8XG with ROS 7.10 at home that started to misbehave on 10 Gb/s links.
For example, I have my PC plugged into ether2 with a 5 meters cat 7 patch cable. PC has an Intel X550-T2.
Noticed local transfers were slow and while checking the connection, I discovered that the link was established at only 1 Gb/s.
The only way to fix it initially was to change the port I was connecting my PC in the switch. Changed from ether2 to ether5 and the connection established at 10 Gb/s.
After one day, same issue. Back to 1 Gb/s.
Checked the plugs on my patch cable and everything was OK.
Tried plugging and unplugging the cable in the switch, but only got 1Gb/s
So…I plugged it into ether6. Instant 10Gb/s connection.
After another day, same issue. Back to 1 Gb/s.
Tried switching the cable from ether6 to ether5 and ether2 (the only ones available) and still only 1 Gb/s.
Left the cable plugged into ether6 and removed the power leads for 1 minute.
Plugged them back and instant 10Gb/s connection on all ports that were only connecting at 1 Gb/s prior (ether2,5 and 6)
No error showing up in logs.

Anyone else with this model that might observed the same behavior?

2

Possible fix

At the suggestion of Mikrotik support, I have disabled auto-negotiation on the ether port my PC connects to and set it to 10 Gbps.
Tested and no more reduced link speed.
I have enabled auto-negotiation again and tested some more, yet, no matter how many link downs, the connection is established at 10 Gbps pretty fast.
Weirdly, the simple act of disabling/enabling auto-negotiation cured my switch.
Hope it lasts.

3

I have a very similar problem starting with 7.10 and continuing through 7.11beta*. CRS312 and Aquantia AQtion 10Gbit attached to ether2 via 15m (or 10m) cable. I have only noticed it after waking the computer from sleep after 30-60 minutes. It will take 30+ seconds to negotiate to 1gbe. Normally it takes 5s to negotiate to 10gbe. When it happens, disabling and enabling the port or rebooting the switch fixes the problem.

Support also told me to disable auto-negotiation, but after waking from sleep one time it didn’t negotiate at all. Enabling auto-negotiation allowed it to connect at 1gbe.

What doesn’t work:

  • netinstall 7.11beta6
  • while the problem occurs: reboot the pc, disconnect and reconnect cable, disable and enable the NIC
  • changing cable

What works:

  • disable and enable port on switch
  • reboot switch

I have moved back to 7.9 temporarily. Hopefully they can reproduce the problem - I have sent several supouts.

4

I spoke too soon.
It seems after re-enabling auto-negotiation on the switch, the problem occurs again. 1Gbps only on 10 Gbps port after a couple of link-downs.
Disabled auto-negotiation again and so far, it seems stable.

5

I’m going to try out 7.11rc1. I’ve only seen the problem after the link negotiates to 100mbe. Do you think you may have the same problem or is it happening when the link has only been running at 10gbe?

What’s new in 7.11rc1 (2023-Jul-28 09:52):
…
*) ethernet - fixed forced half-duplex 10/100 Mbps link speeds on CRS312 device;
…

6

My PC has entered and exited sleep many times on 7.11rc1 without encountering the bug. It looks like this ROS release resolves the problem for me, but I will continue to test it.

7

Waiting for Mikrotik to fix something is a long game usually, so made a basic script to check, disable and enable the interface to fix the issue and notify me when this happens again.

:local port ether6
:local DeviceName [/system identity get name];
/interface ethernet monitor numbers=$port once do={
:if ("$status" = "link-ok") do {
:if ("$rate" != "10Gbps") do {
/interface ethernet disable $port
/interface ethernet enable $port
/tool e-mail 
set from=RouterOS-Warning<noreply@domain.xxx>
send to=myemail@domain.yyy \
subject="$DeviceName | Link speed warning for $port" body="The $port interface speed is lower than expected at $rate . Interface was disabled/enabled" \
}}}
8

Seems 7.11 fixed the issue. Hope permanently.

 08-22 02:51:20 interface,info ether6 link up (speed 10G, full duplex)
 08-22 02:52:37 interface,info ether6 link down
 08-22 02:53:04 interface,info ether6 link up (speed 10G, full duplex)
 08-22 02:53:07 interface,info ether6 link down
 08-22 02:53:14 interface,info ether6 link up (speed 10G, full duplex)
 08-22 02:53:35 interface,info ether6 link down
 08-22 02:54:02 interface,info ether6 link up (speed 10G, full duplex)
 08-22 02:54:03 interface,info ether6 link down
 08-22 02:54:11 interface,info ether6 link up (speed 10G, full duplex)
 08-22 02:54:23 interface,info ether6 link down
 08-22 02:54:58 interface,info ether6 link up (speed 10G, full duplex)
 08-22 02:55:00 interface,info ether6 link down
 08-22 02:55:07 interface,info ether6 link up (speed 10G, full duplex)
 08-22 02:55:21 interface,info ether6 link down
 08-22 02:55:28 interface,info ether6 link up (speed 10G, full duplex)
 08-22 02:57:16 interface,info ether6 link down
 08-22 02:57:28 interface,info ether6 link up (speed 10G, full duplex)
 08-22 02:57:29 interface,info ether6 link down
 08-22 02:57:36 interface,info ether6 link up (speed 10G, full duplex)
 08-22 03:00:46 interface,info ether6 link down
 08-22 03:01:13 interface,info ether6 link up (speed 10G, full duplex)
 08-22 03:01:16 interface,info ether6 link down
 08-22 03:01:23 interface,info ether6 link up (speed 10G, full duplex)
 08-22 03:01:27 interface,info ether6 link down
 08-22 03:01:34 interface,info ether6 link up (speed 10G, full duplex)
 08-22 03:06:18 interface,info ether6 link down
 08-22 03:06:38 interface,info ether6 link up (speed 10G, full duplex)
 08-22 03:07:24 interface,info ether6 link down
 08-22 03:07:52 interface,info ether6 link up (speed 10G, full duplex)
 08-22 03:07:54 interface,info ether6 link down
 08-22 03:08:01 interface,info ether6 link up (speed 10G, full duplex)
 08-22 03:08:05 interface,info ether6 link down
 08-22 03:08:12 interface,info ether6 link up (speed 10G, full duplex)
 08-22 03:10:23 interface,info ether6 link down
 08-22 03:10:42 interface,info ether6 link up (speed 10G, full duplex)
 08-22 03:10:59 interface,info ether6 link down
 08-22 03:11:25 interface,info ether6 link up (speed 10G, full duplex)
 08-22 03:11:28 interface,info ether6 link down
 08-22 03:11:35 interface,info ether6 link up (speed 10G, full duplex)
 08-22 03:11:39 interface,info ether6 link down
 08-22 03:11:46 interface,info ether6 link up (speed 10G, full duplex)
 08-22 17:06:12 interface,info ether6 link down
 08-22 17:06:31 interface,info ether6 link up (speed 10G, full duplex)
 08-22 17:06:49 interface,info ether6 link down
 08-22 17:07:16 interface,info ether6 link up (speed 10G, full duplex)
 08-22 17:07:18 interface,info ether6 link down
 08-22 17:07:26 interface,info ether6 link up (speed 10G, full duplex)
 08-22 17:07:44 interface,info ether6 link down
 08-22 17:08:07 interface,info ether6 link up (speed 10G, full duplex)
 08-22 17:08:09 interface,info ether6 link down
 08-22 17:08:16 interface,info ether6 link up (speed 10G, full duplex)
 08-22 17:08:24 interface,info ether6 link down
 08-22 17:08:31 interface,info ether6 link up (speed 10G, full duplex)
 08-22 17:15:20 interface,info ether6 link down
 08-22 17:15:40 interface,info ether6 link up (speed 10G, full duplex)
 08-22 17:16:28 interface,info ether6 link down
 08-22 17:16:56 interface,info ether6 link up (speed 10G, full duplex)
 08-22 17:16:58 interface,info ether6 link down
 08-22 17:17:06 interface,info ether6 link up (speed 10G, full duplex)
 08-22 17:17:10 interface,info ether6 link down
 08-22 17:17:17 interface,info ether6 link up (speed 10G, full duplex)
 08-22 17:24:02 interface,info ether6 link down
 08-22 17:24:20 interface,info ether6 link up (speed 10G, full duplex)
 08-22 17:26:04 interface,info ether6 link down
 08-22 17:26:30 interface,info ether6 link up (speed 10G, full duplex)
 08-22 17:26:33 interface,info ether6 link down
 08-22 17:26:40 interface,info ether6 link up (speed 10G, full duplex)
 08-22 17:26:44 interface,info ether6 link down
 08-22 17:26:51 interface,info ether6 link up (speed 10G, full duplex)
 08-22 18:18:52 interface,info ether6 link down
 08-22 18:19:10 interface,info ether6 link up (speed 10G, full duplex)
 08-22 18:19:29 interface,info ether6 link down
 08-22 18:19:55 interface,info ether6 link up (speed 10G, full duplex)
 08-22 18:19:59 interface,info ether6 link down
 08-22 18:20:05 interface,info ether6 link up (speed 10G, full duplex)
 08-22 18:20:10 interface,info ether6 link down
 08-22 18:20:16 interface,info ether6 link up (speed 10G, full duplex)
 08-22 18:23:54 interface,info ether6 link down
 08-22 18:24:14 interface,info ether6 link up (speed 10G, full duplex)
 08-22 18:24:25 interface,info ether6 link down
 08-22 18:24:51 interface,info ether6 link up (speed 10G, full duplex)
 08-22 18:24:55 interface,info ether6 link down
 08-22 18:25:03 interface,info ether6 link up (speed 10G, full duplex)
 08-22 18:25:06 interface,info ether6 link down
 08-22 18:25:12 interface,info ether6 link up (speed 10G, full duplex)
 08-22 18:29:53 interface,info ether6 link down
 08-22 18:30:16 interface,info ether6 link up (speed 10G, full duplex)
 08-22 18:30:28 interface,info ether6 link down
 08-22 18:30:35 interface,info ether6 link up (speed 10G, full duplex)
 08-22 18:30:53 interface,info ether6 link down
 08-22 18:31:12 interface,info ether6 link up (speed 10G, full duplex)
 08-22 18:31:16 interface,info ether6 link down
 08-22 18:31:23 interface,info ether6 link up (speed 10G, full duplex)
 08-22 19:46:38 interface,info ether6 link down
 08-22 19:46:51 interface,info ether6 link up (speed 10G, full duplex)
 08-22 19:46:52 interface,info ether6 link down
 08-22 19:46:58 interface,info ether6 link up (speed 10G, full duplex)
 08-22 19:47:03 interface,info ether6 link down
 08-22 19:47:09 interface,info ether6 link up (speed 10G, full duplex)
 08-22 19:48:38 interface,info ether6 link down
 08-22 19:48:56 interface,info ether6 link up (speed 10G, full duplex)
 08-22 19:50:26 interface,info ether6 link down
 08-22 19:50:45 interface,info ether6 link up (speed 10G, full duplex)
 08-22 19:50:49 interface,info ether6 link down
 08-22 19:50:56 interface,info ether6 link up (speed 10G, full duplex)
 08-22 19:51:56 interface,info ether6 link down
 08-22 19:52:16 interface,info ether6 link up (speed 10G, full duplex)
 08-22 19:52:19 interface,info ether6 link down
 08-22 19:52:27 interface,info ether6 link up (speed 10G, full duplex)
 08-22 19:55:44 interface,info ether6 link down
 08-22 19:56:04 interface,info ether6 link up (speed 10G, full duplex)
 08-22 19:56:07 interface,info ether6 link down
 08-22 19:56:16 interface,info ether6 link up (speed 10G, full duplex)
 08-23 01:09:39 interface,info ether6 link down
 08-23 01:09:59 interface,info ether6 link up (speed 10G, full duplex)
 08-23 01:10:40 interface,info ether6 link down
 08-23 01:11:07 interface,info ether6 link up (speed 10G, full duplex)
 08-23 01:11:10 interface,info ether6 link down
 08-23 01:11:17 interface,info ether6 link up (speed 10G, full duplex)
 08-23 01:11:21 interface,info ether6 link down
 08-23 01:11:28 interface,info ether6 link up (speed 10G, full duplex)
 08-23 01:51:17 interface,info ether6 link down
 08-23 01:51:36 interface,info ether6 link up (speed 10G, full duplex)
 08-23 01:55:25 interface,info ether6 link down
 08-23 01:55:51 interface,info ether6 link up (speed 10G, full duplex)
 08-23 01:55:55 interface,info ether6 link down
 08-23 01:56:01 interface,info ether6 link up (speed 10G, full duplex)
 08-23 01:56:06 interface,info ether6 link down
 08-23 01:56:12 interface,info ether6 link up (speed 10G, full duplex)
 08-23 02:01:09 interface,info ether6 link down
 08-23 02:01:28 interface,info ether6 link up (speed 10G, full duplex)
 08-23 02:01:49 interface,info ether6 link down
 08-23 02:02:16 interface,info ether6 link up (speed 10G, full duplex)
 08-23 02:02:19 interface,info ether6 link down
 08-23 02:02:27 interface,info ether6 link up (speed 10G, full duplex)
 08-23 02:02:30 interface,info ether6 link down
 08-23 02:02:37 interface,info ether6 link up (speed 10G, full duplex)
 08-23 02:06:16 interface,info ether6 link down
 08-23 02:06:35 interface,info ether6 link up (speed 10G, full duplex)
 08-23 02:12:02 interface,info ether6 link down
 08-23 02:12:28 interface,info ether6 link up (speed 10G, full duplex)
 08-23 02:12:31 interface,info ether6 link down
 08-23 02:12:38 interface,info ether6 link up (speed 10G, full duplex)
 08-23 02:12:42 interface,info ether6 link down
 08-23 02:12:49 interface,info ether6 link up (speed 10G, full duplex)
 08-23 03:12:39 interface,info ether3 link down
 08-23 03:12:47 interface,info ether3 link up (speed 10G, full duplex)
 08-23 07:43:55 interface,info ether6 link down
 08-23 07:44:15 interface,info ether6 link up (speed 10G, full duplex)
 08-23 07:44:44 interface,info ether6 link down
 08-23 07:45:11 interface,info ether6 link up (speed 10G, full duplex)
 08-23 07:45:14 interface,info ether6 link down
 08-23 07:45:21 interface,info ether6 link up (speed 10G, full duplex)
 08-23 07:45:25 interface,info ether6 link down
 08-23 07:45:32 interface,info ether6 link up (speed 10G, full duplex)
 08-23 07:51:09 interface,info ether6 link down
 08-23 07:51:28 interface,info ether6 link up (speed 10G, full duplex)
 08-23 07:51:59 interface,info ether6 link down
 08-23 07:52:34 interface,info ether6 link up (speed 10G, full duplex)
 08-23 07:52:37 interface,info ether6 link down
 08-23 07:52:44 interface,info ether6 link up (speed 10G, full duplex)
 08-23 07:52:48 interface,info ether6 link down
 08-23 07:52:56 interface,info ether6 link up (speed 10G, full duplex)
 08-23 07:55:33 interface,info ether6 link down
 08-23 07:55:53 interface,info ether6 link up (speed 10G, full duplex)
 08-23 07:56:03 interface,info ether6 link down
 08-23 07:56:29 interface,info ether6 link up (speed 10G, full duplex)
 08-23 07:56:33 interface,info ether6 link down
 08-23 07:56:39 interface,info ether6 link up (speed 10G, full duplex)
 08-23 07:56:44 interface,info ether6 link down
 08-23 07:56:50 interface,info ether6 link up (speed 10G, full duplex)
 08-23 07:58:17 interface,info ether6 link down
 08-23 07:58:30 interface,info ether6 link up (speed 10G, full duplex)
 08-23 07:58:32 interface,info ether6 link down
 08-23 07:58:38 interface,info ether6 link up (speed 10G, full duplex)
 08-23 07:59:09 interface,info ether6 link down
 08-23 07:59:35 interface,info ether6 link up (speed 10G, full duplex)
 08-23 07:59:39 interface,info ether6 link down
 08-23 07:59:45 interface,info ether6 link up (speed 10G, full duplex)
 08-23 07:59:50 interface,info ether6 link down
 08-23 07:59:56 interface,info ether6 link up (speed 10G, full duplex)
 08-23 08:03:41 interface,info ether6 link down
 08-23 08:04:00 interface,info ether6 link up (speed 10G, full duplex)
 08-23 08:04:20 interface,info ether6 link down
 08-23 08:04:48 interface,info ether6 link up (speed 10G, full duplex)
 08-23 08:04:50 interface,info ether6 link down
 08-23 08:04:59 interface,info ether6 link up (speed 10G, full duplex)
 08-23 08:05:01 interface,info ether6 link down
 08-23 08:05:08 interface,info ether6 link up (speed 10G, full duplex)
 08-23 18:24:05 interface,info ether6 link down
 08-24 03:14:56 interface,info ether6 link up (speed 10G, full duplex)
 08-24 03:14:58 interface,info ether6 link down
 08-24 03:15:05 interface,info ether6 link up (speed 10G, full duplex)
 08-24 03:15:20 interface,info ether6 link down
 08-24 03:15:28 interface,info ether6 link up (speed 10G, full duplex)
 08-25 15:29:40 interface,info ether6 link down
 08-25 15:29:53 interface,info ether6 link up (speed 10G, full duplex)
 08-25 15:29:54 interface,info ether6 link down
 08-25 15:30:00 interface,info ether6 link up (speed 10G, full duplex)
 08-25 15:30:21 interface,info ether6 link down
 08-25 15:30:48 interface,info ether6 link up (speed 10G, full duplex)
 08-25 15:30:51 interface,info ether6 link down
 08-25 15:30:57 interface,info ether6 link up (speed 10G, full duplex)
 08-25 15:31:02 interface,info ether6 link down
 08-25 15:31:08 interface,info ether6 link up (speed 10G, full duplex)
 08-25 15:38:55 interface,info ether6 link down
 08-25 15:39:14 interface,info ether6 link up (speed 10G, full duplex)
 08-25 15:39:28 interface,info ether6 link down
 08-25 15:39:54 interface,info ether6 link up (speed 10G, full duplex)
 08-25 15:39:58 interface,info ether6 link down
 08-25 15:40:04 interface,info ether6 link up (speed 10G, full duplex)
 08-25 15:40:09 interface,info ether6 link down
 08-25 15:40:15 interface,info ether6 link up (speed 10G, full duplex)
 08-25 15:43:40 interface,info ether6 link down
 08-25 15:43:58 interface,info ether6 link up (speed 10G, full duplex)
 08-25 15:44:29 interface,info ether6 link down
 08-25 15:44:56 interface,info ether6 link up (speed 10G, full duplex)
 08-25 15:44:59 interface,info ether6 link down
 08-25 15:45:06 interface,info ether6 link up (speed 10G, full duplex)
 08-25 15:45:10 interface,info ether6 link down
 08-25 15:45:17 interface,info ether6 link up (speed 10G, full duplex)
 08-25 15:49:30 interface,info ether6 link down
 08-25 15:49:49 interface,info ether6 link up (speed 10G, full duplex)
 08-25 15:50:16 interface,info ether6 link down
 08-25 15:50:42 interface,info ether6 link up (speed 10G, full duplex)
 08-25 15:50:45 interface,info ether6 link down
 08-25 15:50:52 interface,info ether6 link up (speed 10G, full duplex)
 08-25 15:50:56 interface,info ether6 link down
 08-25 15:51:03 interface,info ether6 link up (speed 10G, full duplex)
 08-26 04:48:10 interface,info ether6 link down
 08-26 04:48:29 interface,info ether6 link up (speed 10G, full duplex)
 08-26 04:50:06 interface,info ether6 link down
 08-26 04:50:34 interface,info ether6 link up (speed 10G, full duplex)
 08-26 04:50:36 interface,info ether6 link down
 08-26 04:50:44 interface,info ether6 link up (speed 10G, full duplex)
 08-26 04:50:47 interface,info ether6 link down
 08-26 04:50:54 interface,info ether6 link up (speed 10G, full duplex)
 08-26 06:46:04 interface,info ether6 link down
 08-26 16:01:32 interface,info ether6 link up (speed 10G, full duplex)
 08-26 16:01:34 interface,info ether6 link down
 08-26 16:01:41 interface,info ether6 link up (speed 10G, full duplex)
 08-26 16:01:56 interface,info ether6 link down
 08-26 16:02:04 interface,info ether6 link up (speed 10G, full duplex)
 08-26 22:25:18 interface,info ether6 link down
 08-26 22:29:29 interface,info ether6 link up (speed 10G, full duplex)
 08-26 22:29:31 interface,info ether6 link down
 08-26 22:29:38 interface,info ether6 link up (speed 10G, full duplex)
 08-26 22:29:53 interface,info ether6 link down
 08-26 22:30:00 interface,info ether6 link up (speed 10G, full duplex)
 08-26 22:41:34 interface,info ether6 link down
 08-27 06:46:16 interface,info ether6 link up (speed 10G, full duplex)
 08-27 06:46:17 interface,info ether6 link down
 08-27 06:46:24 interface,info ether6 link up (speed 10G, full duplex)
 08-27 06:46:39 interface,info ether6 link down
 08-27 06:46:46 interface,info ether6 link up (speed 10G, full duplex)
 08-27 08:39:09 interface,info ether6 link down
 08-27 16:52:50 interface,info ether6 link up (speed 10G, full duplex)
 08-27 16:52:52 interface,info ether6 link down
 08-27 16:53:00 interface,info ether6 link up (speed 10G, full duplex)
 08-27 16:53:14 interface,info ether6 link down
 08-27 16:53:21 interface,info ether6 link up (speed 10G, full duplex)
 08-27 17:57:11 interface,info ether6 link down
 08-27 18:43:48 interface,info ether6 link up (speed 10G, full duplex)
 08-27 18:43:51 interface,info ether6 link down
 08-27 18:43:57 interface,info ether6 link up (speed 10G, full duplex)
 08-27 18:44:12 interface,info ether6 link down
 08-27 18:44:19 interface,info ether6 link up (speed 10G, full duplex)
 08-27 19:48:09 interface,info ether6 link down
 08-27 20:28:48 interface,info ether6 link up (speed 10G, full duplex)
 08-27 20:28:50 interface,info ether6 link down
 08-27 20:28:57 interface,info ether6 link up (speed 10G, full duplex)
 08-27 20:29:12 interface,info ether6 link down
 08-27 20:29:20 interface,info ether6 link up (speed 10G, full duplex)
 08-27 20:32:02 interface,info ether6 link down
 08-28 08:25:22 interface,info ether6 link up (speed 10G, full duplex)
 08-28 08:25:24 interface,info ether6 link down
 08-28 08:25:35 interface,info ether6 link up (speed 10G, full duplex)
 08-28 08:25:49 interface,info ether6 link down
 08-28 08:25:56 interface,info ether6 link up (speed 10G, full duplex)
 08-28 22:57:02 interface,info ether6 link down
 08-29 03:32:09 interface,info ether6 link up (speed 10G, full duplex)
 08-29 03:32:11 interface,info ether6 link down
 08-29 03:32:18 interface,info ether6 link up (speed 10G, full duplex)
 08-29 03:32:33 interface,info ether6 link down
 08-29 03:32:40 interface,info ether6 link up (speed 10G, full duplex)
 01:40:32 interface,info ether6 link down
 02:50:42 interface,info ether6 link up (speed 10G, full duplex)
 02:50:45 interface,info ether6 link down
 02:50:51 interface,info ether6 link up (speed 10G, full duplex)
 02:51:07 interface,info ether6 link down
 02:51:13 interface,info ether6 link up (speed 10G, full duplex)
 03:58:54 interface,info ether6 link down
 14:19:14 interface,info ether6 link up (speed 10G, full duplex)
 14:19:16 interface,info ether6 link down
 14:19:24 interface,info ether6 link up (speed 10G, full duplex)
 14:19:38 interface,info ether6 link down
 14:19:46 interface,info ether6 link up (speed 10G, full duplex)