Community discussions

MikroTik App
 
rpress
Member Candidate
Member Candidate
Topic Author
Posts: 113
Joined: Thu May 07, 2009 5:13 am

i40e problem with DAC

Sat Aug 15, 2020 8:53 pm

1. Version 7.1beta1
2. CHR x86
3. Using the i40e driver, bring an interface up with a DAC already inserted and linked. The interface does not register "link ok". The DAC must be removed and plugged again to get the port to link.
4.
/interface ethernet
set [ find default-name=ether6 ] mtu=9100 name=ether6
5.

I have noticed a problem using the i40e driver. My adapter is an Intel X710-DA4. With a DAC in the SFP+ port, the interface does not register "link ok" when it is brought up. The DAC must be removed and plugged again to get the port to link. I also have a LR module (Edimax MG-10GAS1) and interestingly this does not have any problem. The problem occurs both on boot of the router and also if I manually disable and enable the interface.

The port shows it is receiving bytes, but there is no transmit. No surprise because it thinks the link is down.
   #      NAME                 RX-BYTE        TX-BYTE  RX-PACKET  TX-PACKET  R  T  TX-QU  R  T
   7  RS  ether6                 3 972              0         66          0  0  0      0  0  0

I didn't see this problem when using Linux.

Here is my DAC from ethtool:
	Identifier                                : 0x03 (SFP)
	Extended identifier                       : 0x04 (GBIC/SFP defined by 2-wire interface ID)
	Connector                                 : 0x21 (Copper pigtail)
	Transceiver codes                         : 0x00 0x00 0x00 0x00 0x00 0x04 0x00 0x00 0x00
	Transceiver type                          : Passive Cable
	Encoding                                  : 0x00 (unspecified)
	BR, Nominal                               : 10300MBd
	Rate identifier                           : 0x00 (unspecified)
	Length (SMF,km)                           : 0km
	Length (SMF)                              : 0m
	Length (50um)                             : 0m
	Length (62.5um)                           : 0m
	Length (Copper)                           : 1m
	Length (OM3)                              : 0m
	Passive Cu cmplnce.                       : 0x01 (SFF-8431 appendix E) [SFF-8472 rev10.4 only]
	Vendor name                               : CISCO-MOLEX
	Vendor OUI                                : 00:09:3a
	Vendor PN                                 : 74752-9519
	Vendor rev                                : 09
	Option values                             : 0x00 0x00
	BR margin, max                            : 0%
	BR margin, min                            : 0%
	Date code                                 : 120917
 
rpress
Member Candidate
Member Candidate
Topic Author
Posts: 113
Joined: Thu May 07, 2009 5:13 am

Re: i40e problem with DAC

Mon Sep 21, 2020 3:02 pm

This problem is unchanged with 7.1beta2.

I have also tried a couple different SFP+ to copper transceivers. They also have this problem.

My theory is that there is a race condition with the link status. When using the LR fiber transceiver (the one which works fine) the link is a bit slower to come up. This additional time delay allows for the Mikrotik software to register the link up status. With the DAC and copper transceivers, the link comes up faster. This causes the Mikrotik software to miss the link transition.
 
rpress
Member Candidate
Member Candidate
Topic Author
Posts: 113
Joined: Thu May 07, 2009 5:13 am

Re: i40e problem with DAC

Tue Dec 08, 2020 11:02 pm

This problem still exists with 7.1beta3.
 
rpress
Member Candidate
Member Candidate
Topic Author
Posts: 113
Joined: Thu May 07, 2009 5:13 am

Re: i40e problem with DAC

Thu Feb 11, 2021 1:15 pm

This problem still exists with 7.1beta4.
 
User avatar
krafg
Forum Guru
Forum Guru
Posts: 1024
Joined: Sun Jun 28, 2015 7:36 pm

Re: i40e problem with DAC

Thu Feb 11, 2021 9:40 pm

Contact directly with MikroTik Support.

Regards.
 
Spring00
just joined
Posts: 4
Joined: Fri May 28, 2021 4:11 pm

Re: i40e problem with DAC

Mon Jan 02, 2023 6:04 pm

I have the same problem in 7.7rc3, do you have any progress


1. Version 7.1beta1
2. CHR x86
3. Using the i40e driver, bring an interface up with a DAC already inserted and linked. The interface does not register "link ok". The DAC must be removed and plugged again to get the port to link.
4.
/interface ethernet
set [ find default-name=ether6 ] mtu=9100 name=ether6
5.

I have noticed a problem using the i40e driver. My adapter is an Intel X710-DA4. With a DAC in the SFP+ port, the interface does not register "link ok" when it is brought up. The DAC must be removed and plugged again to get the port to link. I also have a LR module (Edimax MG-10GAS1) and interestingly this does not have any problem. The problem occurs both on boot of the router and also if I manually disable and enable the interface.

The port shows it is receiving bytes, but there is no transmit. No surprise because it thinks the link is down.
   #      NAME                 RX-BYTE        TX-BYTE  RX-PACKET  TX-PACKET  R  T  TX-QU  R  T
   7  RS  ether6                 3 972              0         66          0  0  0      0  0  0

I didn't see this problem when using Linux.

Here is my DAC from ethtool:
	Identifier                                : 0x03 (SFP)
	Extended identifier                       : 0x04 (GBIC/SFP defined by 2-wire interface ID)
	Connector                                 : 0x21 (Copper pigtail)
	Transceiver codes                         : 0x00 0x00 0x00 0x00 0x00 0x04 0x00 0x00 0x00
	Transceiver type                          : Passive Cable
	Encoding                                  : 0x00 (unspecified)
	BR, Nominal                               : 10300MBd
	Rate identifier                           : 0x00 (unspecified)
	Length (SMF,km)                           : 0km
	Length (SMF)                              : 0m
	Length (50um)                             : 0m
	Length (62.5um)                           : 0m
	Length (Copper)                           : 1m
	Length (OM3)                              : 0m
	Passive Cu cmplnce.                       : 0x01 (SFF-8431 appendix E) [SFF-8472 rev10.4 only]
	Vendor name                               : CISCO-MOLEX
	Vendor OUI                                : 00:09:3a
	Vendor PN                                 : 74752-9519
	Vendor rev                                : 09
	Option values                             : 0x00 0x00
	BR margin, max                            : 0%
	BR margin, min                            : 0%
	Date code                                 : 120917
 
Spring00
just joined
Posts: 4
Joined: Fri May 28, 2021 4:11 pm

Re: i40e problem with DAC

Sun Jan 29, 2023 7:36 pm

I noticed that 7.8beta2 has the following updates:

*) x86 - added support for TP-Link TG-3468;
*) x86 - fixed SR-IOV support for Intel X710 series NIC;
*) x86 - improved Intel 500 series 10G SFP module support;
*) x86 - improved stability for Intel X550 series NIC with SR-IOV;

But unfortunately when I pass through the x710 network card to chr on the pve 7.3-3 platform, I still have the same problem.
 
arm920t
Frequent Visitor
Frequent Visitor
Posts: 61
Joined: Sat Aug 03, 2019 8:02 am

Re: i40e problem with DAC

Mon Jan 30, 2023 5:11 am

XXV710 SPF0 failed to get link on power up. The interface must be disabled and enabled once.
Auto negotiation is always failed
Snipaste_2023-01-30_11-07-00.png
Snipaste_2023-01-30_11-05-33.png
You do not have the required permissions to view the files attached to this post.
 
arm920t
Frequent Visitor
Frequent Visitor
Posts: 61
Joined: Sat Aug 03, 2019 8:02 am

Re: i40e problem with DAC

Mon Jan 30, 2023 5:14 am

IRQs are read-only and cannot be changed manually. RPS uses a lot of CPU resources
Snipaste_2023-01-30_11-12-23.png
You do not have the required permissions to view the files attached to this post.
 
irghost
Member
Member
Posts: 302
Joined: Sun Feb 21, 2016 1:49 pm

Re: i40e problem with DAC

Wed Oct 04, 2023 2:34 pm

any update on this?
 
hoeser
just joined
Posts: 17
Joined: Wed Jan 13, 2021 5:45 pm

Re: i40e problem with DAC

Tue Jan 30, 2024 8:41 pm

This appears to still be an issue even in 7.13.3

Minisforum Ms-01 with the X710 has the same issue. Disabling / enabling the interface on the X86 ROS box does nothing, but disabling/enabling on the CRS326 will bring it back up. Works fine until the X86 box is rebooted again. Really frustrating. Was hoping to build a killer routerOS box out of this thing but it looks like it's going to have to go to OpnSense or something.
 
User avatar
inteq
Member
Member
Posts: 416
Joined: Wed Feb 25, 2015 8:15 pm
Location: Romania

Re: i40e problem with DAC

Fri Apr 12, 2024 3:00 am

Having the same issue with MS-01 and X710 on latest Proxmox and passthrough.
I have to disable/enable the interface on the switch or unplug and plug back the DAC.
Tried two DACs and two SPF+ RJ45 modules. No difference,
Using CRS312-4C+8XG as switch and Mikrotik XS+DA0001 DAC. Mikrotik S+RJ10 also not working.
Contacted Mikrotik support and they advised to upgrade to a 7.15.something beta. Same issue.
So, 7.14.2, still same issue.
 
User avatar
inteq
Member
Member
Posts: 416
Joined: Wed Feb 25, 2015 8:15 pm
Location: Romania

Re: i40e problem with DAC

Sun Apr 14, 2024 8:24 pm

I found a strange "workaround" for this issue.
I was pretty sure this was an issue with Mikrotik CHR, but it seems it has nothing to do with it.

Youtube short recording with the issue and "workaround": https://www.youtube.com/watch?v=Dt97NDDTiU0

Short description of what happens in the video:

Machine was just started.
SSH to Proxmox host and do an ip a to show the interface is there.
Login to Proxmox web.
Start Mikrotik CHR VM.
Open VM's console and check ethernet for link.
Result: no link.

Reboot machine.
SSH to Proxmox host and issue ethtool enp3s0f0 (with or without watch -n1 does not matter)
Login to Proxmox web.
Start Mikrotik CHR VM.
Open VM's console and check ethernet for link.
Result: link.

I have repeated this multiple times to make sure it was not just a fluke.

Now, I have absolutely no clue why just issuing ethtool enp3s0f0 before starting the VM fixes this problem.
There is nothing changed on the nic, just displaying some info about it.
Issuing the command after the VM has been started will throw some error that it cannot find the interface and does not fix the issue.


How to automate it until a fix will be available (if ever):

Edit /etc/network/interfaces file and add pre-up /usr/sbin/ethtool <eth-name> like:
auto enp3s0f0
iface enp3s0f0 inet manual
        pre-up /usr/sbin/ethtool enp3s0f0

auto enp3s0f1
iface enp3s0f1 inet manual
        pre-up /usr/sbin/ethtool enp3s0f1
Need auto enp3s0f0 and auto enp3s0f1 or else the pre-up command is not executed.



Maybe anyone more versed can shed a light on this?
 
User avatar
inteq
Member
Member
Posts: 416
Joined: Wed Feb 25, 2015 8:15 pm
Location: Romania

Re: i40e problem with DAC

Sun Apr 21, 2024 6:00 am

Forget all the stuff above.
Just setting RX and TX flow control on the CHR interface to auto fixed my issue. (off by default)
Strangely, off or auto is still off when doing a
/interface/ethernet/monitor 0
, but somehow it works. I guess setting to auto changes something else also.

Who is online

Users browsing this forum: No registered users and 5 guests