Strange issue when using SFP+ port compared to ether1

RouterOS Beginner Basics
1

Device: RB4011iGS+
Config: https://pastebin.com/Uc8mNVX6

My configuration is currently working with ether1 connected to my fibre ONT. Since I have 2.5Gb fibre supplied and wanted to get the full speed, I purchased a 10GTek 10GB SFP+ so that I can take full advantage of my 2.5Gbe ONT.

The configuration I assumed would be relatively simple:

  • update the vlan interface to use sfp-spfplus1 instead of ether1, doing the same for the “WAN”
  • update dhcp client
  • remove the sfp-spfplus1 interface from the bridge configuration.

I carried out these changes, rebooted. The interface is up, the DHCP client gets the static IP correctly from the ISP and my devices can ping 8.8.8.8, but nothing else seems to work. DNS resolution is broken, despite the fact the devices are bypassing the router configuration and hitting 8.8.8.8 directly. Same behaviour with the router via the terminal.

I’m a noob so I asked ChatGPT and tried a bunch of things:

  • disabling the firewall rules to confirm no rule was blocking
  • confirming the nat masquerade was set to the correct interface
  • setting the MTU to 1480
  • Set the MSS for outgoing connections : /ip firewall mangle add chain=forward protocol=tcp tcp-flags=syn action=change-mss new-mss=1360 out-interface=FastWebWAN
  • Reset connection tracking : /ip firewall connection tracking reset

What possible reason could icmp to 8.8.8.8 be working and yet DNS resolution fails? The only difference is switching from ether1 to the sfp+ port.

Compatibility with Ubiquiti SFP+ transcoders on Mikrotik switch
2

Do you think it’s possible that this SFP is not properly negotiating on the ONT side to 2.5Gbe?

I always figured that a ping means the link is up - is it possible that icmp can get through a dodgy link but TCP/UDP can’t?

3

Wild guess without seeing your configuration, but is the SFP interface added to the WAN interface list (and removed from LAN)?

4

Yes the WAN interface list was updated. I tried the physical port and the VLAN interface.

Given how simple this change is, I’m leaning on the SFP module being a problem. I have ordered another model from another vendor to try.

5

You need to check your part number against the table here https://www.10gtek.com/sfp1g%202.5g%2010g
Only 1 of the 10Gtek SFP+'s can negotiate to anything less than 10Gb

6

So I’ve got both a ASF-10G2-T and ASF-10G-T and they both exhibit the same behavior.

My understanding from some other review I saw, the ASF-10G-T only links on the SPF+ side at 10Gbe, but the Ethernet side will still auto negotiate at 2.5GBE.

I tried the ASF-10G2-T - and that has the option in the GUI at least to negotiate on the SFP+ side at 2500Base-T, however the SFP+ link did not come online with that setting, so I reverted back to 10G-BaseT and the link came online.

So both modules exhibit the same behaviour:

  • Link comes online with 10G-BaseT on the router side.
  • DHCP Client receives static IP when connected.
  • Public IP can be pinged from external addresses (when I enable it in firewall)
  • I can ping outbound to the internet.

However, no DNS resolution, either at the router side or device side, and testing TCP traffic with a HTTP request to public IP, also fails.

Not really sure what is going on.

7

I’ve verified that the ASF-10G2-T works on the lan side, auto negotiating to 1Gbps and traffic is routing from lan SFP out to the WAN just fine.

So I would assume this is negotiating just fine with the ONT.

So it’s just a question of why I can only pass ICMP traffic through when it’s configured on the WAN side.

8

Might be easier to post your latest config here between [__code][/__code] quotes.
Few are going to look for info behind an external link.

9

Static IP over DHCP on some ISPs binds to the MAC address it sees. If you’ve moved from an ether to sfpplus the MAC address will have changed.

As a test create a bridge, put both your ether and sfpplus into bridge and admin mac of your ether, dhcp client on the bridge.

Or

Call your isp and ask them to update the mac?

10

Bingo! Doohhh! That was the problem. I got totally sidetracked by the fact my DHCP was working and could ICMP ping, but they must have some deeper filtering.
I realised this when I browed to http IP address and I got a redirect to an internal ISP page, so routing was obviously working. I cloned the mac and tada!!