RouterOS x86 Performance Issue – One PC Limited to 200-300 Mbps While Another Reaches 900 Mbps

RouterOS General
1

Hello everyone,

I am facing a very strange issue with RouterOS x86 running on a Mini PC with 6 Ethernet ports.

Hardware

RouterOS x86 v7.23.1

RouterOS installed on a Mini PC with 6 LAN ports

WAN connected to ISP modem

LAN ports bridged together

FastTrack enabled

No Queue Tree

No Simple Queues

No Mangle rules

Default firewall rules

Network

LAN: 172.30.0.0/24

Router IP: 172.30.0.254

DHCP enabled

Problem

I have two Windows PCs connected directly to the RouterOS x86 box.

PC #1 (my PC)

Intel I219-LM NIC

Negotiated at 1 Gbps

Speedtest: 200-300 Mbps download

PC #2 (my son's PC)

Realtek 2.5 Gbps NIC

Speedtest: 900+ Mbps download

Important Findings

Same PC with old MikroTik router

When I reconnect my old RB3011 router:

My PC gets 900 Mbps

Son's PC gets 900 Mbps

Same PC directly to ISP modem

When I bypass RouterOS x86 completely and connect my PC directly to the ISP modem:

My PC gets 900 Mbps

USB Ethernet Adapter Test

I purchased and tested a USB-C to 2.5GbE adapter on my PC.

Results:

Adapter links at 2.5 Gbps

Speedtest still limited to around 200-300 Mbps through RouterOS x86

Therefore the issue is not related to the Intel I219-LM adapter.

Cable and Port Swap Tests

I tested:

Swapping Ethernet cables

Swapping RouterOS ports

Moving my PC to my son's room

Using the exact same cable and exact same port used by my son's PC

Result:

My PC still gets 200-300 Mbps

My son's PC still gets 900 Mbps

RouterOS Configuration Tests

I tested:

Factory reset

Fresh RouterOS configuration

New bridge

New DHCP server

Default firewall

FastTrack enabled

No queues

No mangle

No change.

Internal LAN Performance

File transfers inside LAN are normal.

Example:

Windows copy speed around 111 MB/s

Therefore LAN switching/bridging performance appears normal.

DHCP and IP Tests

I tested:

Different DHCP lease

Static IP

Using another IP address

No change.

VMware and Software Tests

I disabled:

VMware Bridge Protocol

VMware services

Avast Antivirus

No change.

Traceroute

Traceroute to fast.com appears normal.

Fast.com Results

Around 50 Mbps.

Speedtest.net Results

Around 220-300 Mbps.

Same ISP Connection

The exact same ISP connection gives:

900 Mbps with RB3011

900 Mbps when connected directly to modem

200-300 Mbps only on my PC through RouterOS x86

Question

Has anyone seen RouterOS x86 interact differently with a specific Windows PC while other PCs on the same router achieve full speed?

Could this be related to:

FastPath/FastTrack?

Bridge Hardware Offload?

Ethernet driver support in RouterOS x86?

MSS/MTU handling?

Connection tracking?

Intel vs Realtek interoperability?

Any ideas or diagnostic commands would be greatly appreciated.

Thank you.

2

Something you might be able to try that would be low impact is get a live usb live linux distro and boot your pc off that. If the speed results are the same, then you've seemingly isolated it to hardware or router os.

If the speed is fine on linux, then it would seem to be something related to the windows drivers or config.

At least it might point you in the right direction for what to focus on.

3

Thank you for the suggestion.

I tested exactly what you recommended using a Linux Mint Live USB boot (not VMware).

Unfortunately, the results were identical to Windows:

  • Windows 11: ~200-300 Mbps

  • Linux Mint Live USB: ~200-300 Mbps

Additional findings:

  • My ISP modem connected directly to the same PC delivers ~900 Mbps.

  • My old MikroTik RB3011 also delivers ~900 Mbps to the same PC.

  • The issue only appears when using my new RouterOS x86 mini PC.

  • I also tested a USB-C to 2.5GbE adapter and obtained the same 200-300 Mbps result.

  • Another PC with a Realtek 1Gb NIC also gets only 200-300 Mbps through the x86 router.

  • A third PC with a Realtek 5Gb NIC gets the full 900 Mbps through the same x86 router.

The mini PC uses six Intel I226-V Ethernet controllers and RouterOS 7.x.

Because the problem persists under both Windows and Linux Live USB, it seems unlikely to be an operating system or driver issue on the client side.

At this point I am starting to suspect a compatibility or negotiation issue involving Intel I226-V interfaces and 1Gb clients under RouterOS x86.

Has anyone seen similar behavior with Intel I226-V based RouterOS x86 systems?

4

Does the ISP modem also have a 2.5GbE port (that connects to the 2.5GbE I226-V port of your PC router), or is that port only 1GbE?

If the link between the modem and the PC router negotiates at 2.5Gbps, try to modify that port on the router to advertise only 1G-baseT-full so that the negotiated speed with the modem is only 1Gbps (let's assume ether1 is the WAN port).

/interface ethernet
set [ find default-name=ether1 ] advertise=1G-baseT-full

You can also try to set the interface queue on all the ethernet interface to mq-pfifo instead of only-hardware-queue.

/queue type
add kind=mq-pfifo mq-pfifo-limit=1000 name=mq-ethernet

/queue interface 
set [find default-queue=only-hardware-queue] queue=mq-ethernet
5

Thank you, this was a very useful suggestion.

I tested it and found something important:

My ISP modem negotiates at 2.5Gbps with the RouterOS x86 WAN port (Intel I226-V).

When the WAN port is left at 2.5Gbps:

  • PCs with 1Gb NICs only achieve about 200-300 Mbps.
  • A PC with a 5Gb Realtek NIC achieves the full ~900 Mbps internet speed.

When I force the WAN port to advertise only 1G-baseT-full and the link negotiates at 1Gbps:

  • All affected 1Gb clients immediately achieve ~900 Mbps.

So this clearly identifies the trigger, but it does not seem like a proper solution because the whole reason for purchasing this x86 router with Intel I226-V 2.5GbE ports was to use multi-gigabit connectivity.

The old MikroTik RB3011 and direct ISP modem connection both provide full speed to the same 1Gb clients without any issue.

Has anyone seen similar behavior with Intel I226-V interfaces under RouterOS x86 when the WAN is negotiated at 2.5Gbps and the clients are connected at 1Gbps?

Could this be related to:

  • Intel I226-V driver behavior in RouterOS?
  • Flow control?
  • Interface queue type (only-hardware-queue vs mq-pfifo)?
  • Hardware offloading?
  • Known Intel I225/I226 multi-gig interoperability issues?
  • A RouterOS v7 bug affecting Intel I226-V interfaces when the WAN link negotiates at 2.5Gbps and LAN clients are connected at 1Gbps?

I tested more than one RouterOS v7 release, including downgrading from 7.23.1 to an earlier stable version, and the behavior remained exactly the same.

I would like to understand the root cause rather than permanently limiting the WAN interface to 1Gbps.

6

There have been many similar experiences as yours on this forum. The speed issue usually occurs when packets coming from a higher throughput port (the 2.5Gbps WAN link) have to be squeezed into a narrower link (the 1Gbps link between the router and your PC). Packets will accumulates at the 1Gbps port. When the queuing / buffering is not adequate, then packets will have to be dropped.

Normally with TCP, the sender side (the remote host on the internet where you are downloading stuffs from) tries to regulate the sending speed, and use a congestion control algorithm for that. Usually, the sender starts with a small sending window (equal lower throughput), then tries to increase it. It depends on the algorithm used, but many are very sensitive to packet drop. If the sender detects packet drops, it will drastically reduce the sending speed, so you'll see for example the throughput climbing to over 900Mbps, then the sender sees a lot of packet drops, and reduces the congestion window, lowering the sending speed to 50Mbps for example. Then gradually increases that again because there will be no more packet drop, then sees packet drop again when throughput is higher than 1Gbps, and reduces that again. As a result, you experience only 200-300Mbps on average.

In your case, traffic between ISP modem port and router can temporarily jump above 1Gbps, but between router and PC, there is the 1Gbps bottleneck. And because the queue/buffer on the individual I226-V cards is not great, a lot of packets are dropped.

When you limit the link between ISP modem and router to 1Gbps, the there is no bottleneck within the router. The throughput reduction happens on the ISP modem, and that device has much better buffer/queue handling than whatever there is with the I226-V when the interface queue is only-hardware-queue.

Same when the port from router to PC runs at 2.5Gbps. There is no problem because there is no bottleneck caused by the speed reduction.

That's why I also suggested the 2nd part with the interface queue. If you don't want to reduce the speed on the port to the ISP modem, then you'll need to implement some better queueing inside the x86 router, that can better handle the speed reduction from 2.5Gbps WAN-in to 1Gbps LAN-out.

You can play around with different queue types and parameters.

Normally with MikroTik hardware you can try another workaround, that is to enable flow control on the ethernet ports. However I don't know whether RouterOS x86 can change the flow control setting of the Intel adapters. I only have the CHR version to test, and that one has no possibility to manipulate the flow control setting.

7

Thank you everyone for your help and suggestions.

After extensive testing, I found that the issue affects my RouterOS x86 system with Intel I226-V interfaces when the ISP modem is connected at 2.5Gbps and some clients are connected at 1Gbps.

Symptoms:

  • 1Gb clients achieved only 200-300 Mbps.
  • A 5Gb client achieved the full ~900 Mbps.
  • Direct connection to the ISP modem or my old RB3011 provided full speed.

I found two independent solutions:

Solution #1 Which was proposed by CGGXANNX

Change the interface queue type from:

only-hardware-queue

to:

mq-pfifo

This resolved the throughput issue for my 1Gb clients while keeping the WAN link at 2.5Gbps.

Solution #2

Keep the existing queue settings and instead enable:

  • Rx Flow Control
  • Tx Flow Control

on both:

  • The WAN interface connected to the ISP modem
  • The LAN interface connected to the affected client

This also resolved the issue while keeping the WAN link at 2.5Gbps.

Thank you to everyone who helped. Hopefully this information helps others using RouterOS x86 with Intel I226-V interfaces and mixed 2.5Gbps / 1Gbps environments.

8

did you disable the EEE for I-226V chip ?

With Proxmox and CHR/x86 , i am able to use ethtool to disable EEE and also increase the ring buffer to its max which is 4092 or something like that.

But for Mik to manage the driver, i could not find a solution to disable the EEE .