Slower download speeds with MikroTik hEX S router and FTTP

RouterOS
1

I'm seeing some unexpected results with my MikroTik hEX S router.

Hope the experts here can help me understand what the problem might be.

Situation:

  • I recently upgraded our broadband link from ADSL2 to FTTP

  • FTTP upgrades WAN link download speed from 50Mbps to 900Mbps

  • I setup an iperf3 test to a privately-hosted AWS server

  • With a test device connected directly to the router, the Service Provider's router consistently provided the 900 - 930 Mbps download speed over many iperf3 tests (tested every 5 mins for 1 day)

Problem:

  • When I switched to the MikroTik hEX S router, the same test provided inconsistent results and with a lower average download speed.

  • See below for an example test result. Note the variability: 789 - 927 Mbps.

  • Compared with the service provider's router, the speeds are more variable and the average is generally 100 - 150 Mbps lower.

  • Repeating the tests over a week gave similar results.

Things I've tried - with no improvement:

  • Swapping the router devices & retesting.

  • Resetting MikroTik router configuration back to factory, & reconfiguring.

  • Upgrading MikroTik router to current version 6 (LTS)

  • Upgrading MikroTik router to current version 7 (stable)

  • Monitoring MikroTik router CPU when under sustained maximum download (CPU-1 & CPU-2 remained <= 85%)

  • Checking MikroTik WAN interface performance, by using Service Provider router as internet gateway. Reconfiguring MikroTik router as a DHCP Client connected to Service Provider router. Running iperf3 via the MikroTik router WAN interface to an iperf3 server connected to Service Provider router's switch. (Observed a stable 930+ Mbps over the WAN interface. Which indicates MikroTik device can perform as required with NAT, Firewall etc all active.)

Example iperf3 test results for MikroTik device, when acting as WAN gateway:

Connecting to host xxx.xxx.xxx.xxx, port 5201
Reverse mode, remote host xxx.xxx.xxx.xxx is sending
[ 5] local 192.168.1.232 port 39642 connected to xxx.xxx.xxx.xxx port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 107 MBytes 898 Mbits/sec
[ 5] 1.00-2.00 sec 106 MBytes 891 Mbits/sec
[ 5] 2.00-3.00 sec 98.1 MBytes 823 Mbits/sec
[ 5] 3.00-4.00 sec 94.0 MBytes 789 Mbits/sec
[ 5] 4.00-5.00 sec 95.7 MBytes 803 Mbits/sec
[ 5] 5.00-6.00 sec 94.9 MBytes 796 Mbits/sec
[ 5] 6.00-7.00 sec 104 MBytes 873 Mbits/sec
[ 5] 7.00-8.00 sec 110 MBytes 927 Mbits/sec
[ 5] 8.00-9.00 sec 105 MBytes 880 Mbits/sec
[ 5] 9.00-10.00 sec 99.8 MBytes 837 Mbits/sec

[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.01 sec 1019 MBytes 854 Mbits/sec 367 sender
[ 5] 0.00-10.00 sec 1015 MBytes 852 Mbits/sec receiver

Does anybody have any thoughts on what the problem might be?

2

Further detail: The FTTP link requires PPPoE Client on the router. MikroTik router was configured with a vanilla PPPoE client.

(The DHCP Client I mentioned was only for enabling the WAN interface speed test).

3

Sorry, one question, but about how you write your post.

you select line-by-line and press </> for each line for do this?:
Connecting to host xxx.xxx.xxx.xxx, port 5201
Reverse mode, remote host xxx.xxx.xxx.xxx is sending
[ 5] local 192.168.1.232 port 39642 connected to xxx.xxx.xxx.xxx port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 107 MBytes 898 Mbits/sec
[ 5] 1.00-2.00 sec 106 MBytes 891 Mbits/sec
[ 5] 2.00-3.00 sec 98.1 MBytes 823 Mbits/sec
[ 5] 3.00-4.00 sec 94.0 MBytes 789 Mbits/sec
[ 5] 4.00-5.00 sec 95.7 MBytes 803 Mbits/sec
[ 5] 5.00-6.00 sec 94.9 MBytes 796 Mbits/sec
[ 5] 6.00-7.00 sec 104 MBytes 873 Mbits/sec
[ 5] 7.00-8.00 sec 110 MBytes 927 Mbits/sec
[ 5] 8.00-9.00 sec 105 MBytes 880 Mbits/sec
[ 5] 9.00-10.00 sec 99.8 MBytes 837 Mbits/sec

Or you press </> after select all text for obtain this?:

Connecting to host xxx.xxx.xxx.xxx, port 5201
Reverse mode, remote host xxx.xxx.xxx.xxx is sending
[  5] local 192.168.1.232 port 39642 connected to xxx.xxx.xxx.xxx port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   107 MBytes   898 Mbits/sec
[  5]   1.00-2.00   sec   106 MBytes   891 Mbits/sec
[  5]   2.00-3.00   sec  98.1 MBytes   823 Mbits/sec
[  5]   3.00-4.00   sec  94.0 MBytes   789 Mbits/sec
[  5]   4.00-5.00   sec  95.7 MBytes   803 Mbits/sec
[  5]   5.00-6.00   sec  94.9 MBytes   796 Mbits/sec
[  5]   6.00-7.00   sec   104 MBytes   873 Mbits/sec
[  5]   7.00-8.00   sec   110 MBytes   927 Mbits/sec
[  5]   8.00-9.00   sec   105 MBytes   880 Mbits/sec
[  5]   9.00-10.00  sec  99.8 MBytes   837 Mbits/sec

???

4

I pasted in a block of text from a Google Doc. Then selected the whole block. Then pressed </> for the whole block. The title lines got messed up. So I went back again and applied </> in smaller blocks. But not line-by-line. Weird, I agree.

1 Like
5

Instead, regarding the problem,
if the specified model is correct,
the values ​​you get are within the range I would expect from the specifications.

https://mikrotik.com/product/hex_s#product_specification

(PPPoE is everytime single core, no matter RouterOS version!)

6

Ahh! The simplest answers are usually the right ones.

Is there a similar (MikroTik) device which could provide the target 900+Mbps speeds?

7

Is easy to reply, but similar prices, not...

I use RB5009 [7.16.2] for real 2.5G/1G VLAN+PPPoE GPON line.
https://mikrotik.com/product/rb5009ug_s_in#product_specification

1 Like
8

Oh, don't use 7.20.x for testing if you use PPPoE, see the Topic, is full of problems...

9

My advice is: don't change it.

Aside from running speed tests,
then, when you actually browse, not all sites offer more, just if you ask more...
Each has its limits... and maybe the total traffic you generate won't even fill the RouterBOARD...

10

Thanks - good advice.

I’ve got two hEX S devices, 1 with ROS 6 and 1 with ROS 7.

Interestingly, at least in my case, the performance and stability have been about the same so far.

It’s useful learning about the 7.20.x headaches & I’ll revert to v6 on both devices.

11

I agree with @rextended, your results are generally in line with what you can expect. I know this is not what you're looking for, but I would seriously consider leaving it alone (whatever you gain may not be worth it in either time or money.)

Basically all newer Mikrotiks have better cpus, so they will give better results. E.g. the ax3 would be an ideal candidate.

Bear in mind that Mikrotiks handle pppoe entirely in software (even in fastpath) but many isp provided routers use hw offloading, which is bound to be more deterministic. Also, the isp router may be aware of your isp's support for increased mtus, which could also explain some of the discrepancy.

12

Oh yes, I’ve been delighted with the hEX S device.

It has been amazing value over the years I’ve had it.

It is only now - with FTTP arriving - that I’m finally pushing up against its design limits.

And even now, it is still pretty good!

13

MTUs is something I’ve not played around with. I’d be interested in doing some further testing.

As a relative noobie with all this, could you help out & point me in the right direction?

14

First check if the MTU / MRU is 1500, 1492 or 1480 on PPPoE interface...

I'm expecting 1492.

15

I only sporadically deal with pppoe, so I'm not the best person to ask.

You should find plenty of docs/tutorials on this. It's usually referred to as rfc4638 support. Of course the first step is finding out if your isp supports it or not, which is often easiest by trying it.

16

Yes, Actual MTU is showing as 1492 in PPPoE Client

17

If it's not 1480, it means the ISP is compatible.

Does the line come with PPPoE over VLAN?

18

I’m unsure. It’s a retail customer facing service, which hasn’t mentioned VLAN. I’ve not (to my knowledge) used VLAN…

19

Try this:

Put 1500 on bot Max MTU and Max MRU on General on PPPoE Client interface and see if you have link or not.

Then on "Status" you can see the actual negotiated MTU and MRU.

1 Like
20

Done. Link remained good.

Status > MTU = 1500. MRU also 1500.