Slow download speeds with MikroTik RB5009 router and FTTP with PPPoE

RouterOS
1

Following my earlier post re. slow performance over PPPoE with Hex S, I upgraded to an RB5009. I expected to see full speed over my FTTP link (936 Mbps), using the PPoE Client.

Much to my surprise, the RB5009 ROS v7 performance was not significantly better than Hex S (2018) running ROS v6.

I ran several iperf3 tests. Here's an example RB5009 iperf3 result:

[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 105 MBytes 881 Mbits/sec
[ 5] 1.00-2.00 sec 111 MBytes 929 Mbits/sec
[ 5] 2.00-3.00 sec 107 MBytes 899 Mbits/sec
[ 5] 3.00-4.00 sec 110 MBytes 922 Mbits/sec
[ 5] 4.00-5.00 sec 111 MBytes 932 Mbits/sec
[ 5] 5.00-6.00 sec 111 MBytes 930 Mbits/sec
[ 5] 6.00-7.00 sec 110 MBytes 924 Mbits/sec
[ 5] 7.00-8.00 sec 102 MBytes 852 Mbits/sec
[ 5] 8.00-9.00 sec 88.9 MBytes 746 Mbits/sec
[ 5] 9.00-10.00 sec 98.5 MBytes 826 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.01 sec 1.03 GBytes 886 Mbits/sec 1037 sender
[ 5] 0.00-10.00 sec 1.03 GBytes 884 Mbits/sec r

…and here's Hex S (2018) ROS v6:

[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 107 MBytes 898 Mbits/sec
[ 5] 1.00-2.00 sec 106 MBytes 891 Mbits/sec
[ 5] 2.00-3.00 sec 98.1 MBytes 823 Mbits/sec
[ 5] 3.00-4.00 sec 94.0 MBytes 789 Mbits/sec
[ 5] 4.00-5.00 sec 95.7 MBytes 803 Mbits/sec
[ 5] 5.00-6.00 sec 94.9 MBytes 796 Mbits/sec
[ 5] 6.00-7.00 sec 104 MBytes 873 Mbits/sec
[ 5] 7.00-8.00 sec 110 MBytes 927 Mbits/sec
[ 5] 8.00-9.00 sec 105 MBytes 880 Mbits/sec
[ 5] 9.00-10.00 sec 99.8 MBytes 837 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.01 sec 1019 MBytes 854 Mbits/sec 367 sender
[ 5] 0.00-10.00 sec 1015 MBytes 852 Mbits/sec receiver

I checked RB5009 CPU and it remained lightly loaded throughout the test.

With firewall rules, FastTrack hardware and all other rules are at their factory default.

It seems unlikely that RB5009 has no useful performance gain over Hex S 2018.

So I'm guessing I have a config issue. Although am a little unclear where to start.

I suppose with the obvious:

  • Have others got RB5009 working at 1Gbps with a PPPoE link to their ISP?
2

What is your MTU? It is said that RouterOS supports RFC4368 so you should be able to get 1500. I say “it is said that…” because the only references I can find at mikrotik.com about RFC4368 are in forum posts and not the documentation. IPv6 with PPPoE dial to ISP - No browsing - #11 by mducharme

3

Actual MTU is: 1492

4

Try and get it to 1500, might help.

5

Mmmm yes, altho this is a SoHo setup rather than a high-throughput setup. Tuning MTU will probably improve things by a small amount (<1%).

Looking to achieve more like >10% gain on the weak measurements and - importantly - getting that consistent >900 Mbps in the iperf3 measurements.

6

I see that you have 930mbps acchived in your iperf test. How much you want to have when your ISP line is 936mbps as you say?

7

936mbit/s - which ISP is selling such a number? Or is this a minimum guaranteed bandwidth?

8

Yes, > 2.2Gbps download with PPPoE and GPON is possible, even without fasttrack working.

If your result fluctuates a lot, try setting the CPU frequency to the fixed value of 1400MHz instead of auto (System -> RouterBOARD -> Settings) OR disable fasttrack (to produce a constant load). Although on my RB5009 that is not needed.

9

Thanks, and this has helped. Some interesting variation remains.

I adjusted configuration and re-ran several iperf3 tests. The example below is representative.

I noticed:

  • Improved individual results (more are >900 Mbps)
  • Still some “slow” results (<900 Mbps)

It’s surprising seeing any results < 900 Mbps. The rest of the network is virtually dormant, i.e. virtually no traffic.

[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 101 MBytes 847 Mbits/sec
[ 5] 1.00-2.00 sec 95.6 MBytes 802 Mbits/sec
[ 5] 2.00-3.00 sec 98.5 MBytes 826 Mbits/sec
[ 5] 3.00-4.00 sec 98.5 MBytes 826 Mbits/sec
[ 5] 4.00-5.00 sec 98.9 MBytes 829 Mbits/sec
[ 5] 5.00-6.00 sec 104 MBytes 868 Mbits/sec
[ 5] 6.00-7.00 sec 109 MBytes 916 Mbits/sec
[ 5] 7.00-8.00 sec 111 MBytes 931 Mbits/sec
[ 5] 8.00-9.00 sec 111 MBytes 931 Mbits/sec
[ 5] 9.00-10.00 sec 111 MBytes 932 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.01 sec 1.02 GBytes 873 Mbits/sec 232 sender
[ 5] 0.00-10.00 sec 1.01 GBytes 871 Mbits/sec receiver

Given my device is more-or-less vanilla, I ran into the device-mode blocker. I adjusted the CPU frequency in several steps:

  • Enable routerboard configuration:

    • system/device-mode/update mode=advanced routerboard=yes

    • Reboot device within given time (5 mins)

  • Change CPU mode:

    • System > Routerboard > Settings > CPU frequency: 1400MHz
10

936 Mbps is the baseline result I achieved when using the ISP provided router.

(I understand ISP router has hardware support for PPPoE.)

11

For info, here’s a sample of the iperf3 results achieved on the same line with the router provided by ISP

Average speeds is comfortably >900Mbps, and individual tests are mostly 936 Mbps

[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   108 MBytes   903 Mbits/sec
[  5]   1.00-2.00   sec   112 MBytes   936 Mbits/sec
[  5]   2.00-3.00   sec   112 MBytes   936 Mbits/sec
[  5]   3.00-4.00   sec   112 MBytes   936 Mbits/sec
[  5]   4.00-5.00   sec   112 MBytes   937 Mbits/sec
[  5]   5.00-6.00   sec   111 MBytes   931 Mbits/sec
[  5]   6.00-7.00   sec   110 MBytes   926 Mbits/sec
[  5]   7.00-8.00   sec   112 MBytes   936 Mbits/sec
[  5]   8.00-9.00   sec   112 MBytes   936 Mbits/sec
[  5]   9.00-10.00  sec   112 MBytes   936 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec  1.09 GBytes   934 Mbits/sec  304	sender
[  5]   0.00-10.00  sec  1.08 GBytes   931 Mbits/sec    receiver
12

I’d like to achieve the same performance using the RB5009.

13

You had 884 in your initial rb5009 test result. Now it is 873 and you see an improvement?

14

These result are pretty constant. While the bandwidth on ROS seems to fluctuate, but can still max out the line (as you reach 930+ partially).

15

Fair comment. Optimism bias on my part. You are quite right - there’s no real change.

16

I’ve just tried updating ROS to latest stable v7.20.7.

Still seeing similar variability:

[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 97.4 MBytes 816 Mbits/sec
[ 5] 1.00-2.00 sec 97.5 MBytes 818 Mbits/sec
[ 5] 2.00-3.00 sec 105 MBytes 884 Mbits/sec
[ 5] 3.00-4.00 sec 98.0 MBytes 822 Mbits/sec
[ 5] 4.00-5.00 sec 93.0 MBytes 780 Mbits/sec
[ 5] 5.00-6.00 sec 101 MBytes 844 Mbits/sec
[ 5] 6.00-7.00 sec 107 MBytes 901 Mbits/sec
[ 5] 7.00-8.00 sec 110 MBytes 923 Mbits/sec
[ 5] 8.00-9.00 sec 111 MBytes 931 Mbits/sec
[ 5] 9.00-10.00 sec 110 MBytes 923 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.01 sec 1.01 GBytes 866 Mbits/sec 541 sender
[ 5] 0.00-10.00 sec 1.01 GBytes 864 Mbits/sec receiver

17

How is ISP modem performing right now?

18

Yes, I wondered whether modem might be a problem.

I tested this some time ago by switching between ISP router & Mikrotik router several times, and running several iperf3 tests with each config.

Results were clear: ISP router + modem was consistently >900Mbps and generally 936Mbps.

Mikrotik devices were consistently less than this.

19

@rextended Hi again - and you kindly advised on my hEX S / FTTP / PPPoE performance issue.

I’ve got myself an RB5009 - and seeing a similar problem!

Wondered whether you could share your RB5009 config? Offer any more advice?

20

Maybe of help: Trying to understand the need for MSS Clamping - #3 by CGGXANNX