Hello

My ISP (Shaw Canada) has recently offered 600Mbps service using the Arris XB6 modem. I have upgraded to this level and bridged my modem as I prefer to run a Mikrotik router.

If I plug a computer straight into the Shaw modem while bridged, I can run Speedtest.net against Shaw/various local servers and it jumps very quickly to 660Mbps.

The issue I'm having is that when I try to do the same speed tests through my CCR (pc plugged straight into my "lan" port), I can never get more than 350-450Mbps and it "takes time" to get to the very top.

I thought it might be my configuration so I just factory reset the router and tried with no rules other than the very basic one the initial wizard makes. This didn't help.

Thinking it could be specific to my CCR, I borrowed a slightly different CCR1009 (think it is a newer model...one with passive cooling). Test results were the same.

The interesting thing, is that if I run speedtest on two computers at the same time (behind a switch plugged into my CCR) I can observe that the total bandwidth used does reach the expected maximum (I see 660Mbps in Winbox on the WAN interface).

During all of my single session tests, I never saw any CPU core get very high (nothing higher than 30%). Fast track didn't make a difference. Picking different ports to ensure they aren't on a shared switch on the MIkrotik (e.g. 1 and 5) didn't make a difference. All links show up as 1Gbps and I even tried different patch cables.

Firmware is 6.43.7 though I did try 6.40.9 as one vendor I worth with who is better versed in Mikrotik (and also sees this issue on CCR testing with iperf) suggested it. Made no difference.

Realistically, I seldom come close to using this much bandwidth (only got it because the price was right...lower than what I was paying prior) and if I ever did it is more likely that it would be multiple devices consuming at once. That said, I'd like to find out why I can't get full speed through a router that I thought was overkill for the job.

Any suggestions would be appreciated.

Thanks!

Hey

That's the limitation of hardware / design. CCR has 9 cores, and some functionality can be spread over them, but not all can be used by a single session, as you've noticed.
As you have also noticed, with multiple sessions you do get the advertised speed.

Default config is the most performant one for SOHO's, so there isn't anything that can be done to improve single session performance on this hardware. If "single session" is important, you'll need different router.

I had a similar issue, disabled queues / enabling fasttrack helped for me though. I think the problem is the RouterOS kernel is too old to support proper balancing of connections across multiple cores, hopefully this is fixed if / when RouterOS v7 comes out. 350 mbps does seem on the low side though, have you checked your OS TCP tuning? Windows at least can use different settings for different networks which can complicate testing.

viewtopic.php?t=131503

try using only eth5, eth6, eth7, eht8, sfp and sfp+, this interfaces have direct and independent access to CPU, using this interfaces improves performance

try using only eth5, eth6, eth7, eht8, sfp and sfp+, this interfaces have direct and independent access to CPU, using this interfaces improves performance

A nice tip, given the diagram https://i.mt.lv/cdn/rb_files/CCR1009-8G ... 140835.png

Same story on CCR1009-7G, need to enable fasttrack then I can have 800~900Mbps for single connection, without fasttrack it also limited at 300~500Mbps too but none of the CPU goes over 60% usage

I'm getting 10G easily with fasttrack, even without jumbo frames.

Here's more detailed performance analysis for more home oriented usage:
viewtopic.php?t=138626

I'm getting 10G easily with fasttrack, even without jumbo frames.

Here's more detailed performance analysis for more home oriented usage:
viewtopic.php?t=138626

I think this really depends on config, I can also get my CCR1009 go with full 1Gbps with limited set of rules and fastrack disable, but once I add more complicate config it does slow down a lot, with some of my setup I cam see 2 of the core go to 100% while only 300mbps single tcp steam.

I think with 80+ firewall rules, 2 VPN connection and policy based routing using mangle 300~500Mbps for single tcp steam without fasttrack is acceptable, but well, if mikrotik can release a new routerboard with much faster core I will happily to replace the CCR.

I'm getting 10G easily with fasttrack, even without jumbo frames.

Here's more detailed performance analysis for more home oriented usage:
viewtopic.php?t=138626

I think this really depends on config, I can also get my CCR1009 go with full 1Gbps with limited set of rules and fastrack disable, but once I add more complicate config it does slow down a lot, with some of my setup I cam see 2 of the core go to 100% while only 300mbps single tcp steam.

I think with 80+ firewall rules, 2 VPN connection and policy based routing using mangle 300~500Mbps for single tcp steam without fasttrack is acceptable, but well, if mikrotik can release a new routerboard with much faster core I will happily to replace the CCR.

I'm also using CCR1009 as vpn (ovpn) concentrator (lots of connections), several hundred firewall rules, several hundred address list entries, interface lists, PBR for management traffic, dual WAN with 3G attached using usb adapter, various HA monitiring scripts, EoIP with IPSec and buttload of stuff so it's probably hard to put more junk on it xD

The only things that really matters are:
- bridge (bridges heavily cripple performance)
- fasttrack
- jumbo frames

other things are less relevant (unless you're talking strictly about encrypted traffic performance). If you use stateful firewall with fasttrack then whole firewall processing is negligible. If you need QoS then Queue Tree should work with fasttrack.

Basically you should be using fasttrack unless there's absolutely no way ever to do that (which is rare since most of stuff is doable with fasttrack enabled).

That said x86 also struggles with traffic processing without hardware acceleration. Eg. virtual networking even on i7 can drop down to 1gbps. If you're concerned about single stream performance then you can try RB4011. It should perform better in such scenario.

If you need QoS then Queue Tree should work with fasttrack.

From the manual:
Queues (except Queue Trees parented to interfaces), firewall filter and mangle rules will not be applied for FastTracked traffic.

If you need QoS then Queue Tree should work with fasttrack.

From the manual:
Queues (except Queue Trees parented to interfaces), firewall filter and mangle rules will not be applied for FastTracked traffic.

Yes but typically you use Queue Tree parented to interface, not global. Also you always fasttrack already accepted connections so firewall works in stateful manner. So does mangle for connection marks and routing marks. However yeah QoS is kind of problematic with fasttrack.