New High Performance Routers ! ?

Hardware MikroTik hardware general
1

Even CCR 1072 is not enough for us. Many low performance cores are not the way to go and they are easily saturated.
Are there any plans for new high performance routers from mikrotik ?

[r23@noc] /system resource cpu> pr 
 # CPU                                                                                                                   LOAD         IRQ        DISK
 0 cpu0                                                                                                                   28%         28%          0%
 1 cpu1                                                                                                                   44%         43%          0%
 2 cpu2                                                                                                                   22%         22%          0%
 3 cpu3                                                                                                                   27%         27%          0%
 4 cpu4                                                                                                                   24%         24%          0%
 5 cpu5                                                                                                                   61%         59%          0%
 6 cpu6                                                                                                                   59%         59%          0%
 7 cpu7                                                                                                                   34%         34%          0%
 8 cpu8                                                                                                                   29%         29%          0%
 9 cpu9                                                                                                                   42%         42%          0%
10 cpu10                                                                                                                  50%         50%          0%
11 cpu11                                                                                                                   8%          8%          0%
12 cpu12                                                                                                                  44%         44%          0%
13 cpu13                                                                                                                  46%         46%          0%
14 cpu14                                                                                                                  59%         59%          0%
15 cpu15                                                                                                                  43%         43%          0%
16 cpu16                                                                                                                  40%         40%          0%
17 cpu17                                                                                                                  54%         54%          0%
18 cpu18                                                                                                                  50%         50%          0%
19 cpu19                                                                                                                  18%         15%          0%
20 cpu20                                                                                                                  68%         67%          0%
21 cpu21                                                                                                                  38%         38%          0%
22 cpu22                                                                                                                  62%         62%          0%
23 cpu23                                                                                                                  42%         42%          0%
24 cpu24                                                                                                                  35%         35%          0%
25 cpu25                                                                                                                  40%         40%          0%
26 cpu26                                                                                                                  51%         51%          0%
27 cpu27                                                                                                                  36%         36%          0%
28 cpu28                                                                                                                  69%         69%          0%
29 cpu29                                                                                                                  61%         61%          0%
30 cpu30                                                                                                                  62%         62%          0%
31 cpu31                                                                                                                  22%         22%          0%
32 cpu32                                                                                                                   8%          8%          0%
33 cpu33                                                                                                                  11%         11%          0%
34 cpu34                                                                                                                  44%         44%          0%
35 cpu35                                                                                                                  20%         20%          0%
36 cpu36                                                                                                                  14%         14%          0%
37 cpu37                                                                                                                  68%         68%          0%
38 cpu38                                                                                                                  43%         43%          0%
39 cpu39                                                                                                                  44%         44%          0%
40 cpu40                                                                                                                  61%         61%          0%
41 cpu41                                                                                                                  32%         13%          0%
42 cpu42                                                                                                                  36%         36%          0%
43 cpu43                                                                                                                  39%         39%          0%
2

There were some gentle hints some time ago that something is on the way, but no specifics, no timeline.

3

You say that is not enough for you, but you are not using 100% of the CPU. How much traffic are you passing in total?

4

We are passing 9Gbit/s on the uplink. So it makes 18Gbit/s RX/TX traffic as total inside the router all interfaces combined.
Router only does NAT and nothing else. We had to use the “raw” filter to decrease the CPU utilisation. Without “raw” some cores were stuck at %100.
We are not sure what is the next step after ccr1072.
CHR is not stable at all for such traffic. Even it is not stable enough to run only “the dude” on it. Keeps locking up and needs reboot every week.

There is definitely a need for a high performance platform from mikrotik.

5

18 Gbit/s, when in the web page says that it can pass until 80 Gbit/s. There is something weird. Can you post your export with hide-sensitive option?

Regards.

6

MT’s estimates, are extremely optimistic. The tests are done virtually with a blank router, doing absolutely nothing at all - it doesn’t represent real world scenarios at all. Feel free to read up a bit on how MT actually conducts the tests, and what specifically, are tested.

9Gbps on a CCR, is quite impressive actually. I’ve also maxed out these devices in the 7 to 9 Gbps region.

7

Those tests are “standardized tests”, like the gas mileage tests on cars.
A test standard is required to be able to compare between different routers or cars, but the results do not represent at all what your actual daily performance will be.

Indeed, that is unfortunate. But it would probably be difficult to define more realistic tests and still do it in an implementation-agnostic way.
You would not want a test definition that by nature works much better on manufacturer X’s implementation than on your own, while in actual usage there would be no such difference.

I also sometimes think “gas mileage for cars test should be defined as the amount of gas required to drive on normal roads between two defined points divided by the distance” but in reality that would likely also cause some issues. Same with router performance tests.

8

Those tests are described here below the produt, in the Test tab:
https://mikrotik.com/product/CCR1072-1G-8Splus#fndtn-testresults

9

I think that the point of the whole thread is not about the tests, it’s about potential new high-end routers. And I think we’d all love to get some clarifications from Mikrotik about that.

10

@Normis
As I have stated before, we and I am sure most other ISPs as well are maxing out the current CCR products.
Are there any plans for new and more powerful architectures/products ?

11

CHR is indeed the only way forward for the forseeable future.
However, you should be able to get 20(40) or so gbps out of your CCR1072, Number 1: how many nat connections do you have. Number 2: don’t use masquerade anywhere. You’re using fast track, correct?
There are many small things that you can change that will drastically improve the performance of your router. That being said, this seems to be an edge router, are you overloading your router with tasks? Is it also acting as a core router, etc?

12

We are already getting 18gbit with RAW table populated for our public IPs not participating in conn.track table. Also with fast-track rules.
There is not much to do in terms of configuration.
CCR1072 CPU consumption is %50 with 18gbit/s total throuput + firewall + NAT plus some cores hitting %80.

CHR is super unstable product (at least on vmware). We use it only for “the dude” monitoring and 3 ppptp servers for managment. Total throuput is not even 10mbit/s
And Even in that case, it locks up. Needs reboot every week. It looses its winbox port and ssh accesibility. We reboot it from the vm console to make it work.
we cant put CHR in production at all.

What is left ? Should we go for other brands or mikrotik has a timeframe or any plan for the successor of the CCR products ?

13

What ROS is running your CCR?

Regards.

14

I am very surprised that Mikrotik does not use hardware NAT’ing.

15

i know it will not improve the performance, but you should really move away from pptp, esp if it is used for sensitive stuff, like management.

16

A10, one of the real high performance CGN manufacturers doesn’t necessarily use hw NAT either. we have some boxes from them, they handle several 10s of gbps with sw only nat on dual xeon CPUs (their hw appliance). believe or not, NAT - unlike encryption - is not really a CPU intensive process - i mean not more intensive than regular packet forwarding.
in general, the session setup will be always going along the slow path, and once the session is set up, data can flow either through a fastpath-like accelerated sw path (like DDTB/DDNB) or using a dedicated CPU instruction set (hw accelerated nat) which then consumes less resources compared to kernel mode sw path. in exchange you’ll limited by the hw-bound nat table size, thus the number of concurrent sessions - but those limits aren’t necessarily reached in many cases.

in either way, you should consider that hw-accelerated-nat - if it is available as CPU instruction, like with MT7621 (yes, mmips as in hexr3) - only shows its power when dealing with longer flows with lots of data. with short flows you will not see the big a wow effect. with hw accelerated nat you’ll have packet header rewrites done by a single (or few) instruction, sometimes even in cut-through-mode, where the rewrite happens as soon the packet header is received (and the rest of the packet is still in transit). to my best knowledge, RouterOS is using store-and-forward logic only.
on the contrary you’ll lose certain high level functions and most likely “nat helpers” - depending on how the code is actually written for specific types of streams (when to switch over to accelerated path).

17

I have clients with very stable CHR builds that are doing more than 10 Gbps in production ISP and Data Center networks. If you can only get 10 Mbps, something is wrong in your build.

Even in the testing I did for CHR MUM presentations where ESXi wasn’t the best, it still did 5 Gigs of traffic.

https://mum.mikrotik.com/presentations/EU18/presentation_5188_1524562405.pdf

18

We dont use CHR for real world customer traffic. we use it only for our managment traffic which is 10mbits. it freezes and lockes up. maybe dude is making it locking up. I have no idea but it is not stable.

Nevertheless, I really hoped for a CCR successor. I see that it wont be happening at all in the near future.

19

I’d hazard a guess at something by mid-2020, we have QSFP switches now I’m expecting a companion router - and not tile, less cores, more power per core, based on Router OS 7.0 (perhaps even a high powered ARM device given the 7.0beta has been exclusively released on ARM for now)

It wont be too far away.

20

There is a X86 Package. So you could go with plain PC hardware. For sure a core i7 will outperform CCR.