Community discussions

MikroTik App
 
pospanko
Member Candidate
Member Candidate
Topic Author
Posts: 279
Joined: Sun Dec 18, 2005 4:23 pm

Shapeing 10G of traffic

Wed Nov 07, 2018 11:11 pm

Hi all!

I need your opinion of what to use for shapening user bandwith of total 10G throughput?
CCR1072 can't handle that, not even half of it. We use lots of them currently but is hard to maintain. Is x86 way to go?
How do you do load balancing and failover? Do you use DHCP or PPPoE? Simple queues or queue tree? Do you maybe do that on CPE?
Thx
 
mistry7
Forum Guru
Forum Guru
Posts: 1480
Joined: Tue Oct 13, 2009 11:57 am
Location: Germany

Re: Shapeing 10G of traffic

Thu Nov 08, 2018 4:58 am

Until x86 is dead look at CHR
 
pospanko
Member Candidate
Member Candidate
Topic Author
Posts: 279
Joined: Sun Dec 18, 2005 4:23 pm

Re: Shapeing 10G of traffic

Thu Nov 08, 2018 8:59 am

Until x86 is dead look at CHR
How do you mean 'x86' is dead? I see packages in download.
My main question goal was more about HW then SW...
 
User avatar
vecernik87
Forum Veteran
Forum Veteran
Posts: 882
Joined: Fri Nov 10, 2017 8:19 am

Re: Shapeing 10G of traffic

Thu Nov 08, 2018 9:20 am

It is in the downloads, but it is not recommended to use on bare metal due to limited driver support. Keep in mind this was released long time ago when mikrotik was experimenting with x86 architecture (RB230).
In addition you can't transfer the x86 license (for example if your hardware fails). On the other hand, CHR has transferable license.

sorry, I can't help with rest of your question.
 
mistry7
Forum Guru
Forum Guru
Posts: 1480
Joined: Tue Oct 13, 2009 11:57 am
Location: Germany

Re: Shapeing 10G of traffic

Thu Nov 08, 2018 10:35 am

Until x86 is dead look at CHR
How do you mean 'x86' is dead? I see packages in download.
My main question goal was more about HW then SW...
X86_64 Support is only Basic, so you Are not able to Provit from newer Hardware!
 
r00t
Long time Member
Long time Member
Posts: 672
Joined: Tue Nov 28, 2017 2:14 am

Re: Shapeing 10G of traffic

Thu Nov 08, 2018 4:08 pm

It's harder to find network cards that are supported in x86, but once you find one that works and gets the job done for you, it just works.
License is bound to the disk, so best is to buy some good quality USB flash drive, overprovision it greatly (64GB drive, create 256MB partition for ROS) and purchase license for it. ROS doesn't write a lot of stuff to the disk, but even if it did, it would take a LONG time before the flash wears out. Small SSD would also work well if you can't boot from USB and need SATA drive.
 
pospanko
Member Candidate
Member Candidate
Topic Author
Posts: 279
Joined: Sun Dec 18, 2005 4:23 pm

Re: Shapeing 10G of traffic

Fri Nov 09, 2018 5:55 am

Ok, x86 or CHR, there will be small difference. Lets say we done this part of my question.
Much more important thing is: Is it capable of shape that amount of traffic?
 
pospanko
Member Candidate
Member Candidate
Topic Author
Posts: 279
Joined: Sun Dec 18, 2005 4:23 pm

Re: Shapeing 10G of traffic

Tue Nov 13, 2018 8:55 pm

Anyone on other topics?
 
User avatar
Steveocee
Forum Guru
Forum Guru
Posts: 1120
Joined: Tue Jul 21, 2015 10:09 pm
Location: UK
Contact:

Re: Shapeing 10G of traffic

Wed Nov 14, 2018 1:31 pm

How are you planning on shaping the traffic? (Out of interest).
I would recommend CHR as it is a "current" product where X86 has been left behind a bit in terms of hardware support.
 
pospanko
Member Candidate
Member Candidate
Topic Author
Posts: 279
Joined: Sun Dec 18, 2005 4:23 pm

Re: Shapeing 10G of traffic

Wed Nov 14, 2018 6:05 pm

I would like to shape users on that CHR. Mangle based on IP and Queue tree. Is it possible?
 
User avatar
TomjNorthIdaho
Forum Guru
Forum Guru
Posts: 1492
Joined: Mon Oct 04, 2010 11:25 pm
Location: North Idaho
Contact:

Re: Shapeing 10G of traffic

Wed Nov 14, 2018 10:04 pm

I would like to shape users on that CHR. Mangle based on IP and Queue tree. Is it possible?
Mikrotik CHR routers (when installed on a good high-throughput hyper-visor) work well for traffic shaping on less than 4.294-Gig simple queue settings.
However - there is a BIG serious problem when shaping (simpe queue) faster than 4.294-Gig settings.
simple queue settings of 4294M or less work
simple queue settings of 4295M or more are rejected


North Idaho Tom Jones
 
pospanko
Member Candidate
Member Candidate
Topic Author
Posts: 279
Joined: Sun Dec 18, 2005 4:23 pm

Re: Shapeing 10G of traffic

Thu Nov 15, 2018 9:23 am

Is that total throughput or per queue? I would like to use simple queus or queue tree per user, up to 600Mbit...
What do you recomend for Hypervisor?
 
User avatar
TomjNorthIdaho
Forum Guru
Forum Guru
Posts: 1492
Joined: Mon Oct 04, 2010 11:25 pm
Location: North Idaho
Contact:

Re: Shapeing 10G of traffic

Thu Nov 15, 2018 10:30 pm

Is that total throughput or per queue? I would like to use simple queus or queue tree per user, up to 600Mbit...
What do you recomend for Hypervisor?
Re: … Is that total throughput …
I am saying (from my prior post) that simple queue settings of 4294M or less are accepted in the Mikrotik ROS configuration. Attempts to set a simple queue higher/faster than 4294M are rejected (the winbox entry just turns red when 4295M or faster is typed in.

As for the simple queue properly rate limiting at speeds of 4294M or slower , it works very well --- providing you have a physical box with 10-Gig physical network cards and you have the CPU horsepower.

My physical server is a SuperMicro running VmWare ESXi and the Mikrotik ROS CHR is using the VMXNET-3 network card.

FYI - re CHR throughput on a hypervisor ... On your CHR , do a UDP (send or receive) btest to 127.0.0.1 , you should be able to hit at least 15-Gig. A good box should hit at least 20-Gig. A screaming fast box should be close to 25-Gig or faster.


North Idaho Tom Jones
Last edited by TomjNorthIdaho on Fri Nov 16, 2018 6:50 pm, edited 1 time in total.
 
User avatar
vecernik87
Forum Veteran
Forum Veteran
Posts: 882
Joined: Fri Nov 10, 2017 8:19 am

Re: Shapeing 10G of traffic

Fri Nov 16, 2018 8:34 am

just about those "good" box speeds... My CHR running on HyperV (1 core 3.4GHz from i5-7500) was able to get around 4Gbps with some dummy firewall rules and 5Gbps with all rules disabled (interestingly, cpu was not maxed and profiler often showed less than 10% usage.. not sure why it didn't go faster then but there was obviously some packet drop involved)
CHR on amazon went much better despite the CPU is slower (1 core 2.4GHz), I got 12Gbps with firewall rules and 18Gbps without firewall rules. CPU was maxed as expected.
just for fun, I tried the same on hAP ac^2 - with firewall rules, it achieved blazing fast 0.8Gbps and up to 1.3Gbps without firewall rules
last test was on old RB951G which got 0.6Gbps with firewall and 1.3Gbps without firewall (actually, pretty interesting result)

(all tests were with 127.0.0.1, UDP, receive and with user/password)
 
User avatar
TomjNorthIdaho
Forum Guru
Forum Guru
Posts: 1492
Joined: Mon Oct 04, 2010 11:25 pm
Location: North Idaho
Contact:

Re: Shapeing 10G of traffic

Fri Nov 16, 2018 7:17 pm

Re "good" box speeds and btest to the loopback interface 127.0.0.1

There are pros and cons with this kind of btest. A btest only uses 1 CPU , although there could be more CPUs on the virtual or physical Mikrotik.
I use this test on all of my core Mikrotiks to test raw CPU throughput on a single core. I realize that some ROS processes might use multiple-threads (multi CPUs). However , I feel this is a very good test for Layer-3 processing throughput.

Some configurations in the Hypervisor settings can help a CHR. Example - disable Hyper-Threading. On the Hypervisor , try to keep at least 1 CPU available (don't overbook your CPUs to virtual machines). On the physical Hypervisor and on the virtual machines running under the Hypervisor , disable and remove all non-necessary devices such as serial ports , floppy drives, cd-roms). You want to keep your CPU interrupts to a minimum. Also , when possible , use ParaVirtual (Hypervisor optomized) drivers. Paravirtual drivers are always have faster throughput (aka vmxnet3 network devices and Paravirtual SCSI drivers).

O - and on my Hypervisor systems (VmWare ESXi) , I always change the default Delayed-Ack=0 to Delayed-Ack=1. Also , if you are running NFS, consider disabeling SYNC. These little tricks can get you from an original 300 Meg raw throughput to sometimes 6-Gig on continous sustained NFS read and writes.


Also , use an Intel Xeon processor with faster clock speeds (rather than more cores) , and select a Xeon with the most CPU cache you can get - big big big increase in CPU throughput on a Hypervisor system with just a few but fast systems running. Use 10-Gig physical network cards. (also disable Hyperthreading)

Another trick - when possible , avoid Mikrotik bridging (if possible use the hardware switch chip). Mikrotik ROS software bridging just chewes up CPU resources and slows things down because packets have to move through more interfaces. Exampe - A 16 port all 10-Gig Mikrotik software bridging all ports might only yeild just above 300 Meg throughput through the Mikrotik - and drive up the CPU load. Hardware switching using the switch chip has no effect on the CPU and normally results in wire-speed switching (way way totally faster).

North Idaho Tom Jones
 
edwinlai33
newbie
Posts: 28
Joined: Sat May 19, 2012 9:15 am

Re: Shapeing 10G of traffic

Sat Nov 17, 2018 11:43 am

the result of CHR btest is beautiful ,but it mean nothing
I try to replace our bare x86 server with CHR, and tested some CHR, all btest >19G.
but in my environment( 4-5 bgp, no queue, no connection tracking, some route mangle ), 1.5G traffic make CHR high packet drop, no problem in bare x86 server, the server is the same .
so I am trying the newest cpu I9 now, routeros seem not compatible with the latest cpu, The cpu-usage mismatch, but the traffic is very smooth without any packet drop.

viewtopic.php?f=2&t=141611

Who is online

Users browsing this forum: GoogleOther [Bot] and 75 guests