Community discussions

 
User avatar
bigcw
Member Candidate
Member Candidate
Topic Author
Posts: 105
Joined: Mon Sep 08, 2014 2:38 pm

CCR1009 maxes out at 2gbps?

Sat Dec 09, 2017 2:55 pm

Using a CCR1009-7G-1C-1S+ in a large NAT scenario. Basically a big version of a home router with around 5,000 subscribers behind it. DHCP, DNS, etc all handled elsewhere, the CCR just does the NAT.

It has a /27 of public IPs on it and NAT rules to spread the subscribers between those to avoid running out of ports.

10G uplink to a CRS317 onto which are VLANs feeding the subscribers and the IP transit feed.

The CCR seems to max out at 2.1gbps. Flat line on the traffic graph at peak times.

There is plenty of capacity in the network presenting the subscribers to the router. It's all 10G in fact to the aggregation switches so no contention issues there. Also plenty of headroom in the upstream network. The CCR's CPU is at 45% and load is spread evenly over the cores so it's not something like the NAT engine only being single threaded.

Is there any reason why this might be happening? License issue perhaps? Or the CCR running out of capacity? (I would have assumed if it was the latter the CPU load would be close to 100%)

Anyone got any tips?

Chris
Ecom International Network - Operators of AS61337 with POPs in Europe and North America - www.ecomltd.co.uk
Colocker Data Centre - The data centre with a difference! - www.colocker.com
 
InoX
Forum Guru
Forum Guru
Posts: 1962
Joined: Tue Jan 09, 2007 6:44 pm

Re: CCR1009 maxes out at 2gbps?

Sat Dec 09, 2017 10:38 pm

So the wan is sfp+ , how many LAN ports are you using for client traffic? If you are using 2 ports=2gbps
 
pe1chl
Forum Guru
Forum Guru
Posts: 5523
Joined: Mon Jun 08, 2015 12:09 pm

Re: CCR1009 maxes out at 2gbps?

Sun Dec 10, 2017 12:47 am

So the wan is sfp+ , how many LAN ports are you using for client traffic? If you are using 2 ports=2gbps
No, it looks like both the WAN and the customers are on the SFP+ on different VLANs.

I have no idea what the limits are, I run CCR1009 only up to 1Gbps and without NAT... but with lots of other filter and mangle rules.
Of course he could get a CCR1036 instead.
 
User avatar
bigcw
Member Candidate
Member Candidate
Topic Author
Posts: 105
Joined: Mon Sep 08, 2014 2:38 pm

Re: CCR1009 maxes out at 2gbps?

Mon Dec 11, 2017 11:24 am

So the wan is sfp+ , how many LAN ports are you using for client traffic? If you are using 2 ports=2gbps
Everything is on the SFP+. It is done with two VLANs; clients in one and WAN in the other.
Of course he could get a CCR1036 instead.
Here's the thing: the CPU is nowhere near max'ed out on the 1009 so will a 1036 actually fix the problem? OK so I could bring the clients in on a separate SFP+ to the WAN but I am not sure that gains me anything.

Chris
Ecom International Network - Operators of AS61337 with POPs in Europe and North America - www.ecomltd.co.uk
Colocker Data Centre - The data centre with a difference! - www.colocker.com
 
User avatar
bigcw
Member Candidate
Member Candidate
Topic Author
Posts: 105
Joined: Mon Sep 08, 2014 2:38 pm

Re: CCR1009 maxes out at 2gbps?

Mon Dec 11, 2017 11:41 am

Some data collected last night. Note that a tweak in the client access network has allowed us to squeeze a bit more performance out of it (around 2.8gbps max) but there is definitely a 'flat top' to the graph which suggests to me that something is being pushed to it's limit somewhere in the network.
You do not have the required permissions to view the files attached to this post.
Ecom International Network - Operators of AS61337 with POPs in Europe and North America - www.ecomltd.co.uk
Colocker Data Centre - The data centre with a difference! - www.colocker.com
 
pe1chl
Forum Guru
Forum Guru
Posts: 5523
Joined: Mon Jun 08, 2015 12:09 pm

Re: CCR1009 maxes out at 2gbps?

Mon Dec 11, 2017 11:43 am

I'm not sure either... I don't know what is the limiting factor, I never drive the CCR's to saturation because my internet links are too slow for that.
45% load is a figure I have never seen on a CCR I run... but with NAT my guess would be that the limit would be a mix between the amount of traffic and the number of new sessions per second, so there would not be a fixed capacity cap in Gbps.
Maybe there are others who have experience with a setup like that.

Edit: you made a post while I typed that and the capacity cap is not as flat as I guessed.
 
User avatar
bigcw
Member Candidate
Member Candidate
Topic Author
Posts: 105
Joined: Mon Sep 08, 2014 2:38 pm

Re: CCR1009 maxes out at 2gbps?

Mon Dec 11, 2017 12:32 pm

Mikrotik support: is there any way of seeing how many NAT sessions are open?

Chris
Ecom International Network - Operators of AS61337 with POPs in Europe and North America - www.ecomltd.co.uk
Colocker Data Centre - The data centre with a difference! - www.colocker.com
 
pe1chl
Forum Guru
Forum Guru
Posts: 5523
Joined: Mon Jun 08, 2015 12:09 pm

Re: CCR1009 maxes out at 2gbps?

Mon Dec 11, 2017 12:57 pm

This forum is not a way to ask questions to MikroTik Support.
For that you need to mail support@mikrotik.com
 
User avatar
cdiedrich
Forum Veteran
Forum Veteran
Posts: 892
Joined: Thu Feb 13, 2014 2:03 pm
Location: Basel, Switzerland // Bremen, Germany
Contact:

Re: CCR1009 maxes out at 2gbps?

Mon Dec 11, 2017 1:18 pm

Mikrotik support: is there any way of seeing how many NAT sessions are open?

Chris
/ip firewall connections print count-only where srcnat=yes
When you omit the "count-only" parameter, you will see all connections in a list.

-Chris
Christopher Diedrich
MTCNA, MTCUME, MTCWE
Basel, Switzerland
Bremen, Germany

There are 10 types of people: Those who understand binary and those who don't.
There are two types of people: Those who can extrapolate from incomplete data
 
User avatar
bigcw
Member Candidate
Member Candidate
Topic Author
Posts: 105
Joined: Mon Sep 08, 2014 2:38 pm

Re: CCR1009 maxes out at 2gbps?

Fri Dec 15, 2017 12:26 pm

/ip firewall connections print count-only where srcnat=yes
When you omit the "count-only" parameter, you will see all connections in a list.
Thanks for that, Chris

We replaced the 1009 with a 1036 2S+ yesterday afternoon. CPU load already massively reduced (as expected as it has 3x as many cores!). Hopefully find out tonight whether there is any improvement in maximum throughput.
graph_image.png
graph_image (1).png
You do not have the required permissions to view the files attached to this post.
Ecom International Network - Operators of AS61337 with POPs in Europe and North America - www.ecomltd.co.uk
Colocker Data Centre - The data centre with a difference! - www.colocker.com
 
pe1chl
Forum Guru
Forum Guru
Posts: 5523
Joined: Mon Jun 08, 2015 12:09 pm

Re: CCR1009 maxes out at 2gbps?

Fri Dec 15, 2017 4:35 pm

Looks good!
Did you also separate the input and output over the 2 different SFP+ or is it still using a VLAN setup?
 
InoX
Forum Guru
Forum Guru
Posts: 1962
Joined: Tue Jan 09, 2007 6:44 pm

Re: CCR1009 maxes out at 2gbps?

Sun Dec 17, 2017 12:29 am

SFP+ ports 1
 
User avatar
bigcw
Member Candidate
Member Candidate
Topic Author
Posts: 105
Joined: Mon Sep 08, 2014 2:38 pm

Re: CCR1009 maxes out at 2gbps?

Wed Dec 20, 2017 9:00 pm

Looks good!
Did you also separate the input and output over the 2 different SFP+ or is it still using a VLAN setup?
Yes, we now have WAN and LAN on separate SFP+. Not seeing any improvement so far unfortunately.
Ecom International Network - Operators of AS61337 with POPs in Europe and North America - www.ecomltd.co.uk
Colocker Data Centre - The data centre with a difference! - www.colocker.com
 
kujo
Member Candidate
Member Candidate
Posts: 158
Joined: Sat Jun 18, 2016 10:17 am
Location: Ukraine

Re: CCR1009 maxes out at 2gbps?

Wed Dec 20, 2017 10:34 pm

Attach output, please:
/tool profile cpu=all


Yours respectfully!
 
aholmes
just joined
Posts: 19
Joined: Wed May 17, 2017 6:16 am

Re: CCR1009 maxes out at 2gbps?

Thu Dec 21, 2017 7:31 am

You might be capping out one or two CPU cores per-VLAN. Maybe a good idea to check that your interface queues are set to multi-queue-ethernet-default?
 
n21roadie
Forum Guru
Forum Guru
Posts: 1886
Joined: Fri Aug 07, 2009 10:36 pm
Location: Limerick,Ireland

Re: CCR1009 maxes out at 2gbps?

Thu Dec 21, 2017 12:50 pm

You might be capping out one or two CPU cores per-VLAN. Maybe a good idea to check that your interface queues are set to multi-queue-ethernet-default?
I note on my CCR1009 is that VLAN's queue interface has "no-queue" and so in queues clients pppoe interface have "no-queue", must compare this evening if "multi-queue-ethernet-default" will make any difference to performance?
 
pe1chl
Forum Guru
Forum Guru
Posts: 5523
Joined: Mon Jun 08, 2015 12:09 pm

Re: CCR1009 maxes out at 2gbps?

Thu Dec 21, 2017 2:17 pm

Those are not really interfaces like ethernet that have a hardware queue, an optional software queue, and an interrupt handler that moves packets between them.
VLAN and PPPoE are just putting on a header and moving data to the next layer (normally ethernet), and so they do not require a queue by themselves.
 
pospanko
Member Candidate
Member Candidate
Posts: 272
Joined: Sun Dec 18, 2005 4:23 pm

Re: CCR1009 maxes out at 2gbps?

Tue Dec 26, 2017 10:48 pm

Do you maybe use bridge?
We have dramatic falling apart at 2Gbps on CCR1072 using RSTP. When we set everything to STP traffic started to run normaly. Problem was that hosts table was falling apart, losing records. Of course, no support response from MT...
Internet, Mikrotik & Network solutions
http://www.pro-ping.hr
 
User avatar
Alessio Garavano
Member
Member
Posts: 304
Joined: Sat May 29, 2004 12:49 am
Location: Corrientes, Argentina
Contact:

Re: CCR1009 maxes out at 2gbps?

Fri Jul 05, 2019 5:11 am

Hi @Bigcw, you have a solution for this problem? i have the same problem in a CCR1036 with Mikrotik SFP+ connected to our carrier... we have 3Gbps but at 2,2Gbps is limited and all disabled(firewall, mangles, etc etc)!
Alessio Garavano
http://www.isparg.com.ar
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 24042
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: CCR1009 maxes out at 2gbps?

Fri Jul 05, 2019 9:23 am

I maybe don't understand something, but the CCR1009-7G-1C-1S+ only has ONE (1) SFP+ port.
No answer to your question? How to write posts
 
pe1chl
Forum Guru
Forum Guru
Posts: 5523
Joined: Mon Jun 08, 2015 12:09 pm

Re: CCR1009 maxes out at 2gbps?

Fri Jul 05, 2019 10:44 am

I maybe don't understand something, but the CCR1009-7G-1C-1S+ only has ONE (1) SFP+ port.
Halfway he changed to a CCR1036 to try to solve the problem...
 
User avatar
Alessio Garavano
Member
Member
Posts: 304
Joined: Sat May 29, 2004 12:49 am
Location: Corrientes, Argentina
Contact:

Re: CCR1009 maxes out at 2gbps?

Thu Jul 11, 2019 7:17 pm

I maybe don't understand something, but the CCR1009-7G-1C-1S+ only has ONE (1) SFP+ port.
Halfway he changed to a CCR1036 to try to solve the problem...
and? was solved? we have a CCR1036 doing the same problem...
Alessio Garavano
http://www.isparg.com.ar
 
maxsaf
Frequent Visitor
Frequent Visitor
Posts: 57
Joined: Tue Mar 06, 2018 8:47 pm

Re: CCR1009 maxes out at 2gbps?

Fri Jul 12, 2019 9:52 pm

and? was solved? we have a CCR1036 doing the same problem...
A had same problem doing bandwidth test on 1009. Solved by changing MTU to something over 8000

Who is online

Users browsing this forum: No registered users and 61 guests