Community discussions

MUM Europe 2020
 
lrl
newbie
Topic Author
Posts: 25
Joined: Sat Sep 28, 2013 10:31 am

CCR issue

Sun Oct 19, 2014 11:04 pm

I have a weird one here. I’ve searched and can’t find anyone or any mention of this type of issues elsewhere, but if I’ve missed it please let me know.

I have a CCR1036 that I use as our Core router. Recently we’ve been noticing slowdowns everywhere. So I’ve ran some speed tests using the built in MT Bandwidth Test. I could only get 50-60 Mbps from the CCR to any other router, even a RB493G that’s directly connected via cat5 and linked at 1Gbps Full.

If I run a speed Bandwidth Test from the RB493G to the CCR1036 I can pull 140 Mbps (TCP), but the other way 58Mbps max.

From here I wasn’t sure I could trust the built in Bandwidth Test, so Loaded up iperf on a laptop and a server directly connected to the CCR1036. Same result, if I go through the CCR1036 I get 50-60Mbps.

I’ve checked Queues to ensure they aren’t the culprit. Checked firewall rules, mangle, queue tree, etc.

I’m at a lost.

The CCR is running 6.18 The firmware is 3.09 but it looks like I can upgrade to 3.18. I’m unable to find a change log for the firmware, and I’m unsure rather that would have any effect on this. I’m also considering upgrading to 6.20. Unfortunately with this being our core router I can’t just run these upgrades. I’m hoping someone else know what’s change on the firmware side and might have a better idea if it would solve this.

Any help, ideas, suggestions on this would be appreciated!

-Lance
 
sonny
Member Candidate
Member Candidate
Posts: 208
Joined: Fri Jan 28, 2005 5:14 pm
Location: Germany
Contact:

Re: CCR issue

Sun Oct 19, 2014 11:36 pm

A true Bandwith test can only run through the routers.
Then you can expect nearly the most possible bandwith
When you test from router to router the CPU is the limiting problem espacially TCP

e.g.

CCR1 or PC with IPERF ->Gigabit link <-CCR-test -> Gigabit link<-RBXXX -> Gigabit link <- CCR2 or PC with IPERF
Karl Sonnleitner
Senior Wireless-Expert
Restlesspowerbox - managed powersupply for Routerboards
 
lrl
newbie
Topic Author
Posts: 25
Joined: Sat Sep 28, 2013 10:31 am

Re: CCR issue

Mon Oct 20, 2014 12:20 am

Correct. That's why I loaded iperf on a server and my laptop and conducted another test.

<Laptop> <eth3 CCR> (software bridge) <eth7 CCR> <Server>

At the time of the test the total bridge throughput was 120Mbps
 
dottxt
just joined
Posts: 15
Joined: Sun Feb 02, 2014 5:53 pm

Re: CCR issue

Mon Oct 20, 2014 7:08 am

Is the CPU on the CCR maxed out during the tests?

Also, in Winbox, under Interfaces, add the columns Tx Drops, Rx Drops, Tx Errors, and Rx Errors if they aren't there already. During your testing, check to make sure these are all holding 0.

Let us know how it goes.
 
lrl
newbie
Topic Author
Posts: 25
Joined: Sat Sep 28, 2013 10:31 am

Re: CCR issue

Mon Oct 20, 2014 7:52 am

Zeros all across. The CPU on the CCR is holding 2-3%. A bandwidth test from the CCR to another router doesn't increment the CPU more than 1% overall and usually just one core raises to 40-50% which in my mind rules out traffic rate being limited by CPU. The receiving router reaches about 65% CPU.

I'm starting to think flow control or something along those lines. However I have flow control turned off on the interfaces. I half wonder if pause frames aren't seeping in from elsewhere on the bridge.

I have a scheduled maintenance window planned for the wee hours to upgrade to 6.20 and update the firmware. I will update after the window.
 
lrl
newbie
Topic Author
Posts: 25
Joined: Sat Sep 28, 2013 10:31 am

Re: CCR issue

Fri Oct 24, 2014 9:05 am

The upgrades had no effect.

The problem appeared to be a weird flow control issue. I had previously turned flow control off because of some lockup issues we had with a ubiquiti switch. I had also turned flow control off on the CCR. But apparently the ubiquiti switch misbehaves even with flow control turned off and still controls the flow. I finally turned flow control on for the CCR interface to the ubnt switch and on within the ubnt switch for the interface facing the CCR and all my throughput issues went away.

I believe what I was seeing was a ubiquiti bug. I do however wonder why it was effecting traffic that didn't or shouldn't have flowed through the ubnt switch.

-LRL
 
ffrauter
just joined
Posts: 17
Joined: Mon May 16, 2011 12:43 pm

Re: CCR issue

Wed Nov 05, 2014 11:36 am

Hello lrl!

I have nearly the same problem with CCR1036 (ROS 6.20) with bridging performance. (aprox 150 Mbit) You wrote, that you could fix it with activating flow control on Interface to ubnt switch.

Where is this Swich in your test scenario, when you testet the following:

Iperf Server < --> CCR1036 <--> Iper Client

Did you have the same issue here?

Thanks for your reply.
 
SystemErrorMessage
Member
Member
Posts: 378
Joined: Sat Dec 22, 2012 9:04 pm

Re: CCR issue

Wed Nov 05, 2014 12:28 pm

run the mikrotik bandwidth test from your PC and at least 10 active sessions at the same time. The problem with the CCR is that each connection/link is only given up to 1 core to run.

I use a CCR1036 with bridge and doing a bunch of other stuff but i realised that to get full performance you need many connections/links.
 
lrl
newbie
Topic Author
Posts: 25
Joined: Sat Sep 28, 2013 10:31 am

Re: CCR issue

Wed Nov 05, 2014 8:49 pm

The internal bandwidth test and the one core per session could affect this, but in my testing I was still able to push into the mid 200's before I reached the limit imposed by one core.

In my situation the UBNT switch was off of int3 and the IPERF server off of int4 and the IPERF client off of int6. I'm not sure why the ubnt switch was causing these issues when it wasn't involved in the test traffic. My hypothesis is the pause frames were passing through the CCR bridge and causing the interfaces on the server or client to pause. Because the nic in the server didn't allow flow control options it was the unknown.

It's a weird one that ultimately got down to a finger pointing game.

Who is online

Users browsing this forum: Google [Bot], MSN [Bot] and 55 guests