Page 1 of 1

Routing Question: Not able to get 1 gbps through our gateway with our router inline can without it

Posted: Fri Jul 12, 2019 6:09 am
by skynetcommky
We operate a WISP and have experienced growth pretty fast in our area. Each year having to upgrade our gateway fiber, first we had 100 mbps then 250 mbps then 500 mbps now we are at 1 gbps. Peak time hours we stay at 500 mbps. We just got our fiber upgrade completed today. I am able to move 945 mbps of bandwidth through the gateway device symmetrical until I place our router inline. I am trying to figure out if we need a better edge router or maybe if our collision domain is too large and overwhelming the switches. I will say this. After I was able to perform a speed test at 945/945 mbps connected right to the fiber gateway simply adding the router then plugging in my pc directly and only my pc to the router I can only get about 550 mbps of throughput, it seems the router is the bottleneck.

Our internal network is a flat layer 2 network / bridged. We have a single public ip on the wan interface of our router and a /24 public subnet on the lan side of our router that is routed through the wan ip on interface 1. If anyone has any insight on what I can try or what would help it would be greatly appreciated. If someone wants to consult on this I would gladly pay for some help as this network has grown past my expertise.

We are using a Mikrotik CCR1009-8G-1S-1S+ router and Netonix switches.

Re: Routing Question: Not able to get 1 gbps through our gateway with our router inline can without it

Posted: Fri Jul 12, 2019 2:00 pm
by R1CH
What kind of speed test are you doing? A single TCP connection will be limited by the CCR per-core frequency, but multiple connections should max out the link no problem. Test with iperf3 through the router for best results. Check profiler to see where load is.

Re: Routing Question: Not able to get 1 gbps through our gateway with our router inline can without it

Posted: Sat Jul 13, 2019 4:32 am
by skynetcommky
It's a multi thread test such as speedtest.net I have also worked with my upstream provider to setup a speedtest called truespeed this is a test from their data center to my gateway. Same poor results on both test. Hooked directly to their copper gateway port I get tcp throughput in the 940mbps area. My PC also sees 900 mbps range with speedtest.net I understand their is not routing this way. The iperf test I'll try that from my router through the switch and to my pc and see what I get. Thanks for the insight. I'll post back my results. I need a sfp connection from the router to my switch soon it seems.

Re: Routing Question: Not able to get 1 gbps through our gateway with our router inline can without it

Posted: Sat Jul 13, 2019 7:51 pm
by mducharme
It's a multi thread test such as speedtest.net I have also worked with my upstream provider to setup a speedtest called truespeed this is a test from their data center to my gateway. Same poor results on both test. Hooked directly to their copper gateway port I get tcp throughput in the 940mbps area. My PC also sees 900 mbps range with speedtest.net I understand their is not routing this way. The iperf test I'll try that from my router through the switch and to my pc and see what I get. Thanks for the insight. I'll post back my results. I need a sfp connection from the router to my switch soon it seems.
The important thing on the CCR router is that no single CPU is maxed out. Run Tools->Profile on all CPUs and sort the list by CPU. If you see any individual CPU going up higher than 80 or 90% you are likely hitting a CPU bottleneck, then it is a matter of determining which process is responsible.

Re: Routing Question: Not able to get 1 gbps through our gateway with our router inline can without it

Posted: Sun Jul 14, 2019 11:38 am
by cdiedrich
Since you stated that you're using a CCR1009-8G-1S-1S+ it is important to know which interfaces you are using for your backhaul and LAN side.
That one still has a switch chip which aggregates ether1-ether4 to a single 1G pipe to the CPU. If you have both interfaces in that port range, it might be a hint for the bottleneck.
Ether5...8 have dedicated 1G pipes.

Have a look at the block diagram of your device:
Image

-Chris