Dear All,

I am having CCR1036-12G-4S on our core side, as most of the clients are from core side on Fiber Optic Network. Now one of our remote Wireless site that was serving 25 Mbps there for about 5 clients we installed RB2011UiAS but now that site load is increasing and currently reached to 63Mbps and 11 clients. When all the clients comes online the CPU usage of 2011 goes high to 75% + and the quality of services goes down.
Should we install another CCR series router over there or can we use RB750 in bridge mode over there to bring all the traffic to CCR at core side. Can we also use CISCO managed switch like 2950 and trunk it back to CCR ?
I will soon post the network diagram to make it clear to all the readers, but in the meantime EXPERTS opinion will be highly appreciable.

Thanks in Advance.

Here is the Network Diagram for the reference.

Assuming you have some sort of tunnel between the two sites, it sounds like the RB2011 is becoming a bottleneck for you. I would switch to a CCR1009, especially the new ones. A CCR1009 will blow the doors off of a 2011 any day of the week.

Introducing an RB750 doesn't really help you. If the 2011 is struggling, the RB750 will even more so.

Assuming you have some sort of tunnel between the two sites, it sounds like the RB2011 is becoming a bottleneck for you. I would switch to a CCR1009, especially the new ones. A CCR1009 will blow the doors off of a 2011 any day of the week.

Introducing an RB750 doesn't really help you. If the 2011 is struggling, the RB750 will even more so.

One of fellow suggested to make a bridge and divert all the load to the CCR because it is not a good solution to install CCR series of high end routers at every site.

Sent from my Z12 using Tapatalk

Something you might try -
Increase the CPU clock speed to the 2nd fastest clock speed available in all of your Mikrotiks.
If it is already at the fastest, then leave it there.
The reason I state the 2nd to fastest (if you do increase) is that sometimes/often, overclocking to the fastest clock speed can result in in-reliable operation.

I am pretty sure the official Mikrotik stance would be to never overclock but instead upgrade hardware.

Other things you can do/try include using the Ethernet switch chip instead of software bidging and simplyfing the configurations.

North Idaho Tom Jones

@normis can you please give your input.

Sent from my Z12 using Tapatalk

750Gr3 is faster than 2011. Maybe just better configuration can help you too. At least it is not clear what occupies the cpu or where the bottleneck is...

750Gr3 is faster than 2011. Maybe just better configuration can help you too. At least it is not clear what occupies the cpu or where the bottleneck is...

Queues and firewall consumes the CPU. I have disabled all the rules in the firewall then only Queues dont let it down.

Sent from my Z12 using Tapatalk

Can we use Cisco managed switch at remote site skipping the routerboard and linking to the main CCR1036 via trunk port ?

Sent from my SM-N910T using Tapatalk

a 2011-UiAS should be able to cope with 80Mbps and 20+ pppoe user along with simple queues, looks "tight" but doable for me; sounds like your config is not optimized for the task, probably in the mangle and filter firewall areas.

Equally important is determining the health of the downlink. If router needs to constantly resend packets (corruption, out of order, packet loss, fragmentation) its limited CPU will be overloaded.

When speaking about optimizations I mean think the trip a data packet goes along the router, and try to make its journey as straightforward as possible.

Posting the config will help in that regard to provide suggestions...

Your approach (remote POP PPPoE termination) makes more sense when there are lots of local users, as you can restrict speeds and apply QoS at origin (as should be); that usually means beefier routers, which is usually dictated by the number of pppoe sessions, but it doesn't need to be CCRs specifically.

That being said, in your scenario is common and best practice, when downlinks aren't a bottleneck, to bring all L2 traffic using MPLS/VPLS back to your BRAS.

This along with optimizations makes the requirements for remote POPs much, much lower, a humble "old" 750 was able to keep wirespeeds (100Mbps) while doing OSPF and MPLS/VPLS with ease.

Re: ... bring all L2 traffic using MPLS/VPLS back to your BRAS ...

My thoughts and what I do ..

I have thousands of networks (most on wireless Mikrotik and many on fiber). Most of my networks consist of three things:
one - put the customer LAN nat devices at the customer location
two - Layer 2 backhaul all customer WANs back to a core central location
three - perform all rate-limiting bandwidth management for all customers back at my core central location.

Although there are many methods to rate-limit individual customer bandwidths, I generally use a captive-portal walled-garden on PfSense and manage bandwidths using the WAN Layer-2 MAC address with PfSense & Radius. I currently Layer-2 backhaul over 2,000 WAN networks.

In general, I prefer a "keep-it-simple" and "do what ever it takes to avoid CPU loads and network congestion. On my busiest wireless (NV2 with 30-to-50+ clients) Mikrotik networks, I hardly ever even see a 10 percent load on anything anywhere in the network.

Also - FYI ... Using a Captive-Portal Walled-Garden, allows me the ability to disable customer Internet access and auto-redirect customer web traffic to one of my web servers which then shows a page stating something like "Your Internet connection is working - however your account has been temporarily suspended. Please call our office at ###-###-####". Unlike disabling a pppoe customer account for a late customer payment which often results in the customer thinking their network is broken, I am able to notify customers instantly and they call us rather quickly to pay/re-activate their accounts.

North Idaho Tom Jones

a 2011-UiAS should be able to cope with 80Mbps and 20+ pppoe user along with simple queues, looks "tight" but doable for me; sounds like your config is not optimized for the task, probably in the mangle and filter firewall areas.

Equally important is determining the health of the downlink. If router needs to constantly resend packets (corruption, out of order, packet loss, fragmentation) its limited CPU will be overloaded.

When speaking about optimizations I mean think the trip a data packet goes along the router, and try to make its journey as straightforward as possible.

Posting the config will help in that regard to provide suggestions...

Your approach (remote POP PPPoE termination) makes more sense when there are lots of local users, as you can restrict speeds and apply QoS at origin (as should be); that usually means beefier routers, which is usually dictated by the number of pppoe sessions, but it doesn't need to be CCRs specifically.

That being said, in your scenario is common and best practice, when downlinks aren't a bottleneck, to bring all L2 traffic using MPLS/VPLS back to your BRAS.

This along with optimizations makes the requirements for remote POPs much, much lower, a humble "old" 750 was able to keep wirespeeds (100Mbps) while doing OSPF and MPLS/VPLS with ease.

Pukkita, thanks for your valuable suggestions, but in my case it is straight PTP link with WDS enabled. The wireless link is established using UBNT Powerbeam M5 ISO having Gigabit interface and the link is established at 300/300 Mbps. Speedtest between both devices shows 240 Mbps tx/rx approx, so the link is not an issue.
At RB 2011, there is no complicated configuration, simple PPPoE server is enabled, I dont think that in this straight forward connectivity we should think about MPLS. What do you say ?

Sent from my SM-N910T using Tapatalk

Pukkita, thanks for your valuable suggestions, but in my case it is straight PTP link with WDS enabled. The wireless link is established using UBNT Powerbeam M5 ISO having Gigabit interface and the link is established at 300/300 Mbps. Speedtest between both devices shows 240 Mbps tx/rx approx, so the link is not an issue.
At RB 2011, there is no complicated configuration, simple PPPoE server is enabled, I dont think that in this straight forward connectivity we should think about MPLS. What do you say ?

Sent from my SM-N910T using Tapatalk

If you keep the pppoe server on it, there's no point on extending L2 via MPLS, right. Unless you may want to have a backup way to send the users to another, uplink, PPPoE AC.

Regarding speedtest, that means nothing in terms of downlink quality. I refer to use a tool like smokeping to see latency, packet loss and link health, the probe should be done from one side of the link to the 2011, so that traffic traverses all the link.

To extract the maximum performance on a system, each 5% counts, and those 5% are scattered around on different areas, link quality (packet loss, jitter, reordering, fragmentation) is key.

This is just to make sure the link is optimal, do not trust whichever appears in ubnt main, but actually probe it, is it needed to load test the link and measure quality.

The most relevant config tidbits that may impact CPU is firewall, but all the config should be taken into account as a whole, I don't have crystal ball so cannot point to where your suboptimal settings are

Pukkita, thanks for your valuable suggestions, but in my case it is straight PTP link with WDS enabled. The wireless link is established using UBNT Powerbeam M5 ISO having Gigabit interface and the link is established at 300/300 Mbps. Speedtest between both devices shows 240 Mbps tx/rx approx, so the link is not an issue.
At RB 2011, there is no complicated configuration, simple PPPoE server is enabled, I dont think that in this straight forward connectivity we should think about MPLS. What do you say ?

Sent from my SM-N910T using Tapatalk

If you keep the pppoe server on it, there's no point on extending L2 via MPLS, right. Unless you may want to have a backup way to send the users to another, uplink, PPPoE AC.

Regarding speedtest, that means nothing in terms of downlink quality. I refer to use a tool like smokeping to see latency, packet loss and link health, the probe should be done from one side of the link to the 2011, so that traffic traverses all the link.

To extract the maximum performance on a system, each 5% counts, and those 5% are scattered around on different areas, link quality (packet loss, jitter, reordering, fragmentation) is key.

This is just to make sure the link is optimal, do not trust whichever appears in ubnt main, but actually probe it, is it needed to load test the link and measure quality.

The most relevant config tidbits that may impact CPU is firewall, but all the config should be taken into account as a whole, I don't have crystal ball so cannot point to where your suboptimal settings are

Thanks Pukkita, well do we have other options for remote site ? Means, isnt it possible to skip mikrotik routers at remote sites, and simply put Cisco managed switch, that further control the connectivity?
Making few interfaces in bridge, isnt it helpful? Because I tried to make 2011 in bridge mode and its CPU usage drops to 10% for the same 80Mbps bandwidth. But was having some DNS issues, hopefully they will be solved as well.

Sent from my SM-N910T using Tapatalk

I'm afraid you think in a "NOC way", possibly because that's your background, SMB networks or NOCs?

WISPs, or wireless networks, are different beasts.

No, I wouldn't bridge it up to the core with a cisco switch, no matter if using VLANs or not, the reason: broadcast domain.

Also - FYI ... Using a Captive-Portal Walled-Garden, allows me the ability to disable customer Internet access and auto-redirect customer web traffic to one of my web servers which then shows a page stating something like "Your Internet connection is working - however your account has been temporarily suspended. Please call our office at ###-###-####". Unlike disabling a pppoe customer account for a late customer payment which often results in the customer thinking their network is broken, I am able to notify customers instantly and they call us rather quickly to pay/re-activate their accounts.

North Idaho Tom Jones

We want to do the same , it would be great help ,if you guide us.

Thanks Shailendra

Sent from my MI MAX using Tapatalk

Have a look at https://wiki.mikrotik.com/wiki/Payment_Reminders it illustrates the approach; you can either use adress-lists, or simply make the non-paying customers to get a different ip pool via radius so that you have a source to redirect to the proxy (or apply more restrictive bandwidth limits for example as an initial courtesy towards non-paying customers).