I am using RB3011 with two ISPs 140mbps + 105mbps. Router was running 7.1 stable and now 7.1.1 stable. However, with just 245mbps of load balancing the CPU usage increases to 75-85%. Earlier in v6 it was 35-45% maximum. Kindly check my PCC configuration and point out if any of my mangle rules are creating excess stress on the cpu and if any alternate syntax can be used to reduce the load. Also if I want to use fastrack then where should I place the rules for it?
[kalpak@home-lab] /ip/firewall/mangle> print
Flags: X - disabled, I - invalid; D - dynamic
0 chain=prerouting action=mark-connection new-connection-mark=exclude_conn passthrough=yes src-address-list=own-lan dst-address-list=bypass-list log=no log-prefix=“”
1 chain=prerouting action=mark-routing new-routing-mark=main passthrough=yes connection-mark=exclude_conn log=no log-prefix=“”
2 chain=input action=mark-connection new-connection-mark=abspl_out_conn passthrough=yes in-interface=ether2
3 chain=output action=mark-routing new-routing-mark=abspl-inbound passthrough=no connection-mark=abspl_out_conn
4 chain=input action=mark-connection new-connection-mark=sswl_out_conn passthrough=yes in-interface=ether4
5 chain=output action=mark-routing new-routing-mark=sswl-inbound passthrough=no connection-mark=sswl_out_conn
6 chain=forward action=mark-connection new-connection-mark=abspl_out_pfw passthrough=no connection-state=new in-interface=ether2
7 chain=prerouting action=mark-routing new-routing-mark=abspl-inbound passthrough=no connection-mark=abspl_out_pfw in-interface-list=lans
8 chain=forward action=mark-connection new-connection-mark=sswl_out_pfw passthrough=no connection-state=new in-interface=ether4
9 chain=prerouting action=mark-routing new-routing-mark=sswl-inbound passthrough=no connection-mark=sswl_out_pfw in-interface-list=lans
10 chain=prerouting action=accept in-interface=ether2
11 chain=prerouting action=accept in-interface=ether4
12 chain=prerouting action=mark-connection new-connection-mark=abspl_conn passthrough=yes dst-address-type=!local connection-mark=!exclude_conn per-connection-classifier=both-addresses-and-ports:2/0 log=no log-prefix=“”
13 chain=prerouting action=mark-connection new-connection-mark=sswl_conn passthrough=yes dst-address-type=!local connection-mark=!exclude_conn per-connection-classifier=both-addresses-and-ports:2/1 log=no log-prefix=“”
14 chain=prerouting action=mark-routing new-routing-mark=abspl-outbound passthrough=yes connection-mark=abspl_conn in-interface-list=lans
15 chain=prerouting action=mark-routing new-routing-mark=sswl-outbound passthrough=yes connection-mark=sswl_conn in-interface-list=lans
Are you saying that if you disable the load distribution rules completely, the CPU load becomes lower by an order of magnitude?
Second, unless something has dramatically changed about the essence of fasttracking in ROS 7, packets belonging to fasttracked connections bypass mangle rules (and other processing steps not relevant here). So to have fasttracking and load distribution work together, you have to allow only connections via the highest bandwidth WAN to ever get fasttracked.
Yes!!! it is lot less like with 140mbps it is 20-30% max.
If I enable fasttrack rules then all packets simply bypass the mangle and therefore the PCC loadbalancing.
That sounds as if disabling the mangle rules was disabling something else (connection tracking?) as well, so post your complete configuration (hide-sensitive is not necessary any more in ROS 7, the rest of my automatic signature remains valid).
This is correct. That’s why I say you have to allow only connections running via the highest bandwidth uplink to get fasttracked. To do that, assign some connection-mark to the connections leaving via other WANs, and add a connection-mark=no-mark match condition to the action=fasttrack-connection rule.
RB5009: 2 wan 1gb+ 500mb pcc about 80% cpu use.
As expected - when the mangle rules are disabled, connection tracking is deactivated because enabled under ip/firewall/connection/tracking is set to the default value auto, and therefore connection tracking is only activated if at least one firewall rule refers to metafields that connection tracking provides or assigns them (i.e. anything related to matching or assigning connection-mark, connection-state, or connection-nat-state).
So first of all, keep the mangle rules disabled, and set enabled under ip/firewall/connection/tracking to yes. This will show you the CPU usage caused by connection tracking alone.
Your mangle rules may be simplified a bit (you seem to assign a connection mark to every packet of a connection), but I’m afraid the biggest difference in CPU load is caused by the activation of connection tracking.
My understanding is that there is no “Route Caching” in ROS7, hence the jump in CPU utilization from V6 to V7.
This post is getting confusing. Seems like V7 has made some changes under the hood that need to be better explained.
It’s because these are two separate issues.
One is the impact of connection tracking being activated by the addition of the load balancing rules (something you normally do not notice if the default stateful firewall is in place, which is not the case here), and this one is common to v6 and v7.
The other one is the impact of the absence of routing cache in v7, which has been mentioned multiple times here and there, but does not explain the difference in this topic.
I am using CHR v7.2 RC1 with AMD athlon 200GE and utilization is 12-17% max with the same set of rules.
The other one is the impact of the absence of routing cache in v7, which has been mentioned multiple times here and there, but does not explain the difference in this topic.
Then why is ip route cache option present in ip>setting?
with v7.2RC3 the cpu utilization has reduced to 55-60% from 86%+ previously. However, a lot of work in optimizing throughput is required. I cannot get more that 270-300mbps on file transfer between two vlans.
They haven’t removed the setting yet. It is now a dummy setting, the only thing it does is disables fast path, and there is already a check box for disabling fast path.
We have noticed the same thing on CPU usage, we just upgraded a NAT firewall that does PCC load balancing from 6.48.9 to 7.11.2 yesterday and you can see the huge difference in CPU usage. I would have thought with the newer OS it would be dramatically lower not signifigantly highter. This is on a CCR2004 router runnning about 2Gps at peak time.
CPU Usage.pdf (83.3 KB)
In RouterOS there is no routing cache anymore. That might have a impact in a high load setting.
Once again, that’s a myth and misconception spread on this forum. The current V7 kernel utilizes a more modern network stack that divides the cache into distinct layers, achieving greater efficiency where it’s most needed.
Some relevant reading on the subject:
Thanks for the response. Is there anything I can do to lower the CPU usage to achieve more than 2Gbps on my CCR2004 in real world scenario? Is there anyway to diagnose and tell if certain mangle or firewall rules are the cause of the slow down or inefficient? If you look at the cpu profile is only shows generic cpu usage for networking or firewall but no details.