I installed ccr 1036-8g-2s+, a brief explanation of config :
2 bgp peers + 6 filters
2 isp (ether 7 &
2 lan outs with one lan having 10 vlans (ether 4 & 5)
41 queues
3 rules for ssh bruteforcing protection 1 rule for packet marking.
all other ports are free
Bandwidth running 800mbps download and 150mbps upload. Router cpu consumption had reached 25% approx. I have few questions as ccr 1072 is not going to be launched soon. They are :
Why such a drastic difference in benchmarks of MT and practical test
What h/w is suggested if I need 5gig of routing (if x86 then which hw)
already ordered 5 ccr-1036
No major stability issue found apart from vlan packet drop (which got solved).
BGP filter behaving strangely (posted somewhere already in bgp section).
This router is already running from 80days. 35days idle, 35days 300mbps and from 14 days full load.
some part of ROS, on which you config relying on so much - obbviously had scalability issues, as you noticed thats 25% CPU load was peak.
as for simple answer - CCR1072 ?
you should profile and see what is most consuming. At the moment I run about 10 queues, 2 port bonding with 5 BGP peers (1 full table + IX tables, for IPv4 and IPv6) with about 500-600mbit aggregate peak (100k-130k pps), sits at about 15%. There are 2 packet marking rules per queue set up and also about 20 filter rules.
A big difference may be your packet rate causing higher CPU.
If Queuing is the big performance hit, try ROS6.19 if you haven’t, meant to be big improvements. If queueing starts to get too much might be an idea to split out the tasks, assign some CCR’s to the edge for BGP and have the queuing routers not having to do big routing lookups etc, so you might be able to get fastpath routing or the like in.
You say you only have 3 or 4 rules for your firewall, maybe do a /ip firewall export (so filtering, NAT and Mangle) so we can have a look - it might be doing more work than it needs to as the firewall shouldn’t be taking so much. My Firewall (with 19 filter rules currently) is usually 3rd or 4th in process list. routing, networking and queuing usually the top.
5 rules to be precise. 4 rules for bruteforcing of telnet and ssh as per wiki
1 rule for blocking dst address list which has 142 ip addresses (zeus network block)
1 rule for making packets based on dscp 14
no nat rule
The reason he asked for that is that it is entirely possible for you to write the rules you described in such a way as to spin your CPU for every packet or only when necessary.
You can change IP addresses in the rules for privacy, but showing us exactly what the rules are is the only way to allow us to help you figure out what is wrong.
I don’t see much room for improvement there. You might bring the block rule for level3 above the add rules.
It might be less cpu intensive to jump to a “new input connections” chain on connection-state=new. It would add one step for new connections but would eliminate 3 evaluations and all address-list lookups for all established and related connections. The other option would be to explicitly allow established and related connections before evaluating the connection-state=new rules but that would be two rules instead of one jump rule.
In fact, eliminating the address-list lookups for every established and related connection state packet may be enough of a win to be worth having allow rules for established and related connections before the drop rule on your forward chain. I do not know if the firewall does the address-list lookup concurrent with the connection-state comparison or only if the connection-state matches when they are written into one rule.
On the mangle rule, it might be faster to make a connection-mark with passthrough and then mark packets based on connection mark. Or just add an accept rule for !dscp=14 before the packet mark. Not sure. I haven’t spent enough time thinking about it. It may be fine the way it is.
I would be very interested to know if you try any of these suggestions and what, if any, effect they have for you.
If you mark connection, and then only mark packet for that marked connection you will also get a performance increase.
I would try a more clean setup for a performance baseline than the one you have, like lambert says, your FW rules have room for performance. Allow rules first since they are hit by traffic you want, lower down put your drops when possible.
