Community discussions

 
hashbang
Member Candidate
Member Candidate
Topic Author
Posts: 151
Joined: Sat Jul 26, 2014 6:38 pm

suggestion for a 5 gig router after testing ccr 1036

Fri Sep 19, 2014 8:04 pm

I installed ccr 1036-8g-2s+, a brief explanation of config :
1) 2 bgp peers + 6 filters
2) 2 isp (ether 7 & 8)
3) 2 lan outs with one lan having 10 vlans (ether 4 & 5)
4) 41 queues
5) 3 rules for ssh bruteforcing protection 1 rule for packet marking.
all other ports are free
Bandwidth running 800mbps download and 150mbps upload. Router cpu consumption had reached 25% approx. I have few questions as ccr 1072 is not going to be launched soon. They are :
Why such a drastic difference in benchmarks of MT and practical test
What h/w is suggested if I need 5gig of routing (if x86 then which hw)
already ordered 5 ccr-1036
No major stability issue found apart from vlan packet drop (which got solved).
BGP filter behaving strangely (posted somewhere already in bgp section).
This router is already running from 80days. 35days idle, 35days 300mbps and from 14 days full load.
 
Zorro
Long time Member
Long time Member
Posts: 676
Joined: Wed Apr 16, 2014 2:43 pm

Re: suggestion for a 5 gig router after testing ccr 1036

Sat Sep 20, 2014 4:09 am

some part of ROS, on which you config relying on so much - obbviously had scalability issues, as you noticed thats 25% CPU load was peak.
as for simple answer - CCR1072 ? :)
 
hashbang
Member Candidate
Member Candidate
Topic Author
Posts: 151
Joined: Sat Jul 26, 2014 6:38 pm

Re: suggestion for a 5 gig router after testing ccr 1036

Sat Sep 20, 2014 8:28 am

which part ? Does MT needs more optimization ? Practical thru is noway near what tilera advertise.
MT is silent about release date of ccr 1072
 
joegoldman
Member
Member
Posts: 498
Joined: Mon May 27, 2013 2:05 am

Re: suggestion for a 5 gig router after testing ccr 1036

Sun Sep 21, 2014 2:15 pm

you should profile and see what is most consuming. At the moment I run about 10 queues, 2 port bonding with 5 BGP peers (1 full table + IX tables, for IPv4 and IPv6) with about 500-600mbit aggregate peak (100k-130k pps), sits at about 15%. There are 2 packet marking rules per queue set up and also about 20 filter rules.

A big difference may be your packet rate causing higher CPU.

If Queuing is the big performance hit, try ROS6.19 if you haven't, meant to be big improvements. If queueing starts to get too much might be an idea to split out the tasks, assign some CCR's to the edge for BGP and have the queuing routers not having to do big routing lookups etc, so you might be able to get fastpath routing or the like in.
 
hashbang
Member Candidate
Member Candidate
Topic Author
Posts: 151
Joined: Sat Jul 26, 2014 6:38 pm

Re: suggestion for a 5 gig router after testing ccr 1036

Sun Sep 21, 2014 8:02 pm

Mine is having 200k pps and 950mbps, cpu usage reaching peak 24%. Screenshots below
You do not have the required permissions to view the files attached to this post.
 
joegoldman
Member
Member
Posts: 498
Joined: Mon May 27, 2013 2:05 am

Re: suggestion for a 5 gig router after testing ccr 1036

Mon Sep 22, 2014 2:19 am

You say you only have 3 or 4 rules for your firewall, maybe do a /ip firewall export (so filtering, NAT and Mangle) so we can have a look - it might be doing more work than it needs to as the firewall shouldn't be taking so much. My Firewall (with 19 filter rules currently) is usually 3rd or 4th in process list. routing, networking and queuing usually the top.
 
hashbang
Member Candidate
Member Candidate
Topic Author
Posts: 151
Joined: Sat Jul 26, 2014 6:38 pm

Re: suggestion for a 5 gig router after testing ccr 1036

Mon Sep 22, 2014 9:47 pm

5 rules to be precise. 4 rules for bruteforcing of telnet and ssh as per wiki
1 rule for blocking dst address list which has 142 ip addresses (zeus network block)
1 rule for making packets based on dscp 14
no nat rule
 
lambert
Long time Member
Long time Member
Posts: 533
Joined: Fri Jul 23, 2010 1:09 am

Re: suggestion for a 5 gig router after testing ccr 1036

Mon Sep 22, 2014 10:45 pm

Joe asked for :
/ip firewall export
The reason he asked for that is that it is entirely possible for you to write the rules you described in such a way as to spin your CPU for every packet or only when necessary.

You can change IP addresses in the rules for privacy, but showing us exactly what the rules are is the only way to allow us to help you figure out what is wrong.
 
hashbang
Member Candidate
Member Candidate
Topic Author
Posts: 151
Joined: Sat Jul 26, 2014 6:38 pm

Re: suggestion for a 5 gig router after testing ccr 1036

Tue Sep 23, 2014 10:42 am

/ip firewall filter
add action=drop chain=forward comment="zeus drop" connection-state=new \
dst-address-list=zeus
add action=add-src-to-address-list address-list=level1 address-list-timeout=\
1m chain=input connection-state=new dst-port=22,23 protocol=tcp
add action=add-src-to-address-list address-list=level2 address-list-timeout=\
1m chain=input connection-state=new dst-port=22,23 protocol=tcp \
src-address-list=level1
add action=add-src-to-address-list address-list=level3 address-list-timeout=\
6h chain=input connection-state=new dst-port=22,23 protocol=tcp \
src-address-list=level2
add action=drop chain=input connection-state=new src-address-list=level3
/ip firewall mangle
add action=mark-packet chain=prerouting comment="dscp" dscp=\
14 new-packet-mark=cache_mark passthrough=no
 
lambert
Long time Member
Long time Member
Posts: 533
Joined: Fri Jul 23, 2010 1:09 am

Re: suggestion for a 5 gig router after testing ccr 1036

Tue Sep 23, 2014 9:57 pm

/ip firewall filter
add action=drop chain=forward comment="zeus drop" connection-state=new dst-address-list=zeus

add action=add-src-to-address-list address-list=level1 address-list-timeout=1m chain=input connection-state=new dst-port=22,23 protocol=tcp
add action=add-src-to-address-list address-list=level2 address-list-timeout=1m chain=input connection-state=new dst-port=22,23 protocol=tcp src-address-list=level1
add action=add-src-to-address-list address-list=level3 address-list-timeout=6h chain=input connection-state=new dst-port=22,23 protocol=tcp  src-address-list=level2
add action=drop chain=input connection-state=new src-address-list=level3

/ip firewall mangle
add action=mark-packet chain=prerouting comment="dscp" dscp=14 new-packet-mark=cache_mark passthrough=no
I don't see much room for improvement there. You might bring the block rule for level3 above the add rules.

It might be less cpu intensive to jump to a "new input connections" chain on connection-state=new. It would add one step for new connections but would eliminate 3 evaluations and all address-list lookups for all established and related connections. The other option would be to explicitly allow established and related connections before evaluating the connection-state=new rules but that would be two rules instead of one jump rule.

In fact, eliminating the address-list lookups for every established and related connection state packet may be enough of a win to be worth having allow rules for established and related connections before the drop rule on your forward chain. I do not know if the firewall does the address-list lookup concurrent with the connection-state comparison or only if the connection-state matches when they are written into one rule.

On the mangle rule, it might be faster to make a connection-mark with passthrough and then mark packets based on connection mark. Or just add an accept rule for !dscp=14 before the packet mark. Not sure. I haven't spent enough time thinking about it. It may be fine the way it is.

I would be very interested to know if you try any of these suggestions and what, if any, effect they have for you.
 
User avatar
TrollMan
Member Candidate
Member Candidate
Posts: 168
Joined: Mon Apr 04, 2011 9:25 pm

Re: suggestion for a 5 gig router after testing ccr 1036

Wed Sep 24, 2014 8:50 am

If you mark connection, and then only mark packet for that marked connection you will also get a performance increase.
I would try a more clean setup for a performance baseline than the one you have, like lambert says, your FW rules have room for performance. Allow rules first since they are hit by traffic you want, lower down put your drops when possible.
 
hashbang
Member Candidate
Member Candidate
Topic Author
Posts: 151
Joined: Sat Jul 26, 2014 6:38 pm

Re: suggestion for a 5 gig router after testing ccr 1036

Wed Sep 24, 2014 12:23 pm

ty, Rule orders changed
/ip firewall filter
add action=add-src-to-address-list address-list=level3 address-list-timeout=\
6h chain=input comment=l3 connection-state=new dst-port=22,23 protocol=\
tcp src-address-list=level2
add action=add-src-to-address-list address-list=level2 address-list-timeout=\
30s chain=input comment=l2 connection-state=new dst-port=22,23 protocol=\
tcp src-address-list=level1
add action=add-src-to-address-list address-list=level1 address-list-timeout=\
1m chain=input comment=l1 connection-state=new dst-port=22,23 protocol=\
tcp
add action=drop chain=forward comment="zeus drop" connection-state=new \
dst-address-list=zeus
add action=drop chain=input comment="ssh brute drop" connection-state=new \
src-address-list=level3
/ip firewall mangle
add action=mark-connection chain=prerouting comment="CACHE Mark" \
dscp=14 new-connection-mark=cache_conn
add action=mark-packet chain=prerouting comment="CACHE Mark" \
connection-mark=cache_conn dscp=14 new-packet-mark=cache_mark \
passthrough=no

Lets see wht changes it does to cpu usage. I'd started graphing the cpu usage.
 
hashbang
Member Candidate
Member Candidate
Topic Author
Posts: 151
Joined: Sat Jul 26, 2014 6:38 pm

Re: suggestion for a 5 gig router after testing ccr 1036

Wed Sep 24, 2014 9:33 pm

No noticeable change
peak cpu usage 24%

Who is online

Users browsing this forum: No registered users and 17 guests