CCR2004 : BGP Benchmarks

Hello,

I just received my new CCR2004 and I couldn’t wait to put it in the grill
For the record, i also tested few other devices :

  • CCR1009 (7g-1c-1s+) v6.47b60
  • CCR1016 (12s-1s+) v6.47b60
  • RB4011 (igs-rm) v6.47b60

The CCR2004 runs its factory version 6.46.3 since there is no other package available to install.

Test protocol :

Four BIRD (v4 only) daemons with 800k+ routes upstreamed from a CCR1072 located into AS57199
Four BGP sessions to each router, no IGP, nothing more than BGP !

I just waited for the VMs to converge, and i configured the 4 bgp peers in the same time and I used “/ip route count print” to monitor the amount of routes.

Here are the results :

BGP Insertion (4xFullviews, ~3,2M routes) :

  1. RB4011 : 3m45s
  2. CCR2004 : 5m38s
  3. CCR1016 : 10m09s
  4. CCR1009 : 10m45s

Impressive results for the RB4011, less for the CCR2004, maybe this is because of the 32bit version of RouterOS on 64bit hardware ?


For the next test, i just deconfigured the 4 peers and waited for the route count to return to almost 0.

BGP Removal (4xFullviews, ~3,2M routes) :

  1. CCR1016 : 3m18s
  2. CCR1009 : 3m25s
  3. RB4011 : 8m25s
  4. CCR2004 : 19m58s

And i have to say that I am a little bit disappointed. Why is the 2004 so bad ? Are some hacks or improvements added to tile version missing ?
i hope that future versions of RouterOS 6 (i’d like to be able to use my router before 2021 :smiley: ) will fix this :frowning:

Hugues

Here you could find a link for 6.46.6:

http://forum.mikrotik.com/t/ccr2004-w-arm64-where-to-download-packages/139426/1

Also a long term version is available:

https://download.mikrotik.com/routeros/6.45.9/routeros-arm64-6.45.9.npk

Yes, i am aware of it, i am the author of this topic. You can also see that this current topic is earlier than the answers in the other topic :wink:

Disappointed by those 2004 numbers if I have to be honest.The removal times makes it a total no go in my edge routing cases pretty much.

Have you run some bandwidth/latency tests through the CCR2004 to compare it to the 1016’s ?
Can it push 2.5+ Gbps of 1500 mtu TCP b/w per stream ?

I didn’t run any bandwith tests for now, i trust the Mikrotik’s Datasheet :slight_smile:

A bit better with 6.46.6 :

BGP Insertion (4xFullviews, ~3,2M routes) :
CCR2004 : 1m50s

BGP Removal (4xFullviews, ~3,2M routes) :
CCR2004 : 7m30s

But still slower than a 1009 :frowning:

Those are much better numbers have to say.

Thanks sir for the update. Mikrotik Datasheet numbers I always look at with a very wary eye.
There tends to be a lot of asterisks and gotchas associated with it.

If it’s possible could you do iperf UDP/TCP tests as well as posting CPU/Memory Resource utilization and latency when doing these tests ? I’d love to know how much CPU you’re using when doing 10+ Gbps routing as sitting at 99% is effectively useless to me.

The community would really appreciate it getting actual results from the equipment.

BGP Benchmarks are much better with a lower amount of routes :


1 Full (800k routes):

  • Insertion : 47 seconds
  • Removal : 7 seconds
    2 Full (1,6M routes):
  • Insertion : 1 minute 9 seconds
  • Removal : 39 seconds
    3 Full (2,4M routes) :
  • Insertion : 1 minute 29 seconds
  • Removal : 3 minutes
    4 Full (3,2M routes) :
  • Insertion : 1 minute 51 seconds
  • Removal : 7 minutes 30 seconds


    For the bandwidth tests, i don’t have much stuff at home, i will try but i can’t guarantee anything :slight_smile:

I simulated a big bgp flap (2 sessions reset + 1 added). It is better than a Tilera, but still miles above a decent router :

it took about 40 minutes to process all the routes to the CCR2004
it took about 1h23 to process all the routes to the CCR1016
It took about 2 minutes to process all the routes to a Cisco ASR9K (with rsp440-se) :stuck_out_tongue:

To be clear : If you experience a big BGP flap, it is still better to reboot all your BGP Core rather than wait for a MikroTik to converge.
(PS : It could be a workaround to let the user flush the table and reset the sessions with a hidden command)

It could be that we’d have to wait for V7 to really see some big BGP performance increases .

We’d have to make a wager: What would get mass released first ? A vaccine for Coronavirus or ROS 7 ?

I’m waiting on my CCR2004 to get here. Then i’m going to benchmark it with our iperf3 and BGP full table performance lab that we’ve used for MUM Presentations in the past.

We maxed out the CCR1072 when it first came out with 80Gbps of traffic, so we should be able to make the CCR1004 fall over :wink:

You can reach 80gbps of unrealistic traffic with a 1072, but with real traffic, it begins to lost packets with a fraction of the max throughput. :confused:

What realistically can you do before you notice issues? I have issues with the 1072’s where they randomly reboot and the watchdog triggers. Maybe 1 time a week or less. No pattern to it and the supout shows nothing.

We have 3 1072’s in a live environment with no reboot issues, albeit with low traffic levels.
However we have run 1-2Gb during testing and they have been fine.

we have a couple 1072’s doing BGP and around 4-5Gbps with no issues. Yes BGP convergence is slow (around 40 peers), but nothing wrong with the 1072.

We will upgrade to the future CCR’s and/or CHR soon though as we want to take in a couple full feeds.

Same here, around 30 CCRs 1072&1036 in production, around 5Gbps in peak, no reboot issues…

is time to test using the new V7 beta 8 :slight_smile:

My 2004 is already in production, but i still have the CCR100X and the 4011 to lab the v7… For the moment, i’m trying to understand the new CLI…

Have you tried with disabled connection tracking?
You will not have stateful firewall without connection tracking, but the performance is way better.

I had the same problems with CCR1036, especially during moderate DDoS attacks. By disabling connection tracking (or using the raw table for the forwarded traffic), the router now forwards 10gbit DDoS traffic without breaking a sweat.
One caveat: even without connection tracking, ROS becomes totally unresponsive during DDoS SYN Flood attacks (even at few Mbps/Kpps).
The only way I found to be able to handle SYN Flood attacks, was to disable Route Cache (which introduces other problems though…)

Here is CPU usage from my ccr2004 router.. 6.47 and ~1M routes

We have route cache disabled on IPv4, what issues has it introduced that has caused you problems?