Community discussions

 
Ascendo
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 67
Joined: Sun Sep 09, 2012 12:06 pm

CCR1009 or RB1100AHx2 for EoIP+IPSec

Tue Jun 10, 2014 10:35 am

Hello all

We currently link two sites together with L2TPv3+IPSec using Cisco 2900 routers, but the IPSec performance is horrible (max 50mbps @100%CPU).

We want to replace with one of the above routers at either side. Obviously CCR is newer but is the IPSec hardware accelerated yet? Will either of the above be able to do say 150mbps of AES-128 of typical traffic?

Thanks

A
 
User avatar
tomaskir
Trainer
Trainer
Posts: 1110
Joined: Sat Sep 24, 2011 2:32 pm
Location: Slovakia

Re: CCR1009 or RB1100AHx2 for EoIP+IPSec

Tue Jun 10, 2014 10:48 am

Both support HW accelerated aes-128-cbc -> aes-256-cbc.
CCR also supports sha1 HW offload.

1100AHx2 will do about 400 MBit/s of IPSec at aes-128-cbc.
CCR will do much more - dependant on model.
Last edited by tomaskir on Tue Jun 10, 2014 11:22 am, edited 2 times in total.
Unimus - configuration management, automation and backup solution
Mass Config Push, network-wide RouterOS upgrades, and more!
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 24200
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: CCR1009 or RB1100AHx2 for EoIP+IPSec

Tue Jun 10, 2014 11:11 am

CCR1036 can do up to 3.5GBit of IPsec encrypted traffic
No answer to your question? How to write posts
 
Ascendo
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 67
Joined: Sun Sep 09, 2012 12:06 pm

Re: CCR1009 or RB1100AHx2 for EoIP+IPSec

Tue Jun 10, 2014 12:14 pm

CCR1036 can do up to 3.5GBit of IPsec encrypted traffic
Thanks. I assume that means CCR1009 can do at least 150mbit/sec IPSec. Will combining with EoIP tunneling cause much slowdown? I read in some other thread about IPSec encrypted GRE tunnels being slow - will the same issue affect me here?
 
Ascendo
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 67
Joined: Sun Sep 09, 2012 12:06 pm

Re: CCR1009 or RB1100AHx2 for EoIP+IPSec

Fri Jun 13, 2014 7:00 pm

Update: we bought 2 RB1100AHx2 and are running EoIP+IPSec between them across the WAN. Works great, getting over 400mbit/sec! :D
 
w0lt
Member
Member
Posts: 484
Joined: Wed Apr 02, 2008 2:12 pm
Location: Minnesota USA

Re: CCR1009 or RB1100AHx2 for EoIP+IPSec

Fri Jun 13, 2014 9:31 pm

Both the CCR1009 and RB1100AHx2 are great for IPsec.
I would spend a little more and get the CCR1009 for more features..
Well worth it..
8)

-tp
MTCNA - 2011

" The Bitterness of Poor Quality Remains Long After the Sweetness of Low Price is Forgotten "

Image
 
User avatar
docmarius
Forum Guru
Forum Guru
Posts: 1219
Joined: Sat Nov 06, 2010 12:04 pm
Location: Timisoara, Romania
Contact:

Re: CCR1009 or RB1100AHx2 for EoIP+IPSec

Sat Jun 14, 2014 1:34 am

Except higher processor speed, 4 ports less, only one switch group and no bypass switch, these "more" features being what?
Torturing CCR1009-7G-1C-1S+, RB450G, RB750GL, RB951G-2HnD, RB960PGS, RB260GSP, OmniTIK 5HnD and NetMetal 922UAGS-5HPacD + R11e-5HnD in my home network.
 
w0lt
Member
Member
Posts: 484
Joined: Wed Apr 02, 2008 2:12 pm
Location: Minnesota USA

Re: CCR1009 or RB1100AHx2 for EoIP+IPSec

Sat Jun 14, 2014 4:00 am

Except higher processor speed, 4 ports less, only one switch group and no bypass switch, these "more" features being what?
Well, let's see....
Dual/Redundant Power Supplies,
Multiple cores,
LCD Display,
Smart Card Slot,
Access to the MicrSD card in the back,
...Should be enough...eh?

8)
MTCNA - 2011

" The Bitterness of Poor Quality Remains Long After the Sweetness of Low Price is Forgotten "

Image
 
User avatar
docmarius
Forum Guru
Forum Guru
Posts: 1219
Joined: Sat Nov 06, 2010 12:04 pm
Location: Timisoara, Romania
Contact:

Re: CCR1009 or RB1100AHx2 for EoIP+IPSec

Sat Jun 14, 2014 4:55 am

Hmmm, let's see, compared to the AHx2.
CCR1009-8G-1S: No redundant power supply, no LCD display, no smart card slot, 1 USB, half the memory, 1 SFP, 4 copper ports less: +75$
CCR1009-8G-1S-1S+: redundant power supply, LCD, same amount of RAM, 1 USB, 1 SFP, 1 SFP+, 4 copper ports less: +145$
So for the basic modell. except the speed (which is pretty much useless until you need more than 1Gb/sec speed), and the SFP, there is nothing to it.

I don't want to sound naggy, but one has to choose the best solution for a given task. And in this case, the CCR is just a waste of money.
A RB100AHx2 would suffice, with an at least 75$ smaller price tag (that is 150$ for the pair).
I would dare to say that the upcoming RB850 would be the device of choice in this case.

Anyway, they have a couple of RB1100AHx2 now.
Torturing CCR1009-7G-1C-1S+, RB450G, RB750GL, RB951G-2HnD, RB960PGS, RB260GSP, OmniTIK 5HnD and NetMetal 922UAGS-5HPacD + R11e-5HnD in my home network.
 
roadracer96
Forum Veteran
Forum Veteran
Posts: 714
Joined: Tue Aug 25, 2009 12:01 am

Re: CCR1009 or RB1100AHx2 for EoIP+IPSec

Tue Jun 17, 2014 4:33 am

For a single connection, you won't likely get 150mbit throughout with gre or eoip over IPSec on any ccr. S
 
Ascendo
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 67
Joined: Sun Sep 09, 2012 12:06 pm

Re: CCR1009 or RB1100AHx2 for EoIP+IPSec

Mon Feb 23, 2015 9:53 pm

Just an update here for anyone interested. After extensive tweaking and simplification of the config, we've managed to extract around 680mbit/sec out of the two RB1100AHx2 routers (including clocking them up to 1333MHz).

We bought 2xCCR1009s for another project, and decided to see how fast they would work for this particular requirement. We copied the config across, and the performance sucks. We were only able to get 25mbit/sec with the same config. It seems as soon as you combine EoIP or other tunneling with IPSec on the CCR series, the performance drops dramatically. Bandwidth test between the EoIP endpoints runs 600M+, and if you disable the encryption, EoIP runs at 900M+, but when you combine them, the performance drops to 25 :(

We tried fiddling with MTU, MSS clamping, changing ports etc, but could not fix the problem, so it looks like the 1100s will have to do for now.

Mikrotik, how about an updated PPC model with higher clock speed? :)
 
coylh
Member Candidate
Member Candidate
Posts: 160
Joined: Tue Jul 12, 2011 12:11 am

Re: CCR1009 or RB1100AHx2 for EoIP+IPSec

Thu Feb 26, 2015 6:39 am

I did a lot of testing this evening. I'm not getting extremely consistent results, but I was able to get around 500Mb/s on a CCR1036 to CCR1009 link. Both were using 6.27 with updated firmware. The only good results were with aes-256-cbc.

I notice that "in-state-sequence-errors" are very high though.

Here's one side of the lab, if someone wants to reproduce. Note that there's no NAT, firewalling, or connection tracking; YMMV. I was using iperf on windows in tcp mode with --window set to 16M and 8-16 parallel streams. Without ipsec, the same test produces about 850Mb/s.
# feb/24/2015 17:53:51 by RouterOS 6.27
#
#
/ip ipsec proposal
set [ find default=yes ] enc-algorithms=aes-256-cbc
/port
set 0 name=serial0
set 1 name=serial1
/ip firewall connection tracking
set enabled=no
/ip address
add address=10.0.0.2/24 interface=ether1 network=10.0.0.0
add address=192.168.1.1/24 interface=ether5 network=192.168.1.0
/ip ipsec peer
add address=10.0.0.1/32 enc-algorithm=aes-256 nat-traversal=no secret=password
/ip ipsec policy
set 0 disabled=yes
add dst-address=192.168.0.0/24 sa-dst-address=10.0.0.1 sa-src-address=10.0.0.2 src-address=\
    192.168.1.0/24 tunnel=yes
/ip route
add distance=1 gateway=10.0.0.1
/system clock
set time-zone-autodetect=no time-zone-name=America/Los_Angeles
/system ntp server
set enabled=yes
/system resource irq rps
set ether1 disabled=yes
set ether2 disabled=yes
set ether3 disabled=yes
set ether4 disabled=yes
/system routerboard settings
set cpu-frequency=1200MHz enter-setup-on=delete-key memory-frequency=1066DDR
 
Ascendo
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 67
Joined: Sun Sep 09, 2012 12:06 pm

Re: CCR1009 or RB1100AHx2 for EoIP+IPSec

Thu Feb 26, 2015 7:24 am

If you have some time, would you be able to create an EoIP tunnel and encrypt the endpoints? I would be very keen to see if your performance drops 90%+ like it did for us.
 
coylh
Member Candidate
Member Candidate
Posts: 160
Joined: Tue Jul 12, 2011 12:11 am

Re: CCR1009 or RB1100AHx2 for EoIP+IPSec

Thu Feb 26, 2015 9:12 am

Yes, I believe I've set this up. I haven't tried this configuration before, so you'll want to look at the config with some skepticism. Still, I see the SA byte counters moving, so it looks like the traffic is crossing both the EOIP tunnel and the IPSEC.

Here's one side:
# feb/24/2015 20:34:11 by RouterOS 6.27
#
#
/interface bridge
add name=bridge1
/interface eoip
add !keepalive mac-address=02:02:96:29:2A:86 name=eoip-tunnel1 remote-address=192.168.0.1 tunnel-id=0
/ip ipsec proposal
set [ find default=yes ] enc-algorithms=aes-256-cbc
/port
set 0 name=serial0
set 1 name=serial1
/interface bridge port
add bridge=bridge1 interface=eoip-tunnel1
add bridge=bridge1 interface=ether5
/ip firewall connection tracking
set enabled=no
/ip address
add address=10.0.0.2/24 interface=ether1 network=10.0.0.0
add address=192.168.1.1/24 interface=ether5 network=192.168.1.0
/ip ipsec peer
add address=10.0.0.1/32 enc-algorithm=aes-256 exchange-mode=main-l2tp nat-traversal=no secret=password
/ip ipsec policy
set 0 disabled=yes
add dst-address=0.0.0.0/0 sa-dst-address=10.0.0.1 sa-src-address=10.0.0.2 src-address=0.0.0.0/0 tunnel=yes
/ip route
add distance=1 gateway=10.0.0.1
/system clock
set time-zone-autodetect=no time-zone-name=America/Los_Angeles
/system ntp server
set enabled=yes
/system resource irq rps
set ether1 disabled=yes
set ether2 disabled=yes
set ether3 disabled=yes
set ether4 disabled=yes
/system routerboard settings
set cpu-frequency=1200MHz enter-setup-on=delete-key memory-frequency=1066DDR
The results are not as good, but not terrible:
C:\iperf>iperf --client 172.16.5.55 --parallel 16 --window 16M
------------------------------------------------------------
Client connecting to 172.16.5.55, TCP port 5001
TCP window size: 16.0 MByte
------------------------------------------------------------
[ 18] local 172.16.5.44 port 51435 connected with 172.16.5.55 port 5001
[ 16] local 172.16.5.44 port 51433 connected with 172.16.5.55 port 5001
[ 15] local 172.16.5.44 port 51432 connected with 172.16.5.55 port 5001
[ 12] local 172.16.5.44 port 51429 connected with 172.16.5.55 port 5001
[ 11] local 172.16.5.44 port 51428 connected with 172.16.5.55 port 5001
[  8] local 172.16.5.44 port 51425 connected with 172.16.5.55 port 5001
[  7] local 172.16.5.44 port 51424 connected with 172.16.5.55 port 5001
[  3] local 172.16.5.44 port 51420 connected with 172.16.5.55 port 5001
[ 17] local 172.16.5.44 port 51434 connected with 172.16.5.55 port 5001
[ 14] local 172.16.5.44 port 51431 connected with 172.16.5.55 port 5001
[ 13] local 172.16.5.44 port 51430 connected with 172.16.5.55 port 5001
[ 10] local 172.16.5.44 port 51427 connected with 172.16.5.55 port 5001
[  9] local 172.16.5.44 port 51426 connected with 172.16.5.55 port 5001
[  6] local 172.16.5.44 port 51423 connected with 172.16.5.55 port 5001
[  5] local 172.16.5.44 port 51422 connected with 172.16.5.55 port 5001
[  4] local 172.16.5.44 port 51421 connected with 172.16.5.55 port 5001
[ ID] Interval       Transfer     Bandwidth
[  8]  0.0-10.0 sec  36.2 MBytes  30.4 Mbits/sec
[  6]  0.0-10.0 sec  42.0 MBytes  35.2 Mbits/sec
[ 17]  0.0-10.0 sec  26.1 MBytes  21.8 Mbits/sec
[  9]  0.0-10.0 sec  46.9 MBytes  39.2 Mbits/sec
[ 10]  0.0-10.0 sec  35.5 MBytes  29.6 Mbits/sec
[ 14]  0.0-10.1 sec  42.4 MBytes  35.3 Mbits/sec
[ 11]  0.0-10.1 sec  36.2 MBytes  30.2 Mbits/sec
[ 16]  0.0-10.1 sec  41.8 MBytes  34.7 Mbits/sec
[ 15]  0.0-10.1 sec  33.6 MBytes  27.9 Mbits/sec
[  7]  0.0-10.1 sec  34.6 MBytes  28.7 Mbits/sec
[ 18]  0.0-10.1 sec  38.1 MBytes  31.5 Mbits/sec
[  4]  0.0-10.1 sec  40.4 MBytes  33.4 Mbits/sec
[ 12]  0.0-10.2 sec  42.2 MBytes  34.9 Mbits/sec
[  5]  0.0-10.2 sec  45.6 MBytes  37.6 Mbits/sec
[  3]  0.0-10.2 sec  37.8 MBytes  30.9 Mbits/sec
[ 13]  0.0-10.7 sec  35.2 MBytes  27.8 Mbits/sec
[SUM]  0.0-10.7 sec   615 MBytes   484 Mbits/sec
I would like to try this across a couple 1100AHx2. I'll post the results if I can cobble them together.
 
Ascendo
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 67
Joined: Sun Sep 09, 2012 12:06 pm

Re: CCR1009 or RB1100AHx2 for EoIP+IPSec

Thu Feb 26, 2015 9:31 am

Yes, I believe I've set this up. I haven't tried this configuration before, so you'll want to look at the config with some skepticism. Still, I see the SA byte counters moving, so it looks like the traffic is crossing both the EOIP tunnel and the IPSEC.

Here's one side:
# feb/24/2015 20:34:11 by RouterOS 6.27
#
#
/interface bridge
add name=bridge1
/interface eoip
add !keepalive mac-address=02:02:96:29:2A:86 name=eoip-tunnel1 remote-address=192.168.0.1 tunnel-id=0
/ip ipsec proposal
set [ find default=yes ] enc-algorithms=aes-256-cbc
/port
set 0 name=serial0
set 1 name=serial1
/interface bridge port
add bridge=bridge1 interface=eoip-tunnel1
add bridge=bridge1 interface=ether5
/ip firewall connection tracking
set enabled=no
/ip address
add address=10.0.0.2/24 interface=ether1 network=10.0.0.0
add address=192.168.1.1/24 interface=ether5 network=192.168.1.0
/ip ipsec peer
add address=10.0.0.1/32 enc-algorithm=aes-256 exchange-mode=main-l2tp nat-traversal=no secret=password
/ip ipsec policy
set 0 disabled=yes
add dst-address=0.0.0.0/0 sa-dst-address=10.0.0.1 sa-src-address=10.0.0.2 src-address=0.0.0.0/0 tunnel=yes
/ip route
add distance=1 gateway=10.0.0.1
/system clock
set time-zone-autodetect=no time-zone-name=America/Los_Angeles
/system ntp server
set enabled=yes
/system resource irq rps
set ether1 disabled=yes
set ether2 disabled=yes
set ether3 disabled=yes
set ether4 disabled=yes
/system routerboard settings
set cpu-frequency=1200MHz enter-setup-on=delete-key memory-frequency=1066DDR
The results are not as good, but not terrible:
C:\iperf>iperf --client 172.16.5.55 --parallel 16 --window 16M
------------------------------------------------------------
Client connecting to 172.16.5.55, TCP port 5001
TCP window size: 16.0 MByte
------------------------------------------------------------
[ 18] local 172.16.5.44 port 51435 connected with 172.16.5.55 port 5001
[ 16] local 172.16.5.44 port 51433 connected with 172.16.5.55 port 5001
[ 15] local 172.16.5.44 port 51432 connected with 172.16.5.55 port 5001
[ 12] local 172.16.5.44 port 51429 connected with 172.16.5.55 port 5001
[ 11] local 172.16.5.44 port 51428 connected with 172.16.5.55 port 5001
[  8] local 172.16.5.44 port 51425 connected with 172.16.5.55 port 5001
[  7] local 172.16.5.44 port 51424 connected with 172.16.5.55 port 5001
[  3] local 172.16.5.44 port 51420 connected with 172.16.5.55 port 5001
[ 17] local 172.16.5.44 port 51434 connected with 172.16.5.55 port 5001
[ 14] local 172.16.5.44 port 51431 connected with 172.16.5.55 port 5001
[ 13] local 172.16.5.44 port 51430 connected with 172.16.5.55 port 5001
[ 10] local 172.16.5.44 port 51427 connected with 172.16.5.55 port 5001
[  9] local 172.16.5.44 port 51426 connected with 172.16.5.55 port 5001
[  6] local 172.16.5.44 port 51423 connected with 172.16.5.55 port 5001
[  5] local 172.16.5.44 port 51422 connected with 172.16.5.55 port 5001
[  4] local 172.16.5.44 port 51421 connected with 172.16.5.55 port 5001
[ ID] Interval       Transfer     Bandwidth
[  8]  0.0-10.0 sec  36.2 MBytes  30.4 Mbits/sec
[  6]  0.0-10.0 sec  42.0 MBytes  35.2 Mbits/sec
[ 17]  0.0-10.0 sec  26.1 MBytes  21.8 Mbits/sec
[  9]  0.0-10.0 sec  46.9 MBytes  39.2 Mbits/sec
[ 10]  0.0-10.0 sec  35.5 MBytes  29.6 Mbits/sec
[ 14]  0.0-10.1 sec  42.4 MBytes  35.3 Mbits/sec
[ 11]  0.0-10.1 sec  36.2 MBytes  30.2 Mbits/sec
[ 16]  0.0-10.1 sec  41.8 MBytes  34.7 Mbits/sec
[ 15]  0.0-10.1 sec  33.6 MBytes  27.9 Mbits/sec
[  7]  0.0-10.1 sec  34.6 MBytes  28.7 Mbits/sec
[ 18]  0.0-10.1 sec  38.1 MBytes  31.5 Mbits/sec
[  4]  0.0-10.1 sec  40.4 MBytes  33.4 Mbits/sec
[ 12]  0.0-10.2 sec  42.2 MBytes  34.9 Mbits/sec
[  5]  0.0-10.2 sec  45.6 MBytes  37.6 Mbits/sec
[  3]  0.0-10.2 sec  37.8 MBytes  30.9 Mbits/sec
[ 13]  0.0-10.7 sec  35.2 MBytes  27.8 Mbits/sec
[SUM]  0.0-10.7 sec   615 MBytes   484 Mbits/sec
I would like to try this across a couple 1100AHx2. I'll post the results if I can cobble them together.
Interesting result. How much can you get on a single thread?

Are you intending to run eoip over l2tp+IPSec?

Our IPSec peer config looks like this:

/ip ipsec peer
add address=10.10.10.6/32 dpd-maximum-failures=10 enc-algorithm=aes-128 generate-policy=port-override lifetime=8h nat-traversal=no proposal-check=exact secret=password
 
coylh
Member Candidate
Member Candidate
Posts: 160
Joined: Tue Jul 12, 2011 12:11 am

Re: CCR1009 or RB1100AHx2 for EoIP+IPSec

Fri Feb 27, 2015 1:58 am

eoip+ipsec single thread is about 135Mb/s. With this test tool, I consider it normal to be unable to saturate a link with a single tcp stream though, regardless of the tunnel configuration.
 
coylh
Member Candidate
Member Candidate
Posts: 160
Joined: Tue Jul 12, 2011 12:11 am

Re: CCR1009 or RB1100AHx2 for EoIP+IPSec

Sat Feb 28, 2015 5:13 am

Ok, I hooked up a couple 1100AHx2 with more or less the same configuration as the CCR pair. I'm able to get around 800Mb/s EoIP + IPSEC. The downside of the better performance is that the dual CPU/irq is maxed out. I think running the routerboards at 90%+ is going to cause system stability problems, so I think you're better off capping the link at something a bit lower (600Mb/s?).

While the CCR's cpu appears to be off the hook for the lower tunnel performance, there's something wrong with either the hw accelerator or the software implementation. I hope this gets fixed. For the time being, it looks like the 1100AHx2 is better for ipsec.
 
User avatar
payday
Member Candidate
Member Candidate
Posts: 233
Joined: Thu Aug 16, 2012 11:05 pm

Re: CCR1009 or RB1100AHx2 for EoIP+IPSec

Sat Feb 28, 2015 10:49 pm

Guys, this is really surprising and baffling! I was sure that CCR with 9 cores and IPsec accelerator is indisputable faster then old RB1100AHx2 with accelerator and only 2 cores! This document http://www.tilera.com/files/drim__TILE- ... B_7663.pdf states it should deliver up to "10Gbps encryption throughput" :
TileGx.JPG
How is this possible that CCR is slower? Is MikroTik working on it's optimization? Is this maximum IPsec performance?
You do not have the required permissions to view the files attached to this post.

Who is online

Users browsing this forum: No registered users and 74 guests