v6.40.rc4 GRE-IPSec SMB

RouterOS General
1

Hi to all,
I wanted to share with you the results.

[computer] – [CCR1009-7G-1C-1S+] – EoIP (actual MTU 1396) – IPSEC – [CCR1009-7G-1C-1S+PC] – IPSEC – EoIP – [CCR1009-7G-1C-1S+] – [computer]

auth-algorithms=sha1 enc-algorithms=aes-256-cbc,aes-192-cbc,aes-128-cbc lifetime=30m pfs-group=modp2048
03-05-2017-mt-eoip-ipsec.png
Did various test with IPSEC + GRE and EoIP. Will check soon IPIP.

the results are more or less there on the attached screenshot, does it seem right ?
Including the history was tested on windows using jperf 2.0.2.

Will test some more.

Feature request - Diffie Hellman groups 19-21
RB - AES HW encrypt desc removed ?
2

Looks very low. Can you force it to AES128 and see if there is improvement? May also be getting fragmentation issues.

3

04-05-2017-128aes.png

/ip ipsec peer print 
 0     address=10.0.1.1/32 auth-method=pre-shared-key secret="test" 
       generate-policy=no policy-template-group=default exchange-mode=main 
       send-initial-contact=yes nat-traversal=no proposal-check=obey 
       hash-algorithm=sha1 enc-algorithm=aes-128 dh-group=modp1024 lifetime=1d 
       dpd-interval=2m dpd-maximum-failures=5
	   
/ip ipsec proposal print
 0    name="default" auth-algorithms=sha1 enc-algorithms=aes-128-cbc lifetime=30m 
      pfs-group=modp1024 
	   
/ip ipsec policy print
 1  A  src-address=10.0.11.0/24 src-port=any dst-address=10.0.22.0/24 
       dst-port=any protocol=all action=encrypt level=require 
       ipsec-protocols=esp tunnel=yes sa-src-address=10.0.1.1 
       sa-dst-address=10.0.2.1 proposal=default priority=0 ph2-count=2

Dosnt get any better

4

What CPU usage are you seeing during that copy?

5

First (source lan from windows copy) - Second (copy destination win box)
(correction for the second CPU , there was divided during the screenshot, for the most part was concentrated on one core)
04-05-2017_mt_cpu.png

6

If you copy a number of files at once, does it increase the throughput?

7

Without testing I can reply with certainty, no, it does not.

Sent from my D5803 using Tapatalk

8

What is your clamp MSS setting on the EoIP? If you disable encryption how much faster is it?

9

200Mbps TCP traffic over single ipsec/eoip tunnel is good result. To increase overall performance you need multiple streams (tunnels)

10

RB1100AHx2 from 5 years ago can do 500mbps+ on a TCP single stream on one EoIP+IPSEC tunnel. 700mbps with multiple TCP streams. Should the CCR not be faster?

11

It is faster with multiple streams

12

Fair enough. Do you think the RB1100AHx4 can beat the x2 with low stream count?

13

With bonding after?
How is it implemented ? With IPSEC from same interfaces and diffrent IPs ?
In paralel ? Multi subnet ipsec on/from the same interface and then EoIP or GRE?
After that bonding the two resulting interfacfes using bonding (balance rr)?

I tried it with´out success, maybe I am doing it wrong .. The second IPSEC has issues for “phase 2”..
Do you have code example for multi stream ipsec config?


Regards.

P.S. Two EoIP on the same IPSEC are doing the same bandwith like one. I am hitting the “rule of tomb 1 ipsec per core”.

14

One must also ask: Source Media and Destination Media 20Mbyte per second (200Mbit) is about what your average 2" spinning harddrive does after all buffers have been depleted. Allways test network with memory transfers so you test the real performance.

15

Update

created:
eth7 WAN
eth6 IP0 + IP1-IP2
eth5 IP3-IP4
(apparently on the CCR 1009 1-4 port are switch ports, only the 3 last ones are HW encryption accelerated)
https://wiki.mikrotik.com/wiki/Manual:IP/IPsec#CCR1009_hardware_optimizations

Update: CCR1009-7G should have all ports accelerated !! Is this info for all CCR1009 or just the CCR1009-8G ?!?!?!?!

VPN:
4 IPSEC (IP1,IP2,IP3,IP4 each on they own subnet)
4 EoIP On each of the end point (Clamp TCP MSS + Fast Path)
1x BONDING balance RR

Routed the IP1 subnet to the BONDING.

the result is almost same as previous ones, I tried even to change the MTU, it got worse:

The traffic is windows copy:
10-05-2017_a1.png
Jperf, two TCP streams in both directions:
10-05-2017_a2.png
CPU, during jperf:
10-05-2017_a3.png
I even disabled EoIP 2 and 4, there was NO change. The throughput was just balanced on the remaining two EoIP tunnels.

any ideas ?

16

Thanks to all the feedback.

To conclude this Odyssey, if you need encryption, take another better product. If you need basic routing/firewall go for this product family, but if you need encryption with tunnels you are in bad luck. This is not one of the accelerated aes puppies, even multi treading benefits on this arhitekture is questionable. This is all old software single tread limit.

I am happy that I had 3 of this to make a test lab, to go beyond the mikrotik brochure and marketing and make conclusion on my own. In the future I will use heavily the return policy, if I want to hurt my self some more. Will not hold to promise’s of the “will be fixed in the next RC” type. For the peace of mind even the pfsense development​ and no warranty doesn’t sound bad after this 3+ years and counting of beta testing for this specific feature. If that’s the norm, the refund policy of your products should be changed to 4 years !

Thanks to my local MT supplier for all the help.

Regards.