Current setup with two CCR1036 running latest routerOS 6.25. A laptop connects to each CCR1036 as a LAN device. WAN interface of CCR1036 is set to auto-negotiate 1000M full-duplex. I want to run IPsec and OSPF, so GRE is used as well.
First test: Without IPsec and GRE, just OSPF. Performance was great, ftp a 200M file in less than 3 seconds, 88M bytes/s.
Second test: Add GRE with OSPF. Performance was great, too. FTP with similar result.
Third test: IPsec, GRE and OSPF. Performance was bad. FTP same 200M file, took 23 seconds, 8.6M bytes/s
Forth test: IPsec, GRE, OSPF and change WAN to 100M full-duplex/ Performance was better than 3rd test. FTP same 200M file, 18 seconds, 11M bytes/s.
I did the test with routerOS 6.23, similar performance.
I am wondering why 100M performs better than 1G. What performance should I expect from CCR1036?
Using AES. Here is the output from ip IPsec installed-sa print:
0 E spi=0x74C2F93 src-address=10.4.2.2 dst-address=10.4.2.1 state=mature auth-algorithm=sha1 enc-algorithm=aes-cbc
auth-key=“86096f71d02017538d4390934e885406be2fe831” enc-key=“673b2fa1a21a35f561372e19c2c1bbe9”
addtime=jan/02/1970 04:34:24 expires-in=20m17s add-lifetime=24m/30m current-bytes=13778 replay=4
Have the same issue with two CCR1036
I’ve connected them with 10Gbit/s link and set up gre tunnel with IPSEC
with aes-256 I have up to 80Mbit/s, with aes-128 without authentication I have up to 150Mbit/s
While downloading file through IPSEC one CPU on each router is 100% loaded. All other cores are idle.
According to manual
RB1000 with enabled HW support can forward up to 550Mbps encrypted traffic.
I expect at least the same or better performance on CCR1036
Are there any ideas how to speed up IPSEC? How to check if hardware encryption really works?
Just upgrade to the latest 6.27, still the same poor performance when I have both CCR back-to-back connected using auto-negotiate (connected at 1G). The fix config with 100M full duplex is far better performance. One note forgot to mention, if the interface was set to no negotiate and fix it at 1G full-duplex on each side, they wouldn’t connect.
Soooo? what happened here? Configuration issue? Firmware issue? Did the issue get resolved?
Or is it just not possible to do a high performance IPSEC between 2 CCR1036’s?
Mikrotik support was able to help me to get the Gigabit interface speed using IPSec close to 100M by selecting the AES-GCM option. It was better but I expected it would go beyond 100M throughput. I followed up with few emails with the support, no reply. Upgrade firmware to 6.28, still did not help at all.
They asked me to try. Actually I gave up on this topic, took me over a year to work with their tech and still couldn’t get my issue resolved. The last message from Mikrotik was my hardware problem, asked me to test with other hardware. I had two CCR1036 and one CCR1016, tried with different combination between them, still the same issue. So I don’t think it is related to hardware.
I think the issue is well understood, as can be found in other threads.
When using the aes-cbc mode, the encryption is hardware-accelerated, and the hardware has many cores that
can do this in parallel. By splitting the acceleration over cores, the packets get re-ordered depending on timing
details, and the end result is that the packets arrive in a slightly different order than they were sent.
This actually should not affect throughput at all, since the internet is specified that way (it can re-order packets at will),
but in practice many broken TCP/IP stacks blindly assume that when a packet is missing between two others, it
has been dropped and will have to be re-sent, and immediately send a request to do so.
Thus, when re-ordering is present a lot of bandwitdh is spent on sending duplicate packets.
This is not a defect of the router, but rather of the endpoint systems. However, it can be worked around in this case
by selecting an encryption method that won’t be hardware-accelerated.
One could argue that some synchronization should be added to RouterOS to assure that even when hardware
accelerated encryption on the multicore architecture is done, the packets will still leave the router in sequence,
to work around those end-system bugs. Of course that might decrease performance in other situations,
especially in benchmarking. Manufacturers often don’t like that, because benchmarking is what is being done
to compare their routers to others.
My question is which encryption that is not hardware-accelerated? If using non hardware-accelerated encryption, is that meant it is using single CPU? That’s why the performance will be poor. In my setup, I want to add Mikrotik at both ends of a Point-to-point wireless link to improve security. The link is capable at 150M bps full duplex. Any suggestion?
>>When using the aes-cbc mode, the encryption is hardware-accelerated and the hardware has many cores that can do this in parallel
No difference. Many cores in parallel can be used in any mode, with and without hardware-accereration.
But, one core with hardware acceleration is faster in 10…100 times than one core without hw-accel.
And one ipsec aes-cbc tunnel on one core should be faster than aes-xxx tunnel on any number cores!
But, ipsec on ccr very slow for unknown reason.
Mikrotik team cannot fix this CCR issue for many years!
possible reasons:
programmers of the Mikrotik are very bad
or selection of TILERA processor for RB was an epic mistake