I found, that RB1200 add 10 -12 ms latency to packet procesing during IPsec enc/decrypting. For example RTT of ICMP packet without ipsec is 2-3 ms, with IPsec is 13-14 ms.
If I tested same configuration on RB2011L, additional IPsec latency is 0-1 ms.
Tested against RB1100AH. ROS 5.21 and 5.22. IPsec with SHA hash and AES-256 cipher.
Do you have same experience with RB1200 or another RBs?
Added later:
Problem is also with ROS 6rc7 regardless on cipher (tested md5/null hash and cipher DES too). Tested with no CPU load and with minimal other traffic (< 0,5 Mbit). When IPsec using null “cipher”, latency is normal < 1 ms.
I tested IPsec with same settings on RB2011, RB750G and old RB600A (ROS 5.21, 5.22 and 6rc7). All of them have normal <1ms IPsec latency.
I tested RB1200 with ROS 6rc11 and both problems (port 9,10 latency problem and IPsec generally problem) persist.
Read again … 5nik wonders why there is latency on rb1200 and not on rb2011 … I have also tested my setup full with some rb2011(uas), rb450g, rb751g and some linux boxes and can also confirm that there is no noticable latency (defintely not 10ms) using this boxes … so, the question from 5nik is why there is latency with rb1200 which has cleary more cpu power than rb2011.
Buying ccr or 1100ahx2 is clearly not necesary for ipsec unless you need to do it with multiple tunnels over (multiple) 100mbit/s links. Yes, those boxes do support hardware encryition, but in smaller enviroments and with slower links, rb2011, 450g and other can do the job just well, especialy if you use aes128 instead of default 3des.
I know about HW accelerated AES support in 1100AH, our core routers (VPN concetrators) are 1100AH and x2.
On some small department, I often need encrypt max 5Mbit/s, and 1100AH is a little more for this purpose.
My topic is not about performance (throughput) but about latency. As JanezFord wrote, why RB with powerfull CPU has higher IPsec latency than low cost RB?
Is such difference between CPU architecture (PPC vs MIPS-BE)? Or this is “bug” in ROS?
If I will have more time, a will test more low cost RB (such RB751G a so on).
In my eyes, RB1200 becomes very unpopular due latency problem (port 9 and 10 generally, IPsec generally).
The problems you describe only happen on upper ports (eth 9 and 10)
Dont use those 2 ports and it will all be fine
There are multiple posts around forums about more problems on the 1200 with these ports (latency increase for all traffic, random packet drops etc.), its mostly due to these ports being connected through a PCI-X interface, which apparently still causes some problems in ROS.
I tested IPsec on RB1200 with other ciphers than AES, and IPsec latency were same (10-12 ms) even when I used less-CPU-consum ciphers like DES.
With null cipher latency fell to normal 0-1 ms.
Thank you ChrisP for links. It is sad, that last post on linked topic is 7 months old and problem is still continuing.
One note: I have never tested RB 1200 with RoS v6, maybe new version solves the latency problem.
I don´t want encrypt traffic higher then 5 Mbit. I’m testing IPsec latency without other trafic through IPsec (only ping). CPU load during test is <5% (no load).
I tested old RB600A, older and slower CPU, same architecture (no AES acceleration). And IPsec latency was 0 ms (RoS 5.22 and 6rc5).
There are four versions of the 19" - 1U rackmount case based routers and you were choosing the low cost variant
for ~$150 less then the RB1100AHx2 with VPN/IPSec hardware acceleration support and now all should running like
the bigger RB1100AHx2? And MikroTik should implement something in software on top to speed this up?
Yes, I expected, that ping (~null traffic) through IPsec on no-loaded CPU will have same latency on all boxes, regardless of HW accelerating AES.
I expect difference between boxes in IPsec throughput.
Who wants to pay more for a RB1100AHx2 if the RB1200 can do the same job?
And who wants pay more for RB1200 if the RB2011L can do the same job (in case IPsec better)?
I found that RB1200 still has two problems: packet latency on ports 9 and 10 and IPsec latency generally on all ports. And as ChrisP wrote, without any satisfactory explanation yet.
Any updates on this issue? Is it fixed? I have a customer with need for rb1200 cpu power and fanless design to run ipsec with other office branches. RB2011 is not enought, anything else is too noisy.
At last, I tested ROS 6.2 and nothing change. I plan to reclaim all RB1200. But due EoL of RB1200, I don’t know which box replace RB1200 (see my topic).
Thak you rjickity for this info… this may indeed help some rb1200 users to get more of their routers. I wonder why does this happen … I am guessing that Camellia encryption is not hardware accelerated by rb1200 cpu and both aes and 3des are and implementation of this acceleration is the key problem. Did you also happen to perform some cpu usage tests with camellia?
Unfortunately I only have an RB2011 available for the otherside at the moment.
95-105Mbit TCP both direction forwarding is achieved before i max out the rb2011uias to 100% CPU. The RB1200 maintains 40% usage at this point. You could probably safely assume at least 200Mbps i guess, just bear in mind no firewall filter or nat is in place with these tests. Latency was ok through the test . Below is the config i used:
