First off, this post is not about thrashing Mikrotik. We are generally very happy with ROS, Mikrotik hardware and Mikrotik’s support.
In this case I’d like to document a hard limit on the CCR routers we’ve purchased so others don’t make the same mistake.
If you ever think about using one of the CCR routers for a road-warrior IKEv2/IPSec setup with more than a few hundred clients: don’t.
As it turns out, the additional 30+ cores on the CCR1036-8G-2S+ can do a nice job once the “quick mode” SAs are in place but will be useless during the establishment of IKE_SAs and/or authentication using signature verification, since the IKE daemon seems to be single threaded and therefore can only use one cpu-core to handle all ISAKMP requests.
When overwhelmed, the IKE daemon will quickly start losing already established security associations and will also become unable to authenticate new peers.
The attached plot shows what happened a few weeks ago during morning hours when we’ve reached a total number of about 1250 peers:
https://i.imgur.com/QIRaQrB.png
(note: “IKE_SA_INIT rate” is the rate of first packets in a IKEv2 handshake sent by clients as seen and logged by the ROS firewall)
If you, for some reason, must use ROS with a scenario like ours (~2500 road warriors), use the x86 image on some high-powered Intel box, where one core can handle more work.
The CCRs were about to replace our x86 HP servers with ROS installed but now we probably have to go back and retrofit them.