Hello,
[at the bottom is the TL,DR...]
I'm trying to figure out what I might be doing wrong with my IPSEC setup, as I get much lower throughput than "advertised" and uplink is capable.
scenario 1)
For test purposes I have:
left side: file server / iperf3 server -> mikrotik ccr 1009-7g-1s-1s+ (LAN gateway + firewall) -> mikrotik ccr 1009-8port (WAN / IPSEC tunnel end - I'm using on this model ports 6-7, so direct-to-cpu, not the atheros-switch chipped)
right side: windows client -> MIKROTIK ccr 1009-7g-1s-1s+ (WAN / IPSEC tunnel other end).
I don't have issues withbasic connectivity (everything connects at routes as it should), but the throughput is much lower than expected based on official data sheet and reading posts on this and other forums.
All 1009ccr run latest stable ROS (6.46.6 right now). I know that earlier there were issues with CCR packet reordering, but I think they were fixed many ROS versions ago.
All IPSEC tests done with IKEV2 and sha1/ aes128 or sha1/ aes256 with different variants:
* GRE
* IPIP
*just ipsec (tunnel mode)
WIth both mikrotiks connected directly to each other via 1 Gb/s ports [port 7 on both devices] maximum throughput I could achieve was roughly 230 Mb/s encrypted (roughly 20 kpp/s )both on iperf3 and SMB traffic.
When I did the test over actual WAN with 300/300 uplink and 600/30 downlink over different ISPsI had even worse, with 150-ish Mb/s (150 upload - 6 download). (12-15 kpp/s).
In LAN I can achieve full 1 Gb/s.unencrypted with mikrotik 1 doing routing.
Both ccr1009 doing encryption use hardware acceleration (at least they say so in installed SA's). On one of them (the 8 port "older" version) CPU load is split on 2 cores from what I've seen, load is around 90% qualified equally as "networking" and "firewall".
As for firewall I used RAW no-track for local subnets that are being involved in the tests and first rule in filter is just ACCEPT for the interfaces.
2) scenario 2:
windows 10 client using ikev2 -> mikrotik ccr 1009 as VPN server -> mikrotik 1009 as router/firewall -> file server/iperf3
client has 600/30 and server 300/300
With various available speedtests I'm getting around 200 Mb/s, with iperf 150-ish and actual file download over SMB is around 14-17 MB/s.
It's roughly 60-65% of max WAN upload (300%) so there's still plenty of room left. In LAN (so client - mikrotik 1009 doing just firewall/routing - file server) i'm getting perfect 1 Gb/s,
Now my general questions:
Should I be looking/vetting my config for a possible flaw/misconfig, or is just it - CCR 1009 is unable to do over 200Mb/s in single IPSEC tunnel? I don't want to run a fool's errand and look for issues where there might be none.
What I would love to hear from someone with ccr 1009 experience is wether I should be technically getting better throughput, or not? The only thing that makes me wonder is that CPU (specific cores) on 'tiks never hit 100%, so it'd look as there is still some potential. THe highest I've seen was around 70% on single core (35%-50% as "networking" and the rest is firewall).
TL,DR
What is the expected (in real production environment) single tunnel Mikrotik CCR 1009 IPSEC throughput (IKEV2 plain and with tunnel protocolos like IPIP or GRE) and is my ~220-ish Mb/s close or way, way behind what I should be getting?
Regards