I spent most of yesterday chasing this and I think it is worth writing down, because I could not find a single thread about it and I lost weeks assuming the hardware was just old.
Setup: CCR1009-8G-1S, the classic 9-core TileGX box, used as a WireGuard hub. One of the sites behind it started complaining in June — "the VPN got slow, it used to be 400". I did not believe it at first. The router has a big firewall, a 20k+ entry blocklist address list, a lot of tunnels, and my first instinct was that we had simply outgrown the box. Turns out no.
What finally made it measurable: I stopped testing through the customer's path and built a clean two-point test instead. One WireGuard tunnel from the CCR to a Linux box in a datacenter, kernel WireGuard on the other side, 28 ms away, nothing else in the path, no-queue on the tunnel, plain /interface/wireguard config, three lines. iperf3 with a fixed UDP rate, and I watched both /system/resource/cpu/print and /tool/profile cpu=all while it ran.
Then I rolled RouterOS backwards on the same box, without touching the configuration at all. Same tunnel, same peer, same everything.
Direction where the CCR encrypts (LAN -> tunnel -> remote), 300 Mbit/s offered UDP:
- 7.21.5 — 300 Mbit/s delivered, 0.06 % loss
- 7.22.3 — 139 Mbit/s, 53 % loss
- 7.24.1 — 118 Mbit/s, 60 % loss
At 500 offered: 416 on 7.21.5, 105 on 7.22.3. TCP with 4 streams and BBR: 368 vs 125. I ran everything twice, different times of day, same result.
Direction where the CCR decrypts is identical on all three versions — around 190-260. So it is one-directional, which is the part that confused me for a long time, because "the tunnel is slow" from the user's side is never one-directional in their head.
What I checked before blaming the version, since I assumed I had broken something myself:
- Fasttrack was active for the inner flows in every run. Adding
rawnotrack rules for the outer UDP changed nothing. - The blocklist. I ran 7.21.5 with an empty address list, then added 20224 dummy entries and ran again — 300/416 vs 299/420. Not it.
- Hardware. Same box forwards ~868 Mbit/s of plain routed traffic, CPU 47 C, and again, decrypt is unaffected.
- Both endpoints do 700+ Mbit/s without the router in the middle.
- MTU 1420, 1350-byte payloads, no fragmentation anywhere.
The interesting bit from /tool/profile: ChaCha20-Poly1305 itself is only 3-14 % of each core. One core sits at 100 % in both the good and the bad version — but on 7.21.5 it saturates at ~300-400 Mbit/s, and from 7.22 onwards it saturates at ~120-140. So the crypto is not the expensive part. Something in the per-packet path around it got roughly three times more expensive.
Because I did not want to hand MikroTik a "your WireGuard is broken" report based on one box, I ran the same three versions on an x86 CHR (20 vCPU) with the same tunnel to the same peer. There the link, not the CPU, is the limit, so instead of speed I compared CPU cost at a fixed rate: 7.21.5, 7.22.3 and 7.24 all land between 1.5 and 3.0 busy cores, ranges overlapping. No difference at all. Which is why I suspect this lives in the tile-specific part of the data path rather than in the WireGuard code everyone shares.
I have opened a bug report with support (SUP-222238). Meanwhile I am staying on 7.21.5 on that box, which is not free — no ACME on 7.21.x, so certificate renewal is manual, and a couple of my scripts use :break and refuse to run. Trade-off I can live with for now, but not forever.
The reason I am posting: if you run a CCR1009, CCR1016, CCR1036 or anything else on tile as a WireGuard server, and you noticed your upload through the tunnel getting worse somewhere around spring, it may not be your config and it may not be your ISP. Try one clean tunnel with a fixed UDP rate before you go looking for a bigger router. And if you have already downgraded and it fixed things — please say so here, one data point from one box is thin, and it would help support a lot to know how wide this is.
Happy to share the exact test commands and the per-core profile output if anyone wants to reproduce it.