I've got issues with a CCR1009 on our network edge which maxes out at around ~500mbps. CPU Load is usually around 5-10% so it doesn't appear to anywhere near maxed out.

On the router there's OSPF, PPPoE Server with ~10 clients, DHCP server and two hotspot servers on separate VLANs. Firewall is fairly simple with ~80 rules (inc hotspot) and 3 SRC-NAT rules.

Currently running 6.44.3 but older versions have been tried as well. Does anyone have any ideas what is limiting the throughput?

How are you testing this limit?

That's both real world usage with 1000+ users and Btest server.

Rather test via IPerf, using a powerful PC on either side of the router. You might be maxing out the CPU using btest.
The CCR1009 should be able to push more than 500Mbps.

I have previously done that with similar results. It is possible to push to about 600mbps with absolutely no other devices connected through the router and with a high stream count. I can guarantee though that the testing methods are not the issue. Something causes a bottleneck at about 500mbps. I can find other references on the forums as well but no real solutions. I've enabled fast track today but there's no measurable difference.

CPU Load is usually around 5-10% so it doesn't appear to anywhere near maxed out.

Always make sure you check the CPU load in Tools->Profile and checking separately per CPU (selection "all").
The CPU load reported by default is the total for all CPUs so in this 9-core system when 1 core is 100% loaded and the other 8 are almost idle the reported load will be 11% but the system may still be CPU limited.
In that case, find what is saturating that single core and see if you can reconfigure the router to avoid that single-thread bottleneck.
(there are some techniques for that, depending on what it is)

Always make sure you check the CPU load in Tools->Profile and checking separately per CPU (selection "all").

That is what I am using. Screenshot attached. Currently about 200mbps going through this router.

Ok so it is considerably more loaded at 200 Mbps than you said earlier!
At 200 Mbps it is like 28% loaded so it is not surprising you get problems at about 500.
But the good thing is that the load appears to be reasonably distributed across the cores.

Click twice on the Usage column to get the top usage and work from there.

The CPU load there is actually at the higher end of what I've seen. This screenshot is a lot closer to normal and overall is steady at 5%.

But that was probably at a lighter load?
Also understand that when you run a connection-tracking firewall your load will be a lot higher when you have a lot of new connections.
It can easily be that the router passes 500 Mbps of continuous data with almost no load, but cannot handle 500 Mbps worth of new connections e.g. resulting from a port scan or DDoS.
On a network edge it is often not a good idea to have a connection-tracking firewall in the forward path. You may want to have it on input/output only.
It all depends on the use case.

Try to do the profile again when it has the loading similar to the previous screenshot and see what is the next load below "firewall", maybe it provides some hint what else could be loading it.

Can this be improved with a config change on the router then?

That is impossible to know because you did not include the config. However, you can research that yourself as well.
There are a couple of common mistakes that can take a lot of performance, you can find some MUM presentations about that in the MikroTik YouTube channel and also on the forum.
When you have analyzed a bit more where the CPU load is, you can focus on the parts of the config that need improvement.

I've attached the firewall config if you want to have a look. Based on CPU usage this is a firewall issue despite it not maxing the CPU at 100%. I've spent a lot of time researching possible causes but not got anywhere. I can find references to similar limitations going back years but not really any resolution.

There are some points from those presentations that you can apply:
- do not use Masquerade for any other cases than src-nat on a dynamic address (e.g. typical home router)
instead use a src-nat with (one of your) external address as a to-address.
- it looks like you do not completely understand the chain=input and chain=forward difference.
there are many rules on chain=input that make no sense, fortunately many of them disabled. you can clean this up.
even for a dst-nat you don't put things in chain=input, but only in chain=forward.
- when you have one specific path where you want to block a lot of different things, but do not want to bother the
router to do those checks on all traffic, consider putting those rules in a separate chain. put a rule in chain=forward
which matches the input interface (or -list) and then jumps to your custom chain (action=jump jump-target=forward-incoming)
and then put your long list of rules in chain=forward-incoming. this can also make things a lot clearer.
however, in a connection tracking setup that makes less of a difference because you accept established,related immediately.

Thanks for the advise so far.

- do not use Masquerade for any other cases than src-nat on a dynamic address (e.g. typical home router)
instead use a src-nat with (one of your) external address as a to-address.

The main IP range is using src-nat. The two hotspots which have very low usage are using masquarade. There's no particular reason so will change those.

- it looks like you do not completely understand the chain=input and chain=forward difference.
there are many rules on chain=input that make no sense, fortunately many of them disabled. you can clean this up.
even for a dst-nat you don't put things in chain=input, but only in chain=forward.

Are there any specific examples? Most of the rules are from Mikrotik recommendations. I assume the

- when you have one specific path where you want to block a lot of different things, but do not want to bother the
router to do those checks on all traffic, consider putting those rules in a separate chain. put a rule in chain=forward
which matches the input interface (or -list) and then jumps to your custom chain (action=jump jump-target=forward-incoming)
and then put your long list of rules in chain=forward-incoming. this can also make things a lot clearer.
however, in a connection tracking setup that makes less of a difference because you accept established,related immediately.

That's something I hadn't spotted. A lot of the config has been imported from various templates we use. It looks like at some point some of the rules that should be on their own chain have ended up on the input chain. Will get those changed.