Which is best a single hardware firewall in front of two CCR1009 or use the firewall on both CCR's?

It depends on your needs.
Do you need stateful failover? Do you need DPI? Do you need address collection in the firewall to do further things with on the CCRs? Do you need application control?
Is firewall latency an issue? Do you need advanced logging facilities? Do you want it cloud managed? Do you need threat protection? What's your budget?

Anything from using routerOS integrated firewall over a dedicated Linux vm, pfSense, CheckPoint, Fortigate up to Palo Alto is possible and basically only depends on your needs, expectations and budget.

There is no ballpark perfect firewall.
-Chris

As it stands I believe that the firewall settings have to duplicated on both CCR's and if not done then there is no failover, so I thought have a hardware firewall in front of the CCR's,

You are correct about the need of duplicating settings, but that counts for every setting, not only Firewall. And once created and proven good, I consider a firewall rather static... And with some scripting you could automate the replication to the other peer.

When getting one firewall in front of the pair, keep in mind that you add another SPOF (unless you deploy a HA cluster of at least two devices) and you certainly add another layer of complexity.
If you still feel like doing so, I personally like Fortigates and Palo Alto PA-series as firewalls. They're pretty easy to configure, fast and resilient. And they offer a decent stateful failover when deployed in clusters. Not really cheap but worth the invest, IMO.

-Chris

There is a mikrotik High Availability setup somewhere on github which deals with synchonisation of configuration between two machines (so changes made on the active are mirrored to the standby), however there is no method of synchronising firewall state (more precisely, the state of connection tracking which is the only stateful part of the firewall), i.e. when the active machine fails and the standby one takes over, some connections get broken and have to be re-established.

@cdiedrich, do you know how HA clusters/pairs of Fortigate and PA behave in this regard?

@cdiedrich, do you know how HA clusters/pairs of Fortigate and PA behave in this regard?

I do - absolutely seamless. All connections are always in sync. Dealing with those as my daily business...

....................
When getting one firewall in front of the pair, keep in mind that you add another SPOF (unless you deploy a HA cluster of at least two devices) and you certainly add another layer of complexity.
...........................

I agree it's another single point of failure but i guess there is less chances of a hardware firewall failure?

I agree it's another single point of failure but i guess there is less chances of a hardware firewall failure?

That's a misconception. It's built from the same components: power supplies (failure #1), fans (failure #2), semiconductors, physical connectors, HDDs/SSDs, etc that are all subject to the same wear and tear.
The benefit of choosing those appliances will definitely be their SLAs...

In case you already have a HA virtualization environment in place, it might be worth considering virtual appliances of the bementioned and protect the single instance through the hypervisor's HA techniques. For the cost of probably added latency as the vms don't have the custom ASICs the hardware has.

-Chris