I currently use a HeX PoE and have it powering a HAP AC3 from port 5. It takes 24V and runs the HAP fine with the port set to "forced on".

I am working on migrating to a RB5009 and noticed that it only outputs over PoE what it receives, so in this case 48V from the provided AC adapter. The PoE port would report "voltage too high" on the interface with the HAP connected.

I got a RBPOE-CON-HP PoE which is supposed to convert the 48V to 24V, which should make the HAP AC3 happy, but it doesn't seem to come up. Even sent to 'forced on' the interface just flaps, and eventually doesn't come up at all. I was afraid it might have fried it, but failing back and connecting everything to the HeX PoE and it was fine again.

Any advice on how to power the HAPAC^3 from the RB5009?
Do I need to just use a separate injector and ignore the rb5009 poe-out?

Never mind! I just had to set it to "forced on" and let the RBPOE-CON-HP stabilize before connecting a load. All good now!

All good now!

... until the first power outage. It seems as if the total of the inrush current of the hAP ac³ and the own inrush current of the RBGPOE-CON-HP was too high for the RB5009 to provide, so it detects an overload and shuts the PoE down.

One idea here would be to add a relay to the RBGPOE-CON-HP output that would only connect the load once the RBGPOE-CON-HP itself is up, but such an approach has a number of drawbacks - if the RBGPOE-CON-HP checks whether a "passive PoE" PD isc onnected before sending power out, the relay would break this detection, plus the relay would consume some power for its own coil, plus it is not a ready-made product. The plus of this approach is that it would allow to power-reset the hAP ac³ by cycling the PoE output on the RB5009.

If I had this issue, I would use a passive PoE injector and either feed it from a 24 V power adaptor, or insert it between the RB5009 and the RBGPOE-CON-HP and feed it from the 48 V power adaptor that feeds the RB5009.

Part of problem is that RB5009 is not really 802.3 at compliant (which specifies up to 30W output from PSE) as it only allows for 20W per port (0.42A at 48V). Making it an 802.3 af PSE (whose specs slightly exceed 802.3 af standard).
On the other hand RBGPOE-CON-HP is supposed to be af/at compliant, but output spec (1A at 24V equals 24W) hints that it should be 802.3 at PD, specially so if one adds own power consumption (due to inefficience of DC-DC voltage conversion).

Meaning that RB5009 is actually unfit to serve as PSE for RBGPOE-CON-HP.

I'm seeing the PoE overload on a RB5009UPr+S+OUT even with a SXT 5 ac - without converting to 24V! The SXT's datasheet - like the hAP ac³'s - specifies its max power consumption as 12W.

Let's see what support has to say about that (SUP-120584)

I'm seeing the PoE overload ...

Depending on UTP cable quality there will be power loss and Power Sourcing Device (RB5009) will have to cover that as well. With short cables (less than a few meters) this is usually not a problem, with longer cables it can be a deal-breaker.