I have now tried forwarding the WireGuard Port to my Internet fallback Mikrotik router instead of the main one and try WireGuard there. I used the same configurations with the same allow rules etc. and I got it working immediately without any issues. Access to internal networks was immediately possible, forwarding to internal LAN required some mangle rules because the default gateway for the internal LAN devices is not the fallback router but once I did so even those worked.
Needed to forward the traffic as the fallback internet is not on an open public IP, so I cannot access the router from it.
On my main router I was not even able to ping the router itself.
So I guess this might be a hardware specifc issue.
My Internet fallback router is a: RBwAPR-2nD (MIPSBE)
And my main router is a CCR1009-7G-1C-1S+ (TILE)