The 100.64.0.1 Is the address of each star link, i.e. you have - more generically - four routers/modems all with the same address?
I used for a similar ( but different ) setup a “reversed” VRF, in my case the scopes were failover and the possibility of removing altogether the Mikrotik device ( that has the same address as the routers/modems) connecting directly to one of the routers without changing anything in the network configuration.
You could try a similar approach, and then change the failover setup into load balance.
The thread is long, but contains GNS3 schemes and reproducible configurations, here:
There may be other ways, without needing a VRF, if you can use a different IP address for the Mikrotik.