Do you configure all the routers? Or at least have detailed info how they are configured? That is very important for such a setup to succeed.
Configuring IPsec while treating the other end as a blackbox is very difficult and time-consuming!
It can be done but it will not be a beginner’s job…
Or of course you could be lucky and it could work 1st try.
My recommendation: configure GRE over IPsec transport, not plain IPsec tunnel.
Configure each GRE tunnel with a /30 network address from some unused range.
Use static routes or some routing protocol. (BGP, OSPF)