Alright, so I had a couple of hours to kill yesterday and decided to look into this. First of all, I got it working! Read on to find out how.
After your comment emils
I set "send initial contact" to no on both client and server (not sure if you meant client and server or both peers
in the ipsec menu. I had already tried it on the client side only. The ipsec connections now connected simultaneosly though with some issues (mainly with l2tp). So I decided to research the wiki again and found the two relevant parts.
send-initial-contact (yes | no; Default: yes) Specifies whether to send "initial contact" IKE packet or wait for remote side, this packet should trigger removal of old peer SAs for current source address. Usually in road warrior setups clients are initiators and this parameter should be set to no. Initial contact is not sent if modecfg or xauth is enabled for ikev1.
This explanation is not abundantly clear to me. The "this packet triggers the removal of SAs for current source address" part makes sense. However it shouldn't apply to my scenario as I have two different IPs on each side. Unless of course the second ipsec connection is going out the same interface as the first, right? The rest makes no sense to me. "In road warrior setups the clients are the initiators" (of course) "and this parameter should be set to no". Why? Unless we're talking 5 guys on laptops in some hotel all connecting from the same IP, it'd make sense to kill old SAs for that peer? In any case it doesn't say if it should be set to no on the server or on the client. It seems to imply the server. But this would also imply that one should leave it on client side.
In any case, based on the above I started to suspect a routing and mangle issue so I kept digging. I also expected that setting the local-address parameter would make the connection use the correct interface.
IPsec, as any other service in RouterOS, uses main routing table regardless what local-address parameter is used for Peer configuration.
This was the final piece of the puzzle. My mangle rules were not set correctly and since you can only have one default gateway up in the main table (the others are shown in blue in winbox) I decided to skip using mangle and use routing rules instead. I created two routing marks and then two rules, each with dst-address set to one of the main office's IPs and action set to "only lookup in table" to its corresponding mark. After this everything's working as expected.
At this point I'm not sure if the routing, the "send initial contact" setting or both were the problem. And I don't want to reenable send initial contact as I'm remoting in and I might lock myself out.
edit: typos. Hit submit instead of preview by mistake.