Have you ever done that? Can you explain to me how you did it?
Of course. Otherwise I wouldn't talk about it :D
One of my current setups is following:
https://app.diagrams.net/#Uhttps%3A%2F% ... 3Ddownload
https://drive.google.com/file/d/1pqnKtG ... sp=sharing
I included only config relevant to this EoIP/IPsec. I did not include config related to LAN or management networks as well as IPsec config related to Site "P" (its just ordinary config compatible with cisco's requirements and will be soon replaced by another EoIP once i get more mikrotiks)
Notice that each site requires bit of "preparation" - create the mesh, assign IP and add firewall rule to accept everything coming from VPN.
Then, for each tunnel, there is relevant config which will create IPsec peer and policy, EoIP tunnel, port to mesh, firewall rule for outgoing data and route.
Now, two things which may be hard to understand:
why do I run EoIP when it creates additional overhead? Well, because then my policies are simple enough to avoid accidental misconfiguration. All internal data flows through normal routing process into EoIP or Mesh.
Why Mesh and what the heck is it? Another overpreparation. Right now, I can send data from Site "R" to "T" and it will just work because mesh will take care of it. (i already do that for management networks). If the amount of data is large enough, I will just create another tunnel directly between R-T so data does not flow through "C". In addition, I can have more redundant EoIP tunnels (e.g. backup mobile connection, direct WLAN etc) and it will create one big L2 network.
But why mesh and not bridge? Well, bridges don't like loops. They create spanning tree topology to prevent loops. With Mesh, loops are not an issue because mesh runs path discovery and transfers the data in a shortest possible way.
Is it ideal? Far from it. Does it work?
Oh yeah baby!
EDIT 2020-10-12 :
Alrighty, I was playing bit more with it and I realized that Mesh's reliability depends highly on keep-alive function of the EoIP tunnels. That's why it worked excellent in the lab, but may not work as expected in the reality. My mistake was, that I was testing the fail-over by disabling EoIP interfaces. Disabling or changing running state will trigger instant response and that works great. However, trouble is, that the running state may not be changed instantly (depends on keep-alive settings, which by default takes 10*10 seconds and that is unacceptable) if the encryption or physical link fails.
I still believe there is something beneficial in running a simple site-to-site policy for symmetrical virtual interface (GRE/EoIP/IPIP/WireGuard) instead of "simple" policy for all data, because you separate tunnel and routing. (which makes it less prone for mistakes) However, I will no longer claim this is a bulletproof solution or that it has instant fail-over capability. The fail-over still happens but it is not instant and there may be packet loss.