Im not super well adversed in network design and am looking for advise how to setup this following network diagram (diagrams.net link)
Couple of remarks:
- Each site / router RN is connected with each other through a (partial) mesh wireguard setup
- The routers will be a mix of routeros and linux devices
- The actual network will be >3 but probably <10 routers, I used only 3 in the diagram to show a minimal point to multipoint setup. The <10 might be interesting to show that some static configuration on each router through f.e. ansible should still be manageable
- The 10.N.10.0/24 and 10.N.11.0/24 networks should be able to reach their counterparts on the other routers. So 10.1.10.3 should be able to reach 10.3.10.4 but not 10.3.11.4 (indicated by the blue and green arrows)
- That said, each router should be in control of what traffic is routed to which subnet. So it should also be possible that if there is also a network 192.168.0.0/24 behind R1, that 192.168.0.4 has only access to 10.2.10.2 but 192.168.9.5 can only access 10.2.11.3.
- I thought it was best to achieve the previous mentioned separation by adding the 10.10.10.0 and 10.10.11.0 networks on top of wireguard, this way R1 is free to SNAT any traffic to 10.10.10.1 and R2 and R3 just need to forward any traffic from 10.10.10.N to 10.N.10.0
- Each router provides access to 0.0.0.0/0, but it should also be possible to add custom static routes to f.e. route traffic from 10.1.11.5 behind R1 to 0.0.0.0/0 through R3 (the red line in the diagram)
More thoughts:
- As said I would prefer to NAT traffic behind each of the subnet’s, so this would probably rule out OSPF? This might be a bit security through obscurity, but afaik with OSPF each router would need to be fully aware of the network topology behind the other routers and I would like to prevent that.
- I looked into vxlan’s but it seems that Mikrotik doesn’t support multipoint vxlan’s like described here? Cause the docs I’ve found about multipoint vxlan’s all seem to add static records to the FDB table but this doesn’t seem possible on routeros. Is that right?
- Instead of multiple multipoint vxlan’s, could it be better to do multiple vlan’s over a single vxlan over wireguard?
- I don’t necessarily need multicast traffic support between the 10.N.10.0 and 10.N.11.0 counterpart networks, so maybe a vxlan is overkill? But then what, GRE tunnels (but those are only ptp not ptmp)?
- Or would it be better to instead of doing multiple v(x)lan’s over a single wireguard to just have multiple wireguard connections between the routers?
Does the above make any sense at all or am I overlooking something (obvious) here? The main issue I seem to be stuck on with the vxlan over wireguard approach is how to create a multipoint vxlan in routeros when you cannot add static FDB table entries. But any other/better suggestions on how to create this topology would also be great 