We are working with a small local HOA ISP. they have have two separate service providers AT&T and Cox with small business services.
We are tunneling them Static IP's from out DataCenter. We are experienced with Mikrotik, but running into issues with building this connection. and thinking that the Tik may not be able to handle all of the services we wish to configure on the CCR2009. We do have static IP's from the service providers, but are using our IP blocks for origination and termination of all traffic so the SP IP's will not be seen from the customers side.
We were trying to use OSPF as the failover however the origination of the routes when you switch over needs to swap in order to not be blocked by AT&T's ip origination filters.
The goal here is to provide service with failover between the providers (Cox is a 1GBx1GB service AT&T is a 300Mb/300MB service so we will preference the COX service.
I should mention that we are running two tunnels from each provider one primary and one backup two our two datacenter locations .
so assuming our configuration skills are up to par we have tried the following things:
1:) We have configured 2 VRF's on the router + the main routing table tried to put each carrier in a vrf and then export only a default route from each with a distance to preference COX to the main routing table
2:) Tried to build tunnels direct tunnels from two RB5009 routers one on each provider in a nat block and then statically route the CCR2009 with distances to the preferred and less preferred router
3) tried direct tunnels from the CCR to the datacenter (and run into spanning tree issues with this config)
part of the issue is the different configurations seem to break the hotspot configuration or the traffic flows and then stops hours later for not good reason. We do notice that the COX tunnel has a different tunnel MTU 1476 vs ATT 1452 as calculated by the tik. we have also tried all three GRE/EOIP/WG tunnels on all configs and still an issue,
This shouldn't be that difficult to configure but perhaps i am overthinking the problem and i have missed something somewhere. FWIW i didn't configure there network, that was a Community Effort (so could be they clicked buttons till it worked or ????)
Any thoughts/ideas would be appreciated.