I have two geographically-distant WISP networks connected by a common WAN. My Network Control Center connects to the same WAN, but is not on the private side of either WISP network (see figure).
VPN channel into the private side of my MikroTik. It worked just fine.
When my network was split in two, obviously I lost the ability to communicate with the CPEs in neighborhood B. To regain it, I followed the instructions at http://www.mikrotik.com/testdocs/ros/2. ... e/eoip.php to create an EoIP link between the two MikroTiks. Although I realized it was a slow path, it was only for monitoring and occasional configuration, not general data traffic. It worked, in principle, but had serious performance side-effects.
Here is the problem. Every CPE we already have installed is configured to have a gateway address of …1.1. (It also makes equipment replacement easier if this value doesn't vary depending on exactly who the CPE is going to.) So both MikroTiks are configured to be …1.1 on their own network.
When I created the EoIP link, I knew I was cheating on normal network configuration rules. I assumed that each router would not go looking to the other side of the link for an address unless it could not be found on its own side first. I hoped that each router would fulfill its own addressing requests for …1.1 and never bother to look across the EoIP link and notice that address had been duplicated.
I was not clever enough to cheat the router. Whenever I establish the link, all traffic in neighborhood B immediately routes itself through neighborhood A's router before leaving the WAN. (I know this because traceroute shows A's public address instead of B's.) Amazingly, this all works, but the implications on performance are horrendous in both neighborhoods.
I tried playing with ARP tables and diddling the priority of the bridge port for the EoIP connection in an attempt to make inter-router traffic an absolutely last-ditch resort, but I was unsuccessful at outwitting the router.
Does anyone have a better idea of how to make this work, short of changing every CPE in one neighborhood to a different and unique gateway address?
In order to monitor and configure customer premise equipment (CPEs), back when this was all one network with one WAN feed, I created a