I have two geographically-distant WISP networks connected by a common WAN. My Network Control Center connects to the same WAN, but is not on the private side of either WISP network (see figure).
In order to monitor and configure customer premise equipment (CPEs), back when this was all one network with one WAN feed, I created a VPN channel into the private side of my MikroTik. It worked just fine.
When my network was split in two, obviously I lost the ability to communicate with the CPEs in neighborhood B. To regain it, I followed the instructions at http://www.mikrotik.com/testdocs/ros/2.9/interface/eoip.php to create an EoIP link between the two MikroTiks. Although I realized it was a slow path, it was only for monitoring and occasional configuration, not general data traffic. It worked, in principle, but had serious performance side-effects.
Here is the problem. Every CPE we already have installed is configured to have a gateway address of …1.1. (It also makes equipment replacement easier if this value doesn’t vary depending on exactly who the CPE is going to.) So both MikroTiks are configured to be …1.1 on their own network.
When I created the EoIP link, I knew I was cheating on normal network configuration rules. I assumed that each router would not go looking to the other side of the link for an address unless it could not be found on its own side first. I hoped that each router would fulfill its own addressing requests for …1.1 and never bother to look across the EoIP link and notice that address had been duplicated.
I was not clever enough to cheat the router. Whenever I establish the link, all traffic in neighborhood B immediately routes itself through neighborhood A’s router before leaving the WAN. (I know this because traceroute shows A’s public address instead of B’s.) Amazingly, this all works, but the implications on performance are horrendous in both neighborhoods.
I tried playing with ARP tables and diddling the priority of the bridge port for the EoIP connection in an attempt to make inter-router traffic an absolutely last-ditch resort, but I was unsuccessful at outwitting the router.
Does anyone have a better idea of how to make this work, short of changing every CPE in one neighborhood to a different and unique gateway address?
what about putting neighborhood B on .2.x, a different subnet and then just route them. EoIP should be used as a last resort in my opinion. Routing is always better than bridging (in most cases). I know its probably a big change to the way you are doing things, I just wanted to throw it out there to digest.
That doesn’t really seem to address my problem. First, it still requires me to pre-configure CPE radios differently for the different neighborhoods. Second, it doesn’t give me a path by which I can ping and communicate with CPEs on the B router from the A router. Unless I am misunderstanding your suggestion.
Yes, both neighborhoods are using the same subnet. It used to be all one single network, but was recently split because we lost the connectivity path we had between them, so we had to buy an extra WAN feed. Chances are slim of me even reaching 100 subscribers at these locations, but if I ever got to 200+ I would be happy to redesign.
I don’t understand why an L2TP tunnel is going to work any differently than an EoIP tunnel. Could you explain? The address routing seems to be the problem, not the particular transport mechanism. The EoIP tunnel “works,” it’s just that when it is enabled, the router starts pushing traffic over it that I never intended.
I’d even be happy with some sort of efficient mangle strategy in the neighborhood B router that performed the equivalent of, “all traffic coming in for …1.1: thank you, just NAT out the WAN port right here, shut up, and be happy.”
an eoip tunnel and bridge will act just like a switch… so you are esentially plugging all your clients, and your routers into the same switch. you cannot have duplicate IP addresses obviously.
a l2tp tunnel can be used to route between the two subnets, but that would require you to use routing instead of bridging.
if you truly want to keep the same subnet across both neighborhoods (bad idea) you can, but you will need to give that router a different IP address and reconfigure clients. or use vrrp, but thats more complex than you need.
i suggest spending the time to fix this now before you bust at the seams later. just give the second neighborhood a .2.1 range and use l2tp between the two. if they aren’t on the same physical lan then don’t give them the same subnet.
