EoIP with Multiple WAN

RouterOS General
1

Hello,

I have been trying to solve a problem with EoIP for a couple of months.

I have a Mikrotik router (CCR1009-7G-1C-1S+; ROS: v7.12.1) with 3 ISPs, IP addresses: x.x.x.x, y.y.y.y and z.z.z.z. let’s call this router: “Server”;
And have two Mikrotik routers (hAP ac^2; ROS: v7.12.1). Let’s call them “Client1”(IP: a.a.a.a) and “Client2”(IP: b.b.b.b).

I think the scheme should be self-explanatory. So the scheme is:
Mikrotik EoIP Problem.png
Problem:
The client connects to the server through ISP’s that are not the default route of the server itself. So the problem I am seeing is that server all the time responds to requests from a default route. I tried to mark packets/connections and set the routing mark but it does not work (although the packet marking is working correctly because all incoming packets that are forwarded to the local network from not a default route are routed back correctly).
Take a look at this scheme:
Mikrotik EoIP Scheme.png
How to force EoIP to respond through the same WAN from which the request was received?

Thanks a lot for any help. I truly appreciate your time and insights!

2

In your case it’s simpler just use two /32 static routes

We need to make two new terms which are the gateways for WANs
WAN y.y.y.y gateway is y.y.y.gw
WAN z.z.z.z gateway is z.z.z.gw

So simply specify a /32 route for the EOIP tunnel traffic

/ip route
add dst-address a.a.a.a/32 gateway=y.y.y.gw
add dst-address b.b.b.bv/32 gateway=z.z.z.gw

The basics here are that only the tunnel traffic itself actually goes to the remote public IP. All the site to site traffic is going thru the tunnel which does not involve the public IP. Your tunnel traffic itself now has no option but to leave via WAN you specified.

It’s a trick you often use when you want to form a tunnel and have the default route go thru the tunnel when you would do something like this

/ip route
add dst-address 0.0.0.0/0 gateway=tunnel.gw 
add dst-address a.a.a.a/32 gateway=y.y.y.gw

Everything in the above goes thru the tunnel except the tunnel communication traffic itself.

3

Thanks for your suggestion,
your solution does work, but the problem arises when the Client tries to connect from its local network to the “Server” on IP x.x.x.x.
For example, a local client on “Client2” that is connected through LAN_L3 tries to access the server that is located on “Server” IP x.x.x.x. Even Ping to “Server” IP x.x.x.x does not work.

So not all traffic that is coming from Client IP b.b.b.b should be routed back from Server IP z.z.z.z.
In my opinion, if I set local IP:

/interface eoip add local-address=z.z.z.z remote-address=b.b.b.b tunnel-id=18 name=eoip2

router should obey this rule and use local IP for outgoing traffic.


This is a bug in firmware. Don’t you agree with me?

4

First why in gods name is a remote client connecting to a server on x.x.x.x that is a public IP … the whole point of the tunnel is to stop having to expose the network to the public IP. The solution works because it is simple and so lets continue along the simple solution path.

The blind freddy obvious safe solution is get the remote clients to access whatever server has it’s head out on x.x.x.x to have one on the inside and clients use that when connected. Worst case at the moment you limited inter-connection between a.a.a.a, b.b.b.b to stop remotes having to use external IP.

For example this often comes up with a mailer so you have a mailer called mail.mycompany.com
On the normal public DNS systems you make mail.mycompany.com = x.x.x.x
However on the LAN DNS systems you make mail.mycompany.com = a.a.a.a
In many cases this is just having an IP and ethernet and server connected to the two networks or worst case a couple of static routes
So when staff take there laptop from home to office there email automatically selects the right IP when they get there DHCP.

You took the time to draw out the diagram now perhaps take the time to think thru the security issues. You are trying to minimize the external weaknesses not make complex routes that make more.

5

By showing the scheme I tried to simplify the problem. But from your response, I understand that I simplified too much…
Let’s make it clear:

  1. The scheme that you see is much simplified.
  2. “LAN_L1” is the network used for specialized hardware. No user is allowed to connect to this network.
  3. “Server” has multiple networks not shown in the scheme.
  4. Thousands of users are connecting from phones, tablets, PCs - depending on the situation some use VPN, some just direct connection. I will not go deep into the actual architecture - let’s just say that it should be like this and I can’t change it.
  5. “Client1”, “Client2” … “ClientX” it’s just the points that are special hardware located. And those points are much fewer than the users.
  6. And yes the security is the main reason why this architecture is so complicated.

Sorry, I can’t show you the full picture.

You took the time to write such a long response (without a solution) instead of just saying that it’s impossible to do that with EoIP.

Anyway, thank you for your time!

6

If an EOIP tunnel is established and it is bridge enabled, then no IP addresses are used between LAN1 hosts, they are in the same broadcast domain and frames go to MAC addresses. So you have an error in the scheme. For simplicity of understanding EOIP is a cable that connects LAN1 bridges into one network at L2 level. The tunnel does not participate in routing of L3 traffic. Therefore, problems with routing between networks with different IPs are in another area (firewall routes, etc.).
attachment=0]Mikrotik EoIP Scheme3.png[/attachment]
Mikrotik EoIP Scheme3.png

7

Sorry, I did not understand what you meant.
Internally EoIP works correctly, I have no issue with LAN_L1 at all, and the hardware I use is communicating in proprietary MAC-level protocol flawlessly. The problem is that EoIP traffic goes through the wrong WAN ports as you can see from the second scheme I posted.

8

Then you need to configure multiwan that the local traffic of the router itself goes to the wan port from which the request came.

9

Yes, but how can I do that?
I tried to mark and then route the output to GW that it came from, but this is not working for the internal local traffic of ROS. Although It does work for DNAT.

10

You’ll have to figure it out for yourself. Because you have already configured multiwan on the router, it depends on these settings how you need to configure it. Otherwise my advice can break the whole work of the config. So look on the forum threads about multiwan there these settings are considered.
Or post the config, maybe some of the forum members will help.
PS http://forum.mikrotik.com/t/multiwan-with-routeros/163698/1

11

If you want security you need to keep things simple and you my friend are down to relying on complex packet marking and if that doesn’t raise hairs on the back of your neck it should. Some poor sucker has to maintain this if you leave or get hit by a bus.

The obvious answer is spin up a third network on the server network and connect the two client remote sites to that network so you have the simple route rules call it c.c.c.c on the server. In reality c.c.c.c could be a subnet inside a.a.a.a and/or b.b.b.b if it makes it easier for you to visualize. Place the two routes per above on that network and it doesn’t interfere with the other direct shit you are getting hung up on and you can leave a.a.a.a and b.b.b.b networks alone.

The new network c.c.c.c can reach a.a.a.a and b.b.b.b natively there gateways are on the same router and if there is stuff you don’t want access you LAN to LAN firewall that down. The reason you do that is the traffic and routes are blind freddy obvious and that makes it more secure. Now your changes to make your remotes sites work does not remotely affect the existing direct network and simplifies things.

It is called breaking up a network to make it secure but you seem hell bent on going about it the most insecure and weird way and waving hands and saying it’s really complex.