help with a wierd routing issue (loop maybe?)

RouterOS General
1

Here is a summary of my setup:

I have a MT running ROS 2.9.11 with 3 interfaces:

Ether1: 192.168.69.1 (LAN)
Ether2: 68.68.68.68 (ISP1, Default Gateway)
Ether3: 158.158.158.158 (ISP2)

I have masquerading and connection/route marking set up so that all traffic that comes in Ether3 gets routed back out Ether3 (traffic on Ether2 goes back out Ether2 since it is the default gateway). I also have dst-nat rules setup to forward traffic coming in Ether2 and Ether3 to local servers connected to Ether1. All of this is working correctly.

Now here’s the problem I’m having. From Ether1, I can ping both Ether2 and Ether3. However, if I try to connect from Ether1 to any dst-nat’d services using Ether2’s or Ether3’s IP address, the connection fails.

Has anyone gotten this to work and if so, how? This isn’t a major issue since I internally point DNS to the private IPs of the servers anyway, but it would be nice to get this working.

Any help/advice is greatly appreciated.

Mezlo

PS - For more details on my routing setup, check out this thread: policy-based routing

2

I believe your problem might be that ISP1 does not know how to route the addresses back to you.

You need to masq outgoing on Ether2, not Ether1.

If you go from 158.158.158.158 → 68.68.68.68, then 68.68.68.68 must know how to route 158.158.158.158 back to you - which it won’t.

if you go from 192.168.69.1 → 68.68.68.68, then 68.68.68.68 must know how to route 192.168.69.1 back to you - which it won’t (you resolved this by masq).

Thus, you have two options:

  1. Masq everything going out on ether3 (the same for ether2 if you want to use that interface), or
  2. Setup BGP and have a peering session with both ISP1 and ISP2 - which is highly unlikely that they will allow it.

Therefore, you don’t have to Masq on ether1, but you DO need to Masq on ether2 and ether3.

Hope it make sense…

–
Chris.

3

Yes, I’m trying to go from 192.168.69.1 → 68.68.68.68 or 158.158.158.158.

Would you mind providing some example code to do this? I’m still relatively new and while I think I understand what you’re saying, I don’t know how to actually implement it.

Thanks for helping with this.

Mezlo

4

Just put a masq rule on the two interfaces, then you should be fine.

The rule’s exactly the same as it is for any other.

5

Here is my existing masq rule for ether1, which allows local computers to access the internet:

add chain=srcnat src-address=192.168.69.0/24 dst-limit=0,5,dst-address/1m40s action=masquerade comment="" \
    disabled=no

Per your instructions, I duplicated the rule for ether3:

add chain=srcnat src-address=158.158.158.158/29 dst-limit=0,5,dst-address/1m40s action=masquerade comment="" \
    disabled=no

However, I still cannot access any dst-nat’d services from my local network by specifying the IP address of ether3 (158.158.158.158).

Any other suggestions?

Mez

PS - I also tried the masq rule without specifying the subnet, but it still did not work.

6

Can’t test this on my MT’s as it will blow my network to kingdom come and nothing will work… But

Remove all your masq rules, add just one:

add chain=src-nat action=masquerade

That is what you need. Masquerade any address coming in from any address, on any interface. The MT would select the best address to masq to automatically.

7

I did this but it still will not work. However, I just thought of something that I probably should have mentioned in my original post. I’ve always been able to connect to the MT’s local services on these IPs. It’s just the dst-nat rules that don’t work.

For example, I have port 22 (SSH) dst-nat’d to another server. If I connect via SSH to Ether3 from the internet, I log into the dst-nat’d server. However, if I do the same from Ether1, I log into the MT instead. If I disable SSH on the MT, then the connection fails.

:bulb: As I’m writing this out and thinking through what is actually happening, I just realized the problem is with my dst-nat rules, not with the routing.

Here is the dst-nat rule I was using:

add chain=dstnat in-interface=Ether3 protocol=tcp dst-port=22 action=dst-nat to-addresses=192.168.69.250 \
    to-ports=22 comment="" disabled=no

I had to specify the in-interface because otherwise all connections from Ether1 would go to the local server as well, no matter what IP I was actually trying to connect to. I could not connect to any remote servers, which was bad.

I just created a rule which adds the dst-address as well and it works. :slight_smile:

add chain=dstnat in-interface=Ether1 dst-address=158.158.158.158 protocol=tcp dst-port=22 action=dst-nat \
    to-addresses=192.168.69.250 to-ports=22 comment="" disabled=no

The only problem with this is that Ether2 is dynamic so if the IP changes (which is rare), I will have to modify the rules for it. This is why I’m leaving the original rules in as well. That way, only connections from Ether1 will break, not incoming connections which are by far more important.

I appreciate your willingness to help me. It was my own poor logic that was the problem. :blush:

Mez

8

If the IP changes, use in-interface instead of dst-address

add chain=dstnat in-interface=Ether2 protocol=tcp dst-port=22 action=dst-nat \ 
    to-addresses=192.168.69.250 to-ports=22 comment="" disabled=no

Or something like that. Then it doesn’t matter what the IP of the interface its, 22/TCP will always go to 192.168.69.250:22

9

That’s the rule I already had in place and which works for remote connections but not local since the local connections are coming in Ether1, not Ether2. I’m leaving this rule in place so that if the IP does change, remote connections will still work, but local connections won’t until I change the new rule I added for Ether1.

Thanks again for your help.

Mez

10

This works in ROS 2.9, but not in 2.8. Anyone know the equivalent command for 2.8? I’ve tried all 3 possible actions for dstnat and none have worked.

Mezlo

11

Dont use 2.8 anymore, so I can’t help.

Why downgrade???

12

This is a second MT running 2.8.28 which doesn’t have a current upgrade license. The network layout is quite a bit different than my 2.9 MT, but I need to do the same thing with the dst-nat rules.

Mez

13

hmmm…

http://www.mikrotik.com/docs/ros/2.8/appex/dmz

Right at the bottom. You’ll need to do something in /ip firewall dst-nat

I’m taking a thumb suck here, but:

add action=nat in-interface=Ether1 dst-address=158.158.158.158 dst-port=22 protocol=tcp to-dst-address=192.168.69.250 to-ports=22