Dual WAN 1 LAN with NAT configuration

mavericklu · Wed May 13, 2020 5:44 am

Hi all,

I am just a server guy and have limited knowledge of networking stuff, feel embarrass to ask this question lol because the Mikrotik router I am using was a gift from a friend...
Recently my ISP upgrade my subscription and gave me totally 4 IP addresses, I have connected the additional one on ETH3-WAN2 by DHCP client. I can see RouterOS generates 2 default routes for me (I don't know what the route means anyway...)

I set the connection up because I have some servers in my LAN. I would like to have ETH3-WAN2 accept some tcp connection, say port 443 and DST-NAT to my nginx inside. I found 2 issues.

1) ETH3-WAN2 IP is not pingable from outside
2) It looks like that it has not accepting any connection at all so the DST-NAT rules does not do anything. I get time out when connecting to ETH2-WAN2 IP from outside, say browser https://1.1.3.10

I have gone through the forum and knows about the terms Mangling, I used to follow some rules and none of it works, i.e. still not pingable, still cannot connect to the WAN2 IP.

Attached is the network diagram. I would like to seek help from expert like you here, and I would like to say sorry if I do not understand your explanation. Please slowly guide me through to provide any information you need about my router current config.

Thanks!
Maverick

sindy · Wed May 13, 2020 3:00 pm

The goal is clear (and nothing unusual). What is not clear is what exactly you mean by

the DST-NAT rules does not do anything

Because there is a difference between "the rule doesn't even count packets/bytes", which means that no packets are coming in, and "the rule appears to do nothing because no responses come back to the client". The thing is that dst-nat rules are processed before an filter rules, but after raw rules which can drop packets as well.

So try to read this, starting from the last paragraph (starting with EDIT) which explains the relationship to your case. What is missing in that post, because it has been mentioned in one of the previous ones in that thread, is that it is not enough to assign a routing-mark but a set of routes with that routing-mark must exist. For your simple case, the routing-mark may be assigned using /ip route rule rather than /ip firewall mangle, but the marked route via the proper gateway is the key.

If it doesn't help, follow the suggestion in my automatic signature below, to get back a set of configuration lines to copy-paste.

mavericklu · Wed May 13, 2020 4:45 pm

Thank you very much for your reply Sindy, I am sure I will spend my time to follow the posts u gave.

DST-NAT rule doesn't do anything - meaning that in the sense of troubleshooting, I try to connect to the server from browser externally and see if the NAT rules kicks in due to the IP is not pingable externally. There is no packet going through the NAT rule and eventually browser responses connection refused.. Since my network knowledge is very limited I am not able to judge where the problem is, either the WAN port configuration, or the nat rules.

2frogs · Wed May 13, 2020 5:00 pm

I am going to ask the more obvious! Are you using the default firewall? And did you add ETH3-WAN2 to WAN Interface List?

sindy · Wed May 13, 2020 5:11 pm

OK, this is an important piece of information. Unless you have something in /ip firewall raw, the fact that the dst-nat rule doesn't even count packets suggests that the packets didn't make it to the router from outside. To be sure, run /tool sniffer quick interface=ETH3-WAN2 and try to connect from outside to the IP assigned to ETH3-WAN2. If you know from what address you'll be connecting, even better, as you can add ip-address=that.known.ip.address to the command above, to get rid of any eventual noise.

You must not test from anything connected to that same Mikrotik, as in that case, the traffic would not come in via ETH3 but would take a shortcut!

mavericklu · Wed May 13, 2020 7:31 pm

Hi Sindy, I have tried /tool sniffer with the ip that uses to connect to my ETH3-WAN2, and saw message flow on terminal window. In dst-nat rule bytes is 624 and packet 12 untils browser times out.

INTERFACE TIME NUM DI SRC-MAC DST-MAC VLAN
ETH3-WAN2 11.425 11 <- F4:C6:13:YY:YY:YY 00:18:09:XX:XX:XX
ETH3-WAN2 11.676 12 <- F4:C6:13:YY:YY:YY 00:18:09:XX:XX:XX
ETH3-WAN2 19.423 13 <- F4:C6:13:YY:YY:YY 00:18:09:XX:XX:XX
ETH3-WAN2 19.423 14 <- F4:C6:13:YY:YY:YY 00:18:09:XX:XX:XX
ETH3-WAN2 19.673 15 <- F4:C6:13:YY:YY:YY 00:18:09:XX:XX:XX

Hi 2frogs, I don't follow you. Would you please let me know the command so that I can show u the information u want to know?

sindy · Wed May 13, 2020 7:48 pm

I have tried /tool sniffer with the ip that uses to connect to my ETH3-WAN2, and saw message flow on terminal window. In dst-nat rule bytes is 624 and packet 12 untils browser times out.

OK, so the dst-nat rule actually does count. It means that the packets do arrive to the Mikrotik, and you likely could even sniff them on the target VM, which may even send responses to them. However, since you don't have a marked route via ETH3-WAN2's gateway, the responses are sent out via ETH1-WAN1's gateway, but since they still have the IP address of ETH2-WAN2 as source (the un-dst-nat uses the address towards which the connection was established), the ISP may drop them.

You can verify the above assumption by sniffing again without indicating any interface, just the remote address. It will show you how far the requests and responses get through your Mikrotik. If the absence of a route via ETH3-WAN2's gateway (and a measure to let the responses from the VM take this route rather than the other one) is the only problem, you'll see the requests on the ETH3-WAN2 and the LAN interface, and the responses on LAN and ETH1-WAN1. If the requests don't make it to LAN, it is a firewall configuration issue.

It is still valid that you may post your current configuration if you get lost.

mavericklu · Wed May 13, 2020 7:59 pm

Hi Sindy,

Thank you very much for your quick response. I confess I am lost lol, and so I have attached the config. However, the /export hide-sensitive doesn't seem to contain any public IPs and so I am not sure if the export if complete.

mutluit · Wed May 13, 2020 8:42 pm

@mavericklu, you have in the nat table the IPs of the servers wrong (or there are other such server IPs listed, but not the said 172.16.10.10 nor 172.16.10.20).
Also check/verify the port numbers, as there are some 8443 (instead of the said 443)...
And are you really sure you want forward port 80 to port 443?

And: your firewall does not protect you fully, ie. there is still much room for some essential improvements.... (just a quick analysis)

mavericklu · Wed May 13, 2020 8:49 pm

Hi @mutluit

The internal IP of the diagram on my original post is just an illustration. Before connecting ETH3-WAN2 all the nat rules are working fine.
Would you give me some suggestion on how to improve the firwall rules to make the protection better?

mavericklu · Wed May 13, 2020 8:53 pm

@mutluit nono that was a mistake, 80 should be nat to 80 lol..
8443 is a port that used for vmware product, it is correct.

mutluit · Wed May 13, 2020 8:57 pm

Hi @mutluit

The internal IP of the diagram on my original post is just an illustration. Before connecting ETH3-WAN2 all the nat rules are working fine.

Would you give me some suggestion on how to improve the firwall rules to make the protection better?

The general flow should be like this:

accept wanted port/protocol
...
and the last rule in the chain should be:
drop (or reject) all the rest of packets

But be aware that if you make an error, then you could possibly block yourself out. Meaning: make a good plan first on paper...
And of course: always have backups (and your notes to it on paper etc.) of your working configurations (very important!)

mavericklu · Wed May 13, 2020 9:02 pm

@multuit

would drop all protects me from being port scanned and ssh attempt? Right now I have like 1000 blocked IPs on the list who tried to ssh me everyday ... meaning that I sacrifice myself from ssh to my server outside as well, without a better rule design....

mutluit · Wed May 13, 2020 9:08 pm

@multuit

would drop all protects me from being port scanned and ssh attempt? Right now I have like 1000 blocked IPs on the list who tried to ssh me everyday ... meaning that I sacrifice myself from ssh to my server outside as well, without a better rule design....

IMO yes!

But port scanning you can't protect/prevent completely, IMO. And there are some different types of port scanning methods possible (connect, syn etc.). But you can control the rate of such scans.

But would need some more info on how you do it currently, and of course a list of all allowed source and dest IPs/ports/protocols.

And: it is also possible to let ssh run on a different port, if that is acceptable for your environment. But the hackers/attacker usually port scan and find it out.
A better solution is to use a tool like fail2ban on the server where ssh runs (actually on all servers where remote login is possible), that blocks these attacking IPs for some minutes or much longer (configurable by you).
Cf. https://en.wikipedia.org/wiki/Fail2ban

mavericklu · Wed May 13, 2020 9:19 pm

@mutluit

fail2ban looks great, i will give it a shot, thank you!

mutluit · Wed May 13, 2020 9:32 pm

@mutluit

fail2ban looks great, i will give it a shot, thank you!

You are welcome.
For professional/advanced use of such tools like fail2ban you should learn some "regex" (Regular Expressions, ie. a Pattern Matching language, a little bit cryptic

, but very logical and powerful) --> https://en.wikipedia.org/wiki/Regular_expression

sindy · Wed May 13, 2020 9:58 pm

the /export hide-sensitive doesn't seem to contain any public IPs and so I am not sure if the export if complete.

Since both your public IPs are assigned dynamically using DHCP, they are not shown in export. The export command shows the static configuration; a print command shows the currently existing items, including those dynamically added. I don't ask for a print output, it's just an explanation.

@mutluit is right that your firewall is not tight enough - you use a mix of "permit everything but a few exceptions" and "deny everything but a few exceptions", but the first one wins.

I personally prefer to deny everything but a few exceptions, because if you forget to permit something that should have been permitted, your legal users will be unhappy and will let you know quickly; if you forget to deny something that should have been denied, your illegal users will be happy and will never let you know.

But that's a subject for another topic, although a very important one if you want to remain the only administrator of your router.

To the topic of your OP - from what you wrote I understand that the IP addresses you get from the ISP are fixed despite the fact that you get them using DHCP, can you confirm that? The difference it makes is whether it is necessary to add a script to the DHCP client configuration, which would update the gateway parameter of a route via ETH3-WAN2, or whether it can be left out because the IP address of the gateway actually never changes.

Given that you actually port-forward the traffic which arrives to ETH3-WAN2 to multiple internal addresses, use of /ip firewall mangle to handle the assignment of the routing-mark seems more appropriate to me than use of /ip route rule. Nevertheless, the /ip route rule rows are used to prevent packets coming in via WAN2 from being routed back through it instead of being delivered to the LAN hosts - it's one of possible ways to ensure that.

So the following will make all connections which come from the internet to the public IP assigned to ETH3-WAN2 be responded through that interface:

/ip route
add dst-address=0.0.0.0/0 routing-mark=via-wan2 gateway=[/ip dhcp-client get [find interface=ETH3-WAN2] gateway]

/ip route rule
add dst-address=172.16.10.0/24 action=lookup-only-in-table table=main
add dst-address=192.168.0.0/24 action=lookup-only-in-table table=main

/ip firewall mangle
add chain=prerouting connection-state=new action=jump jump-target=cmark-pr comment="only the first packet of each connection is eligible for eventual connection marking"
add chain=prerouting connection-mark=wan2-conn action=mark-routing new-routing-mark=via-wan2 comment="all packets, including the first one, may need the connection-mark to routing-mark translation"

add chain=cmark-pr in-interface=ETH3-WAN2 action=mark-connection new-connection-mark=via-wan2 passthrough=yes comment="anything that came in via WAN2 must be responded through there"

add chain=output connection-mark=wan2-conn action=mark-routing new-routing-mark=via-wan2 comment="connections to Mikrotik itself may also gave come in via WAN2"

mavericklu · Thu May 14, 2020 4:07 am

Hi Sindy,

I have pasted the script you provide, unfortunately the symtom is same, dst-nat rules count, wan2 ip is not pingable from outside, browser times out when connecting ETH3-WAN2

I have also implement some filter rules so the export is a bit longer. Would you mind to take a look and help again?
Yes, the WAN IP is obained by DHCP and it is actually quite steady (according to the WAN1 IP, same ISP)

2frogs · Thu May 14, 2020 8:28 am

Your firewall rules need a lot of work! chain=input is for traffic going to the router it's self (Webfig, Winbox, Ping, DNS, etc.) chain=forward is for any traffic being forwarded by the router (from one interface to another.) All the rules you added mostly belonged to the chain=forward since it was for traffic being forwarded from WAN to LAN. You can simplify your rules to the following:

/ip firewall filter
add action=accept chain=input comment="Accept Established and Related to Router" connection-state=established,related,untracked
add action=drop chain=input comment="Drop Invalid to Router" connection-state=invalid
add action=accept chain=input comment="Limit Pings (icmp) to Router" limit=50/5s,2 protocol=icmp
add action=accept chain=input comment=VPN dst-port=500,1701,4500 in-interface=!ETH2-LAN1 protocol=udp
add action=drop chain=input comment="Drop All Input Not from LAN" in-interface=!ETH2-LAN1
add action=accept chain=forward comment="Accept Established and Related to Clients" connection-state=established,related,untracked
add action=drop chain=forward comment="Drop Invalid to Clients" connection-state=invalid in-interface=ETH2-LAN1
add action=drop chain=forward comment="Drop All Forward from WAN Not DST-NAT" connection-nat-state=!dstnat 
in-interface!ETH2-LAN1

A different question I have is are both WAN port plugged into the same ISP Router?

/ip dhcp-client
add comment="HGC WAN Leg (LAN3)" default-route-distance=0 dhcp-options=hostname disabled=no interface=ETH1-WAN1
add comment="HGC WAN Leg (LAN4)" dhcp-options=hostname disabled=no interface=ETH3-WAN2

The "HGC WAN Leg (LAN3)" and "HGC WAN Leg (LAN4)" is why ask. Are actually getting a Public IP form this router or does it give you a Private IP that your Public IPs are being routed to (DMZ)?

mavericklu · Thu May 14, 2020 11:33 am

@2frogs Thank you for your help.
A fiber optics cable goes into my house and connect to the modem, then the modem has 4 ethernet ports. The ETH1-WAN1 and ETH3-WAN3 of Mikrotik are connected to different port of the modem, and through DHCP client to get the ip addresses accordingly and both IP address are public IPs.

2frogs · Thu May 14, 2020 8:05 pm

This is a strange way of handing out multiple IPs. It is usually considered to be a big No-No to make multiple connections to the same device. You normally have to configure manually for the additional IPs. You might contact your ISP to be sure you have done this correctly.

mavericklu · Thu May 14, 2020 8:42 pm

@2frogs what do u mean configure manually?

ISP is leasing the IP via their DHCP server and I can get it with mikrotik dhcp client. I cannot control how ISP does their cabling work as well..
I do believe for home use internet service, they don't expect their customers to have enterprise grade router at home. They suggest customers to have a seperate router to make connection to their modem and obtain IP address, which I won't follow them because I believe mikrotik can do it all in one.

sindy · Thu May 14, 2020 8:53 pm

What @2frogs says (or say? hard to guess) is important, and it did not come to my mind at all from reading your OP, although it should.

Some ISPs do provide multiple IP addresses this way (one IP addres per MAC address via DHCP, no other way can be agreed, "that's how our product specification looks like, take it or leave it"), but I haven't guessed from your OP that the two WANs are connected to the same L2 segment and thus both may get and address from the same subnet and therefore the same gateway IP. Can you check that both the default routes added by the DHCP clients (/ip route print shows them with a D for Dynamic in the lefmost column) show the same IP address as gateway?

If it is the case, you'll need to configure also the interface name in the gateway item of the routes, which requires to disable the automatic addition of the routes by teh DHCP client and to add them manually instead.

I'll give you the script to do these changes in a while.

mavericklu · Thu May 14, 2020 9:09 pm

Hi Sindy,

below is the /ip route print result. please have a look.

DST address Gateway Distance routing mark
AS 0.0.0.0/0 223.x.x.1 reachable ETH3-WAN2 1 via-wan2
DAS 0.0.0.0/0 221.x.x.1 reachable ETH1-WAN1 0
DS 0.0.0.0/0 223.x.x.1 reachable ETH3-WAN2 1
DAC 172.16.10.0/24 ETH2-LAN1 reachable 0
DAC 221.x.x.0/24 ETH1-WAN1 reachable 0
DAC 223.x.x.0/24 ETH3-WAN2 reachable 0

sindy · Thu May 14, 2020 9:30 pm

OK, so same subnet and gateway IP at both WAN interfaces is not the reason why it doesn't work. So I've checked your updated export and found my mistake.

To fix that mistake, run
/ip firewall mangle set [find chain~"cmark-pr"] new-connection-mark=wan2-conn

I wanted to emphasize the fact that connection-mark and routing-mark are not the same thing, and ended up with mixing their names up. So I was assigning one value of connection-mark based on in-interface, and translating another connection-mark value to a routing-mark.

Unrelated to the above, try "reply with quote" to see how I've forced fixed character width to the table below:

     DST address           Gateway                  Distance    routing mark
AS       0.0.0.0/0   223.x.x.1 reachable ETH3-WAN2     1          via-wan2
DAS      0.0.0.0/0   221.x.x.1 reachable ETH1-WAN1     0
DS       0.0.0.0/0   223.x.x.1 reachable ETH3-WAN2     1
DAC 172.16.10.0/24   ETH2-LAN1 reachable               0
DAC   221.x.x.0/24   ETH1-WAN1 reachable               0
DAC   223.x.x.0/24   ETH3-WAN2 reachable               0

mavericklu · Thu May 14, 2020 9:36 pm

yeah~ @sindy it works now, thank you so much!

sorry for the messy formatting, seldom post on phpbb lol

sindy · Thu May 14, 2020 9:42 pm

Good. So now it may be the right time to read e.g. this supercharged introduction into how firewall rules work.

mavericklu · Thu May 14, 2020 9:51 pm

Good. So now it may be the right time to read e.g. this supercharged introduction into how firewall rules work.

nice post indeed, didn't know firewall rules setting (skill-wise) can be that flexible, lesson learnt!

Dual WAN 1 LAN with NAT configuration [SOLVED]

Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration [SOLVED]

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Re: Dual WAN 1 LAN with NAT configuration

Who is online