Community discussions

MikroTik App
 
XTLMeth
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 54
Joined: Mon Sep 07, 2009 7:10 am

Inbound dst-nat with multiple wan interfaces

Fri Oct 30, 2009 4:59 am

OK here goes. I setup PCC with 2 internet connections.

wan1 is on ether1 with 72.24.205.34/24 as it's ip
wan2 is on ether2 with 216.161.237.204/29 as it's ip
lan is on ether5 with 172.16.5.12/22 as it's ip

PCC is working great but now I need to get to step 2. I have a server behind the firewall and I would like to dst-nat from each wan interface so that no matter which wan interface I come in from I can get to the server. So from a remote browser I should be able to browse to either ip and get to the server behind the firewall.


# ADDRESS NETWORK BROADCAST INTERFACE
0 172.16.5.12/22 172.16.4.0 172.16.7.255 ether5
1 216.161.237.204/29 216.161.237.200 216.161.237.207 ether2
2 D 72.24.205.34/24 72.24.205.0 72.24.205.255 ether1


/ip route
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/0 gateway=\
72.24.205.1 routing-mark=to_wan1 scope=30 target-scope=10
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/0 gateway=\
216.161.237.206 routing-mark=to_wan2 scope=30 target-scope=10
add check-gateway=ping disabled=no distance=1 dst-address=0.0.0.0/0 gateway=\
72.24.205.1 scope=30 target-scope=10
add check-gateway=ping disabled=no distance=2 dst-address=0.0.0.0/0 gateway=\
216.161.237.206 scope=30 target-scope=10


/ip firewall filter
add action=accept chain=input comment="" connection-state=established disabled=no
add action=accept chain=input comment="" connection-state=related disabled=no
add action=accept chain=forward comment="" connection-state=established disabled=no
add action=accept chain=forward comment="" connection-state=related disabled=no
add action=accept chain=output comment="" connection-state=established disabled=no
add action=accept chain=output comment="" connection-state=related disabled=no
add action=accept chain=input comment="" connection-state=new disabled=no icmp-options=8:0-255 limit=1,0 protocol=icmp
add action=accept chain=input comment="" connection-state=new disabled=no dst-port=22 limit=2/1m,0 protocol=tcp
add action=accept chain=input comment="" connection-state=new disabled=no dst-port=8291 limit=2/1m,0 protocol=tcp
add action=accept chain=forward comment="" disabled=no dst-address=172.16.5.4 dst-port=80 out-interface=ether5 protocol=\
tcp
add action=accept chain=forward comment="" connection-state=new disabled=no in-interface=ether5 src-address=\
172.16.4.0/22
add action=drop chain=input comment="" disabled=no
add action=drop chain=forward comment="" disabled=no


/ip firewall mangle
add action=mark-connection chain=input comment="" disabled=no in-interface=ether1 new-connection-mark=wan1_conn \
passthrough=yes
add action=mark-connection chain=input comment="" disabled=no in-interface=ether2 new-connection-mark=wan2_conn \
passthrough=yes
add action=mark-routing chain=output comment="" connection-mark=wan1_conn disabled=no new-routing-mark=to_wan1 \
passthrough=yes
add action=mark-routing chain=output comment="" connection-mark=wan2_conn disabled=no new-routing-mark=to_wan2 \
passthrough=yes
add action=accept chain=prerouting comment="" disabled=no dst-address=72.24.205.0/24 in-interface=ether5
add action=accept chain=prerouting comment="" disabled=no dst-address=216.161.237.200/29 in-interface=ether5
add action=mark-connection chain=prerouting comment="" disabled=no dst-address-type=!local in-interface=ether5 \
new-connection-mark=wan1_conn passthrough=yes per-connection-classifier=both-addresses:2/0
add action=mark-connection chain=prerouting comment="" disabled=no dst-address-type=!local in-interface=ether5 \
new-connection-mark=wan2_conn passthrough=yes per-connection-classifier=both-addresses:2/1
add action=mark-routing chain=prerouting comment="" connection-mark=wan1_conn disabled=no in-interface=ether5 \
new-routing-mark=to_wan1 passthrough=yes
add action=mark-routing chain=prerouting comment="" connection-mark=wan2_conn disabled=no in-interface=ether5 \
new-routing-mark=to_wan2 passthrough=yes


/ip firewall nat
add action=dst-nat chain=dstnat comment="" disabled=no dst-port=80 in-interface=ether1 protocol=tcp to-addresses=\
172.16.5.4
add action=dst-nat chain=dstnat comment="" disabled=no dst-port=80 in-interface=ether2 protocol=tcp to-addresses=\
172.16.5.4
add action=masquerade chain=srcnat comment="" disabled=no out-interface=ether1
add action=masquerade chain=srcnat comment="" disabled=no out-interface=ether2


so here's what I got going on at the moment. When I try to hit the interfaces on from the web I get mixed results. Sometimes it loads and sometimes it doesn't.
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1734
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: Inbound dst-nat with multiple wan interfaces

Fri Oct 30, 2009 8:01 am

make a custom chain and create a jump rule if packet comes from each interface, this way every change you make in custom chain will apply to all connections
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
XTLMeth
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 54
Joined: Mon Sep 07, 2009 7:10 am

Re: Inbound dst-nat with multiple wan interfaces

Fri Oct 30, 2009 4:52 pm

Do you think you could give me a little example. I'm struggling a little bit. Once I'm in the custom chain I have to mark my connections right?
 
XTLMeth
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 54
Joined: Mon Sep 07, 2009 7:10 am

Re: Inbound dst-nat with multiple wan interfaces

Fri Oct 30, 2009 5:05 pm

OK a little more info. Here's what happening. When I'm comming in wan1 sometimes it's get's marked with wan2 marking instead so obviously the return packets are getting sent out the wrong interface on the reply. So how do I fix this??
 
fewi
Forum Guru
Forum Guru
Posts: 7734
Joined: Tue Aug 11, 2009 3:19 am

Re: Inbound dst-nat with multiple wan interfaces

Fri Oct 30, 2009 5:17 pm

Looking at the Packet Flow diagram (http://wiki.mikrotik.com/wiki/Packet_Flow), it looks as if a packet dst-nat'd to a server behind the firewall hits the following chains in sequence, listing only the relevant parts:

prerouting (connection tracker, prerouting mangle, dst-nat) -> make routing decision, at this point it would choose 'forward' instead of 'input' since the destination IP is no longer the router itself.

Your rules only mark packets in the input chain, or in the prerouting chain if they are from the LAN interface - so that's the first time a mark is applied, and it is chosen somewhat randomly, resulting in packets potentially going out the wrong interface.

Insert two rules like this:
/ip firewall mangle 
add action=mark-connection chain=forward comment="" disabled=no in-interface=ether1 connection-state=new new-connection-mark=wan1_conn passthrough=yes 
add action=mark-connection chain=forward comment="" disabled=no in-interface=ether2 connection-state=new new-connection-mark=wan2_conn passthrough=yes
to mark new incoming connections that are forwarded through the router. I think you'll also have to change:
/ip firewall mangle 
add action=mark-connection chain=prerouting comment="" disabled=no dst-address-type=!local in-interface=ether5 
new-connection-mark=wan1_conn passthrough=yes per-connection-classifier=both-addresses:2/0
add action=mark-connection chain=prerouting comment="" disabled=no dst-address-type=!local in-interface=ether5
new-connection-mark=wan2_conn passthrough=yes per-connection-classifier=both-addresses:2/1
and add 'connection-mark=no-mark' (available in 4.x only, I think?) to make sure that you don't overwrite the connection-mark in prerouting on the reply from the server.

That's off the top of my head - maybe I'll have time at work today to try it in a lab.
 
XTLMeth
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 54
Joined: Mon Sep 07, 2009 7:10 am

Re: Inbound dst-nat with multiple wan interfaces

Fri Oct 30, 2009 6:20 pm

The mangle rules I have in the firewall at the moment are apart of the tutorial for PCC load balancing but I also need my dst-nat working for both wans. I will try a couple things to today to see if I can get it to work. I think the hardest part is walking through the packet flow and making sure you don't violate or break rules that are already in place. :-)
 
fewi
Forum Guru
Forum Guru
Posts: 7734
Joined: Tue Aug 11, 2009 3:19 am

Re: Inbound dst-nat with multiple wan interfaces

Fri Oct 30, 2009 6:23 pm

Did you try the adding the rules (and changing the two) I posted? It should do the trick.
 
XTLMeth
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 54
Joined: Mon Sep 07, 2009 7:10 am

Re: Inbound dst-nat with multiple wan interfaces

Fri Oct 30, 2009 7:53 pm

I added the new mangle rules for the forward table but the other PCC rules are exactly the same as what I have in there already so I didn't change them. I will try this now and see if it works.
 
XTLMeth
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 54
Joined: Mon Sep 07, 2009 7:10 am

Re: Inbound dst-nat with multiple wan interfaces

Fri Oct 30, 2009 8:03 pm

ok so with those 2 forward rules in place 2 things have happened. I can't get to the router from the internet now and it always marks the connections from either interface was wan1_conn so it only works from the 72.24.x.x ip.

From the conntrack table ---->

11 S protocol=tcp src-address=199.253.130.17:39516 dst-address=216.161.237.204:80 reply-src-address=172.16.5.4:80
reply-dst-address=199.253.130.17:39516 tcp-state=syn-recv timeout=0s connection-mark="wan1_conn" p2p=none

As you can see I tried to hit it form wan2's ip 216.161.237.204
 
fewi
Forum Guru
Forum Guru
Posts: 7734
Joined: Tue Aug 11, 2009 3:19 am

Re: Inbound dst-nat with multiple wan interfaces

Fri Oct 30, 2009 8:03 pm

Alright, more explicitly. Change:
/ip firewall mangle
add action=mark-connection chain=prerouting comment="" disabled=no dst-address-type=!local in-interface=ether5 new-connection-mark=wan1_conn passthrough=yes per-connection-classifier=both-addresses:2/0
add action=mark-connection chain=prerouting comment="" disabled=no dst-address-type=!local in-interface=ether5 new-connection-mark=wan2_conn passthrough=yes per-connection-classifier=both-addresses:2/1
to
/ip firewall mangle
add action=mark-connection chain=prerouting comment="" disabled=no dst-address-type=!local in-interface=ether5 connection-mark=no-mark new-connection-mark=wan1_conn passthrough=yes per-connection-classifier=both-addresses:2/0
add action=mark-connection chain=prerouting comment="" disabled=no dst-address-type=!local in-interface=ether5 connection-mark=no-mark new-connection-mark=wan2_conn passthrough=yes per-connection-classifier=both-addresses:2/1
That will require 4.x
 
XTLMeth
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 54
Joined: Mon Sep 07, 2009 7:10 am

Re: Inbound dst-nat with multiple wan interfaces

Fri Oct 30, 2009 8:23 pm

Works. I also see what this does. I will keep playing around but I really appreciate your help on this. :-D Many thanks!!!
 
XTLMeth
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 54
Joined: Mon Sep 07, 2009 7:10 am

Re: Inbound dst-nat with multiple wan interfaces

Fri Oct 30, 2009 8:27 pm

If I think I got this right what the no-mark does it prevent the dst-nat return traffic from getting tagged by the PCC rule which causes it to be sent out the wrong interface.
 
fewi
Forum Guru
Forum Guru
Posts: 7734
Joined: Tue Aug 11, 2009 3:19 am

Re: Inbound dst-nat with multiple wan interfaces

Fri Oct 30, 2009 8:41 pm

Yes. The connection gets marked by the new rules in the forward chain when it is established (connection-state=new) depending on which interface the request came in from. You need to make sure that the PCC rules don't overwrite the connection-mark on the outbound return traffic as with your former rules they somewhat randomly choose an outbound interface for _all_ traffic coming in via the LAN interface, that's done by PCC marking only connections that have not been marked yet (connection-mark=no-mark).
 
XTLMeth
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 54
Joined: Mon Sep 07, 2009 7:10 am

Re: Inbound dst-nat with multiple wan interfaces

Fri Oct 30, 2009 8:52 pm

Yep that makes sense. That was the link I have been missing the whole time. Thank you for explaining it.

Who is online

Users browsing this forum: eworm and 86 guests