Here is what I am trying to do.
Two ISPs 192.168.10.x and 192.168.14.x I want to setup failover which I have done and is working. Now I want to have other devices behind my Mikrotik that get other static IPs in my range. So if I have a dlink behind my mikrotik at 192.168.0.50 i want to set it to NAT with addresses 192.168.10.x and 192.168.14.x this all seems to work until I test the fail over at which point the dlink address no longer is responsive. If i un-fail my test the dlinks Ip becomes responsive again. I am at a loss as to what to do. Im pretty sure it has something to do with a route and a mark but have no clue.
Provide:
/ip address print detail
/ip route print detail
/ip firewall export
Depending on how you have failover/load balancing setup will determine what needs to be configured to meet what you want to do, and without that basic information people will not be able to really help you get it going.
In my testing I had turned off one of my input routing marks turned on and working fine…
[admin@MikroTik] /ip firewall mangle> print detail
Flags: X - disabled, I - invalid, D - dynamic
4 chain=input action=mark-connection new-connection-mark=wlan1_conn passthrough=yes in-interface=ether1
5 chain=input action=mark-connection new-connection-mark=wlan2_conn passthrough=yes in-interface=ether2
6 chain=output action=mark-routing new-routing-mark=to_wlan1 passthrough=yes connection-mark=wlan1_conn
7 chain=output action=mark-routing new-routing-mark=to_wlan2 passthrough=yes connection-mark=wlan2_conn
Then add this route
0 A S dst-address=0.0.0.0/0 gateway=4.43.217.153 gateway-status=4.43.217.153 reachable ether2 distance=2 scope=30 target-scope=10 routing-mark=to_wlan2
Ok so i thought I got it However it just fixxed the main failover portion. I still can not get failover with staict mapping to other devices behind my mikrotik to work below are the details you asked for.
[admin@MikroTik] > ip address print detail
Flags: X - disabled, I - invalid, D - dynamic
0 ;;; Default DHCP
address=192.168.11.1/24 network=192.168.11.0 interface=Hotspot_BR
actual-interface=Hotspot_BR
1 ;;; passthrough addresses
address=192.168.14.1/24 network=192.168.14.0 interface=Hotspot_BR
actual-interface=Hotspot_BR
2 ;;; bwoia ips
address=24.227.116.189/29 network=24.227.116.184 interface=ether1
actual-interface=ether1
3 ;;; aps
address=192.168.10.1/24 network=192.168.10.0 interface=Hotspot_BR
actual-interface=Hotspot_BR
4 address=192.168.12.1/24 network=192.168.12.0 interface=Hotspot_BR
actual-interface=Hotspot_BR
5 address=71.43.217.156/29 network=71.43.217.152 interface=ether2
actual-interface=ether2
6 address=71.43.217.157/29 network=71.43.217.152 interface=ether2
actual-interface=ether2
[admin@MikroTik] > ip route print detail
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit
0 A S dst-address=0.0.0.0/0 gateway=71.43.217.153 gateway-status=71.43.217.153 reachable ether2 distance=2 scope=30 target-scope=10 routing-mark=to_wlan2
1 S dst-address=0.0.0.0/0 gateway=24.227.116.185 gateway-status=24.227.116.185 unreachable check-gateway=ping distance=1 scope=30 target-scope=10
2 A S dst-address=0.0.0.0/0 gateway=71.43.217.153 gateway-status=71.43.217.153 reachable ether2 check-gateway=ping distance=2 scope=30 target-scope=10
3 X SB dst-address=0.0.0.0/0 type=blackhole distance=20
4 X SB dst-address=0.0.0.0/0 type=blackhole distance=20
5 ADC dst-address=24.227.116.184/29 pref-src=24.227.116.189 gateway=ether1 gateway-status=ether1 reachable distance=0 scope=10
6 ADC dst-address=71.43.217.152/29 pref-src=71.43.217.156 gateway=ether2 gateway-status=ether2 reachable distance=0 scope=10
7 ADC dst-address=192.168.10.0/24 pref-src=192.168.10.1 gateway=Hotspot_BR gateway-status=Hotspot_BR reachable distance=0 scope=10
8 ADC dst-address=192.168.11.0/24 pref-src=192.168.11.1 gateway=Hotspot_BR gateway-status=Hotspot_BR reachable distance=0 scope=10
9 ADC dst-address=192.168.12.0/24 pref-src=192.168.12.1 gateway=Hotspot_BR gateway-status=Hotspot_BR reachable distance=0 scope=10
10 ADC dst-address=192.168.14.0/24 pref-src=192.168.14.1 gateway=Hotspot_BR gateway-status=Hotspot_BR reachable distance=0 scope=10
[admin@MikroTik] > ip firewall export
jan/01/1970 19:56:32 by RouterOS 5.11
software id = CKNW-7QCC
/ip firewall connection tracking
set enabled=yes generic-timeout=10m icmp-timeout=10s tcp-close-timeout=10s tcp-close-wait-timeout=10s tcp-established-timeout=1d tcp-fin-wait-timeout=10s
tcp-last-ack-timeout=10s tcp-syn-received-timeout=5s tcp-syn-sent-timeout=5s tcp-syncookie=no tcp-time-wait-timeout=10s udp-stream-timeout=3m udp-timeout=
10s
/ip firewall filter
add action=passthrough chain=unused-hs-chain comment=“place hotspot rules here” disabled=yes
/ip firewall mangle
add action=mark-connection chain=input connection-mark=no-mark disabled=no in-interface=ether1 new-connection-mark=GW1 passthrough=yes
add action=mark-connection chain=input connection-mark=no-mark disabled=no in-interface=ether2 new-connection-mark=GW2 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=GW1 disabled=no in-interface=Hotspot_BR new-routing-mark=GW1 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=GW2 disabled=no in-interface=Hotspot_BR new-routing-mark=GW2 passthrough=yes
add action=mark-connection chain=input disabled=no in-interface=ether1 new-connection-mark=wlan1_conn passthrough=yes
add action=mark-connection chain=input disabled=no in-interface=ether2 new-connection-mark=wlan2_conn passthrough=yes
add action=mark-routing chain=output connection-mark=wlan1_conn disabled=no new-routing-mark=to_wlan1 passthrough=yes
add action=mark-routing chain=output connection-mark=wlan2_conn disabled=no new-routing-mark=to_wlan2 passthrough=yes
add action=mark-routing chain=output connection-mark=wlan2_conn disabled=no new-routing-mark=to_wlan2 passthrough=yes
add action=mark-routing chain=output connection-mark=wlan1_conn disabled=no new-routing-mark=GW1 passthrough=yes
/ip firewall nat
add action=src-nat chain=srcnat connection-mark=wlan2_conn disabled=no routing-mark=GW2 src-address=192.168.10.200 to-addresses=71.43.217.157
add action=dst-nat chain=dstnat connection-mark=wlan2_conn disabled=no dst-address=71.43.217.157 routing-mark=GW2 to-addresses=192.168.10.200
add action=passthrough chain=unused-hs-chain comment=“place hotspot rules here” disabled=yes
add action=dst-nat chain=dstnat disabled=no dst-port=60003 protocol=tcp to-addresses=192.168.10.13 to-ports=80
add action=masquerade chain=srcnat disabled=no src-address=192.168.10.0/24
add action=masquerade chain=srcnat disabled=no src-address=192.168.11.0/24
add action=dst-nat chain=dstnat disabled=no dst-port=60005 protocol=tcp to-addresses=192.168.10.15 to-ports=80
add action=masquerade chain=srcnat disabled=no src-address=192.168.14.0/24
add action=dst-nat chain=dstnat disabled=no dst-port=60004 protocol=tcp to-addresses=192.168.10.14 to-ports=80
add action=dst-nat chain=dstnat disabled=no dst-port=60001 protocol=tcp to-addresses=192.168.10.11 to-ports=80
add action=dst-nat chain=dstnat disabled=no dst-port=60006 protocol=tcp to-addresses=192.168.10.16 to-ports=80
add action=dst-nat chain=dstnat disabled=no dst-port=60002 protocol=tcp to-addresses=192.168.10.12 to-ports=80
add action=masquerade chain=srcnat disabled=no src-address=71.43.217.157
/ip firewall service-port
set ftp disabled=no ports=21
set tftp disabled=no ports=69
set irc disabled=no ports=6667
set h323 disabled=no
set sip disabled=no ports=5060,5061 sip-direct-media=yes
set pptp disabled=no
Any thoughts on the above
The problem is you aren’t doing anything with the traffic being forwarded coming into the router from the outside. Since those connections are not being marked for what interface they are coming in on, and then marked for routing, it fails through to the main routing table causing your issue.
/ip firewall mangle
add action=mark-connection chain=forward disabled=no in-interface=ether1 new-connection-mark=wlan1_conn passthrough=no
add action=mark-connection chain=forward disabled=no in-interface=ether2 new-connection-mark=wlan2_conn passthrough=no
That should resolve the issue for you.
Ok the last issue I have when I fail my main isp connection port one. The backup connection comes right up on both IPS however the Ip I am forwarding over to the internal dlink will not allow internet traffic to go out unitll I disable the nat forwarding rule for the dlink when it is going through my mian isp connection. I could make a rule that turns it off an on when the mina isp comes back but I would rather avoid this if possible.
ADDRESS NETWORK INTERFACE
0 ;;; Default DHCP
192.168.11.1/24 192.168.11.0 Hotspot_BR
1 ;;; passthrough addresses
192.168.14.1/24 192.168.14.0 Hotspot_BR
2 ;;; bwoia ips
24.227.116.189/29 24.227.116.184 ether1
3 ;;; aps
192.168.10.1/24 192.168.10.0 Hotspot_BR
4 192.168.12.1/24 192.168.12.0 Hotspot_BR
5 71.43.217.156/29 71.43.217.152 ether2
6 71.43.217.157/29 71.43.217.152 ether2
7 24.227.116.187/29 24.227.116.184 ether1
DST-ADDRESS PREF-SRC GATEWAY DISTANCE
0 S 0.0.0.0/0 24.227.116.185 1
1 A S 0.0.0.0/0 71.43.217.153 2
2 S 0.0.0.0/0 24.227.116.185 1
3 A S 0.0.0.0/0 71.43.217.153 2
4 S 0.0.0.0/0 24.227.116.185 1
5 A S 0.0.0.0/0 71.43.217.153 2
6 ADC 24.227.116.184/29 24.227.116.187 ether1 0
7 ADC 71.43.217.152/29 71.43.217.156 ether2 0
8 ADC 192.168.10.0/24 192.168.10.1 Hotspot_BR 0
9 X S 192.168.10.200/32 71.43.217.157 2
10 ADC 192.168.11.0/24 192.168.11.1 Hotspot_BR 0
11 ADC 192.168.12.0/24 192.168.12.1 Hotspot_BR 0
12 ADC 192.168.14.0/24 192.168.14.1 Hotspot_BR 0
ip firewall connection tracking
set enabled=yes generic-timeout=10m icmp-timeout=10s tcp-close-timeout=10s
tcp-close-wait-timeout=10s tcp-established-timeout=1d tcp-fin-wait-timeout=
10s tcp-last-ack-timeout=10s tcp-syn-received-timeout=5s
tcp-syn-sent-timeout=5s tcp-syncookie=no tcp-time-wait-timeout=10s
udp-stream-timeout=3m udp-timeout=10s
/ip firewall filter
add action=passthrough chain=unused-hs-chain comment=“place hotspot rules here”
disabled=yes
/ip firewall mangle
add action=mark-connection chain=input disabled=no in-interface=ether1
new-connection-mark=wlan1_conn passthrough=yes
add action=mark-connection chain=input disabled=no in-interface=ether2
new-connection-mark=wlan2_conn passthrough=yes
add action=mark-routing chain=output connection-mark=wlan1_conn disabled=no
new-routing-mark=to_wlan1 passthrough=yes
add action=mark-routing chain=output connection-mark=wlan2_conn disabled=no
new-routing-mark=to_wlan2 passthrough=yes
add action=mark-routing chain=prerouting disabled=no new-routing-mark=APS
passthrough=yes src-address=192.168.10.200
add action=mark-connection chain=forward disabled=no in-interface=ether1
new-connection-mark=wlan1_conn passthrough=no
add action=mark-connection chain=forward disabled=no in-interface=ether2
new-connection-mark=wlan2_conn passthrough=no
/ip firewall nat
add action=src-nat chain=srcnat disabled=yes src-address=192.168.10.200
to-addresses=24.227.116.189
add action=dst-nat chain=dstnat disabled=yes dst-address=24.227.116.189
to-addresses=192.168.10.200
add action=passthrough chain=unused-hs-chain comment=“place hotspot rules here”
disabled=yes
add action=src-nat chain=srcnat disabled=no src-address=192.168.10.200
to-addresses=71.43.217.157
add action=dst-nat chain=dstnat disabled=no dst-address=71.43.217.157
to-addresses=192.168.10.200
add action=masquerade chain=srcnat disabled=no src-address=192.168.11.0/24
add action=dst-nat chain=dstnat disabled=no dst-port=60005 protocol=tcp
to-addresses=192.168.10.15 to-ports=80
add action=masquerade chain=srcnat disabled=no src-address=192.168.10.0/24
add action=masquerade chain=srcnat disabled=no src-address=192.168.14.0/24
add action=dst-nat chain=dstnat disabled=no dst-port=60004 protocol=tcp
to-addresses=192.168.10.14 to-ports=80
add action=dst-nat chain=dstnat disabled=no dst-port=60003 protocol=tcp
to-addresses=192.168.10.13 to-ports=80
add action=dst-nat chain=dstnat disabled=no dst-port=60001 protocol=tcp
to-addresses=192.168.10.11 to-ports=80
add action=dst-nat chain=dstnat disabled=no dst-port=60006 protocol=tcp
to-addresses=192.168.10.16 to-ports=80
add action=dst-nat chain=dstnat disabled=no dst-port=60002 protocol=tcp
to-addresses=192.168.10.12 to-ports=80
/ip firewall service-port
set ftp disabled=no ports=21
set tftp disabled=no ports=69
set irc disabled=no ports=6667
set h323 disabled=no
set sip disabled=no ports=5060,5061 sip-direct-media=yes
Any thoughts
Narrow down your src-nat rules with an out interface. It’s too general so the first one is still catching it causing your issue sending out packets with the wrong IP to an interfaces that doesn’t know about them.
Ok got that part working. Now my only issue is after the main wan connections fails the second back IP address .157 losses it remote access. Usually it take a minute or two before it comes back however now it does not look like it wants to respond anymore. I can still surf out from the address i just can not remote into it. The .156 address is working fine.
Just to let you know .157 just started to respond.
Feklar any thoughts to my above question?
Thanks
Chances are it has something to do with ARP on the ISP modem/router, or the preferred-src. for your route on that interface. As long as the IP address is active and it responds to requests locally, then it should work the same for the outside world unless there is something further upstream that is causing the issue. Are you disabling IP addresses on failover?
No I dont disable on fail over because then check gateway would not work or am I worng in that thought? I tested again and this time the failover took about 2 minutes to work all the way I wanted it to. Then next test took 10 min. I have tried disabling ether 1 and rebooting but wouldnt go either. Finally after 10 min it became responsive. Sounds like you said something with the ISP. I am working on getting a different ISP connection to test again.
Thanks/