Dual ISP with no balancing but with forwarding some ports to lan

RouterOS General
1

Hello,

I have the following setup:

  • primary ISP1, static ip, 300Mbit connection (has rp_filter enabled)
  • secondary ISP2, static ip, 20Mbit connection (no rp_filter)
    I am forwarding some ports (same) from both ISP to some ip’s on the lan. I’m using mangle for marking and routes with rules for both ISPs, like in https://wiki.mikrotik.com/wiki/Manual:PCC but without the PCC rules, as I only need the secondary connection for backup and for some of the services only.
    The problem is that I’m messing something because currently, with my setup, I can connect to the router from the internet, on both ISPs, but I can’t connect to the internal servers from the second ISP. If I change the primary ISP route distance lower than the secondary, I can connect using the external IPs to the internal servers, but the connection is going out from the Mikrotik through the secondary ISP, who’s accepting packets with other ISP source IP.
    I’ve also disabled fasttrack-connection from firewall filter, but all is the same, only with a lower internet throughput and almost 100% cpu usage.
    Please help me understand what I’m doing wrong.
    Here are my rules:
    mangle:
/ip firewall mangle
add action=accept chain=prerouting dst-address-list=Connected src-address-list=Connected
add action=mark-routing chain=prerouting connection-mark=ISP2 new-routing-mark=ISP2 passthrough=no src-address-list=LAN
add action=mark-routing chain=prerouting connection-mark=ISP1 new-routing-mark=ISP1 passthrough=no src-address-list=LAN
add action=mark-connection chain=input connection-mark=no-mark in-interface=ISP1 new-connection-mark=ISP1 passthrough=no
add action=mark-connection chain=input connection-mark=no-mark in-interface=ISP2 new-connection-mark=ISP2 passthrough=no
add action=mark-connection chain=forward connection-mark=no-mark in-interface=ISP1 new-connection-mark=ISP1 passthrough=no
add action=mark-connection chain=forward connection-mark=no-mark in-interface=ISP2 new-connection-mark=ISP2 passthrough=no
add action=mark-routing chain=output connection-mark=ISP1 new-routing-mark=ISP1 passthrough=no
add action=mark-routing chain=output connection-mark=ISP2 new-routing-mark=ISP2 passthrough=no

 /ip route
add distance=10 gateway=x.x.x.1 pref-src=x.x.x.186 routing-mark=ISP1
add distance=10 gateway=y.y.y.209 pref-src=y.y.y.210 routing-mark=ISP2
add check-gateway=ping distance=10 gateway=x.x.x.1 pref-src=x.x.x.186
add check-gateway=ping distance=10 gateway=y.y.y.209 pref-src=y.y.y.210
/ip route rule
add routing-mark=ISP1 table=ISP1
add routing-mark=ISP2 table=ISP2

/ip firewall nat
add action=masquerade chain=srcnat out-interface-list=WAN
add action=dst-nat chain=dstnat dst-address-list=ext-ips dst-port=53 protocol=udp to-addresses=192.168.0.25
add action=dst-nat chain=dstnat dst-address-list=ext-ips dst-port=21,22 protocol=tcp to-addresses=192.168.0.35
add action=dst-nat chain=dstnat dst-port=37771 in-interface-list=WAN protocol=udp to-addresses=192.168.0.11
add action=dst-nat chain=dstnat dst-port=81,37711 in-interface-list=WAN protocol=tcp to-addresses=192.168.0.11

/ip firewall address-list
add address=192.168.0.0/24 list=Connected
add address=192.168.88.0/24 list=Connected
add address=y.y.y.210 list=ext-ips
add address=x.x.x.186 list=ext-ips

/interface list member
add interface=ISP1 list=WAN
add interface=ISP2 list=WAN
add interface=bridge list=LAN

I’ve edited the post as the multiple code snippets seems to broke the display, so I’ve added all code in one.

2

If your secondary ISP is for backup only, then why use PCC ?
Why don’t you just failover those two lines?
If your ISP uses tunnels, most common PPPoE, simply add distance 1 to your primary and 2 to your secondary…
If you are given static IPs, recursive failover is the way to go…*
If you are behind a router and you get access to the internet through DHCP client, then again recursive failover is the solution…

  • Recursive failover can not be used if gateway is an interface instead of IP address.. for example if you are assigned a /31 network, you cant use IP as gateway but only interface. In this case netwatch to check internet connectivity is a must…
3

The servers behind the Mikrotik must be reachable from the secondary ISP also. The servers must be on the internet at all times. I know about the failover, it’s working, but does not apply here. I am not using PCC, I only need the connections coming and going from and to the ISP over they originated.

4

For now, until I’ll understand what’s wrong with mangle/nat setup from above, I’ve added one more subnet on the Mikrotik, 10.0.0.0/24 on the same bridge as the 192.168.0.0/24. Same on the servers. I’ve doubled the nat rules for the servers this time specifying the external ip and added a new rule in the Mikrotik routing so that all pachets coming from 10.0.0.0/24 will be using the ISP2 routing table. I think this is close to the topic here: http://forum.mikrotik.com/t/policy-based-routing-using-two-uplinks/51312/1

So the code added/modified on Mikrotik was:

/ip route rule
add src-address=10.13.13.0/24 table=ISP2

/ip firewall nat
add action=masquerade chain=srcnat out-interface-list=WAN
add action=dst-nat chain=dstnat dst-address=x.x.x.186 dst-port=53 protocol=udp to-addresses=192.168.0.25
add action=dst-nat chain=dstnat dst-address=y.y.y.209 dst-port=53 protocol=udp to-addresses=10.0.0.25
add action=dst-nat chain=dstnat dst-address=x.x.x.186 dst-port=21,22 protocol=tcp to-addresses=192.168.0.35
add action=dst-nat chain=dstnat dst-address=y.y.y.209 dst-port=21,22 protocol=tcp to-addresses=10.0.0.35
add action=dst-nat chain=dstnat dst-port=37771 dst-address=x.x.x.186 protocol=udp to-addresses=192.168.0.11
add action=dst-nat chain=dstnat dst-port=37771 dst-address=y.y.y.209 protocol=udp to-addresses=192.168.0.11
add action=dst-nat chain=dstnat dst-port=81,37711 dst-address=x.x.x.186 protocol=tcp to-addresses=192.168.0.11
add action=dst-nat chain=dstnat dst-port=81,37711 dst-address=y.y.y.209 protocol=tcp to-addresses=192.168.0.11

And now all works as expected, but with added complexity on the servers and on the Mikrotik…