Community discussions

MikroTik App
 
akarpas
Member Candidate
Member Candidate
Topic Author
Posts: 130
Joined: Tue Mar 20, 2018 4:46 pm

How to connect vrrp'ed routers to wan (ISP)

Fri Feb 12, 2021 11:10 am

Hi guys.
I'm playing around with this topology. So on LAN side everything is clear, configured and working.(VRRP + RSTP + DHCP on server). But wats about WAN connections. for ISP 2 Cable broadband I have only 1 static IP issued from ISP and ISP1 DSL connects via PPPoE and again single static IP. So I need two routers to connect to two modems. Cant do VVRP on WAN as have single static IP's. Any suggestions on how to configure to keep failover and redundancy!
You do not have the required permissions to view the files attached to this post.
 
Kindis
Member
Member
Posts: 368
Joined: Tue Nov 01, 2011 6:54 pm
Location: Sweden

Re: How to connect vrrp'ed routers to wan (ISP)

Fri Feb 12, 2021 3:28 pm

So VRRP the dead minimum is 3 IP's and they need to be on the same network and they use broadcast. So I would say no this will not work.
With that said perhaps you can use scripts etc to move the IP between devices. For example if you create a VRRP on the internal network you can add UP and Down scripts. If you in there enable or disable the static IP you can have it on both devices and if one fails the other router will take over the IP. The you can monitor the status of the links as well and change Priority via script if something fails thus moving.

I use this internally but that is for enabling and disabling stuff like DHCP servers and CapsMAN manager.
 
tdw
Forum Veteran
Forum Veteran
Posts: 709
Joined: Sat May 05, 2018 11:55 am

Re: How to connect vrrp'ed routers to wan (ISP)

Fri Feb 12, 2021 4:47 pm

If you have two public IPs provided by two different providers there isn't much you can do if the public addresses are terminated on the Mikrotiks, other than have an active-active setup with each Mikrotik handling one WAN connection and you loose access to that WAN if there is an issue with the attached Mikrotik.

If the devices between the Mikrotiks and WAN connections are capable of NAT & DMZ you could abuse that and use VRRP on the Mikrotiks, however failure of a Mikrotik will still drop any active connections as there is no connection tracking synchronisation.

Sometimes the potential failure modes of redundancy setups actually leads to less reliability as there are more things to fail - you have to analyse all the possible fault scenarios, not forgetting cable faults on the interconnections which is often overlooked.
 
akarpas
Member Candidate
Member Candidate
Topic Author
Posts: 130
Joined: Tue Mar 20, 2018 4:46 pm

Re: How to connect vrrp'ed routers to wan (ISP)

Fri Feb 12, 2021 6:41 pm

So as I understand the best option would be to stick one ISP on one router let's say cable ISP on master router and DLS on backup router. next to set up Netwatch tool to monitor 8.8.8.8 from master router and if it gets to down state allow it to switch off VLAN for VRRP or switch it back if ping to 8.8.8.8 back. So it means if ISP1 one down VRRP kicks to backup router. But how reliable is Netwatch tool can I trust it in business environment.
You do not have the required permissions to view the files attached to this post.
 
tdw
Forum Veteran
Forum Veteran
Posts: 709
Joined: Sat May 05, 2018 11:55 am

Re: How to connect vrrp'ed routers to wan (ISP)

Fri Feb 12, 2021 7:06 pm

You need not switch the LAN VRRP over due to an external WAN failure, you can have backup default routes, i.e. with a greater distance via the other. If they are static routes the packets will bounce back and forth between the two Mikrotiks when both WANs are down, but you probably don't care in that case.

Netwatch itself is fine, but pinging 8.8.8.8 is not always a reliable test, I have seen it stop responding to ICMP but still serve DNS on occasions. There are methods of using recursive routing to make a default route available so you can have multiple targets which do not respond before the default route becomes inactive, usually for multi-WAN on one router but would also be applicable here.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 6157
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: How to connect vrrp'ed routers to wan (ISP)

Mon Feb 15, 2021 7:00 pm

Hmm what about a managed switch in between?
For example ISP1 to port1, ISP2 to port2 (basic port vlan)
Port1 connected to ports 3,4
Port2 connected to ports 5,6

Router1 connections to ports 3,5
Router2 connections to ports 4,6

Assuming will need one dedicated router1 to router2 connection as well.
I'd rather manage rats than software. Follow my advice at your own risk! (Sob & mkx forced me to write that!)
MTUNA Certified, by the Ascerbic Llama!
 
tdw
Forum Veteran
Forum Veteran
Posts: 709
Joined: Sat May 05, 2018 11:55 am

Re: How to connect vrrp'ed routers to wan (ISP)

Mon Feb 15, 2021 7:10 pm

Hmm what about a managed switch in between?
That would become a single point of failure - takes out both WAN connections. Engineering redundancy solutions which does not make your setup less reliable is not straightforward.
 
akarpas
Member Candidate
Member Candidate
Topic Author
Posts: 130
Joined: Tue Mar 20, 2018 4:46 pm

Re: How to connect vrrp'ed routers to wan (ISP)

Tue Feb 16, 2021 11:38 am

need a script which would monitor two host min as 8.8.8.8 and 1.1.1.1 if both, are down disable VRRP bridge and if 8.8.8.8 OR 1.1.1.1 is up do nothing or bring VRRP bridge back up. So I would be hell happy with this scenario. Only I wouldn't be able to script this in Mikrotik environment so need to find out if something similar is already coded and I may edit it to my scenario.
 
sindy
Forum Guru
Forum Guru
Posts: 6865
Joined: Mon Dec 04, 2017 9:19 pm

Re: How to connect vrrp'ed routers to wan (ISP)

Tue Feb 16, 2021 4:39 pm

As suggested by @tdw, you actually don't need any script if the primary default route of each router goes via its own WAN and the secondary one goes via the second router, and you use the script-free monitoring of transparency of both WANs, as described here.

Or you can even use any of those fancy load distribution methods, allowing the active router to distribute traffic between both WANs. The only thing is that you'll have to use either policy routing and connection marking to make sure that the response packets received at standby router's WAN get forwarded via the active router, as otherwise they would take the short path to the LAN host and the TCP connection tracking would fail, and to prevent the looping (to exempt packets coming from the primary router from the load distribution).

The check-gateway process pings the monitored address once every 10 seconds, so this is roughly the fault detection delay. If it is not sufficient for you, you have to use scripting, but it will only shorten the fault detection delay to something like 2-3 seconds, so it is usually not worth the effort.

And the looping will only occur until the router detects the failure of the WAN path via the other router.
Instead of writing novels, post /export hide-sensitive. Use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1.
 
akarpas
Member Candidate
Member Candidate
Topic Author
Posts: 130
Joined: Tue Mar 20, 2018 4:46 pm

Re: How to connect vrrp'ed routers to wan (ISP)

Tue Feb 16, 2021 6:40 pm

As suggested by @tdw, you actually don't need any script if the primary default route of each router goes via its own WAN and the secondary one goes via the second router, and you use the script-free monitoring of transparency of both WANs, as described here.

Or you can even use any of those fancy load distribution methods, allowing the active router to distribute traffic between both WANs. The only thing is that you'll have to use either policy routing and connection marking to make sure that the response packets received at standby router's WAN get forwarded via the active router, as otherwise they would take the short path to the LAN host and the TCP connection tracking would fail, and to prevent the looping (to exempt packets coming from the primary router from the load distribution).

The check-gateway process pings the monitored address once every 10 seconds, so this is roughly the fault detection delay. If it is not sufficient for you, you have to use scripting, but it will only shorten the fault detection delay to something like 2-3 seconds, so it is usually not worth the effort.

And the looping will only occur until the router detects the failure of the WAN path via the other router.
its not clear I didn't get you fully, lets say master router connected to WAN 1 and backup router connected to WAN 2 so as per link provided I set up routes one toward WAN 1 other toward Backup router? If this is what you offer then if WAN 1 is down and alternative route points to WAN2 via router 2 and VRRP master is UP then backup router will direct data back to master router and master router back to backup router: LOOP
Or I didn't get you and would be mush appreciated if you could explain in more detail?
 
sindy
Forum Guru
Forum Guru
Posts: 6865
Joined: Mon Dec 04, 2017 9:19 pm

Re: How to connect vrrp'ed routers to wan (ISP)

Tue Feb 16, 2021 7:02 pm

The key here is the "policy routing" as mentioned above, which is a shortcut for "routing which takes into account not only the destination address but also other properties of the packet being routed".

In particular, you have to distinguish from where a packet came in. If it came in directly from the LAN host, it is free to take either of the two default routes (via the local WAN or via the other router's WAN); if it came in via the other router, it must not use the route back to that router. Similarly, a packet which came in via WAN must not be delivered directly to the IP of the LAN host if it is a response to a packet which previously came in via the other router, as otherwise the firewall at the other router would only see packets in one direction of the respective connection.

There are two ways to achieve this:
  • you can use src-nat when routing via the other router; this will make the other router see that the packet as coming from the local router's own address, so it can use e.g. an /ip route rule row matching on a particular src-address to choose a routing table that only contains a default route via its own WAN, and it will automatically deliver the response back to that address. The address used for the src-nat must not be from the LAN subnet to avoid ICMP redirect to be sent to the sender, but you can use the same link which the LAN subnet is using for the interconnection of the routers
  • you can use a dedicated VLAN (or a physical cable, whatever you prefer) to interconnect the routers for this purpose; in this case, the /ip route rule could match on interface, but you'll have to use /ip firewall mangle and connection-mark anyway as you'll have to use a dedicated routing table also for the responses (instead of the address at the local end, the connection-mark value will distinguish responses to packets which came in via that dedicated interconnection (V)LAN.
Instead of writing novels, post /export hide-sensitive. Use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1.
 
akarpas
Member Candidate
Member Candidate
Topic Author
Posts: 130
Joined: Tue Mar 20, 2018 4:46 pm

Re: How to connect vrrp'ed routers to wan (ISP)

Wed Feb 17, 2021 11:25 am

The key here is the "policy routing" as mentioned above, which is a shortcut for "routing which takes into account not only the destination address but also other properties of the packet being routed".

In particular, you have to distinguish from where a packet came in. If it came in directly from the LAN host, it is free to take either of the two default routes (via the local WAN or via the other router's WAN); if it came in via the other router, it must not use the route back to that router. Similarly, a packet which came in via WAN must not be delivered directly to the IP of the LAN host if it is a response to a packet which previously came in via the other router, as otherwise the firewall at the other router would only see packets in one direction of the respective connection.

There are two ways to achieve this:
  • you can use src-nat when routing via the other router; this will make the other router see that the packet as coming from the local router's own address, so it can use e.g. an /ip route rule row matching on a particular src-address to choose a routing table that only contains a default route via its own WAN, and it will automatically deliver the response back to that address. The address used for the src-nat must not be from the LAN subnet to avoid ICMP redirect to be sent to the sender, but you can use the same link which the LAN subnet is using for the interconnection of the routers
  • you can use a dedicated VLAN (or a physical cable, whatever you prefer) to interconnect the routers for this purpose; in this case, the /ip route rule could match on interface, but you'll have to use /ip firewall mangle and connection-mark anyway as you'll have to use a dedicated routing table also for the responses (instead of the address at the local end, the connection-mark value will distinguish responses to packets which came in via that dedicated interconnection (V)LAN.
I just want to clarify some things I'm happy than someone with great expertise is willing to help. As per my scenario on LAN side I have VRRP for multiple VLAN's so packets suppose to flow only via master router not via a backup router. Backup router kick in only if the master is down. A no-load balance between them. So all the time until the master router is up but WAN1 is down policy-based routing will route it to the backup router but at this point, the backup router knows the master is up and master should handle all the job so packets are sent back to mater router. To be routed via WAN1 to the Internet. It is so complicated. The second way you have advised a link between a router with different network and route via this link is more understandable ill give a try to it.
Maybe I'm wrong but using a script it still looks much more easy way to handle it. As I have tested with netwatch it works like a charm, WAN 1 gone! netwach senses it, disables VRRP master, Backup VRRP gets master all date goes via backup router and WAN2 easy peasy, WAN1 gets back netwatch senses it enables VRRP master , backup master gets backup again, the master gets up all data goes via WAN1. Why this simple method is not good?
I appreciate every reply, knowledge and experience you share with me. Thank a mill.
 
sindy
Forum Guru
Forum Guru
Posts: 6865
Joined: Mon Dec 04, 2017 9:19 pm

Re: How to connect vrrp'ed routers to wan (ISP)

Wed Feb 17, 2021 12:33 pm

Maybe I'm wrong but using a script it still looks much more easy way to handle it. ... Why this simple method is not good?
The above is quite in contrast with what you've stated earlier:
Only I wouldn't be able to script this in Mikrotik environment so need to find out if something similar is already coded and I may edit it to my scenario.
So my understanding was you suffer from scriptophobia, like many other forum members :)

But a more important reason is that coding any algorithms is always prone to errors, so you have to thoroughly test anything you write, and that processing the script introduces additional delay. But aside from this, if you don't mind treating the WAN and its serving router as a single block in the redundancy scheme, there's nothing wrong about it. And in every aspect except creation and debugging of the script, it is definitely simpler to set up than the one with policy routing and transparency monitoring using recursive next-hop search.

In fact, the only scripting part here is that on-up of the netwatch raises the VRRP priority and on-down decreases it. But this is only true if there is a single netwatch item pinging a single host in the internet; if you want to monitor multiple internet hosts, you have to check the state of the other netwatch(es) before changing the VRRP priority.

The advantage of the way I suggest is that it provides not only redundancy but also throughput improvement at WAN side plus no need to reconfigure the VRRP priorities depending on WAN state. If the WAN fails for any reason outside the router whilst the router itself remains fine, the VRRP doesn't need to switch over, it's just that the packets have to go through both routers.

One more point, you don't necessarily need to have just a single VRRP gateway on the LAN. If you create two VRRP addresses in the same subnet, with different priority on each router, one of them can be active at router A and the other one at router B when both are alive. So some LAN hosts may use router A and other may use router B while both work normally, and only use the other router if their preferred one dies. This reduces the impact of the failover on existing connections.
Instead of writing novels, post /export hide-sensitive. Use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1.

Who is online

Users browsing this forum: boocko, Google [Bot], hackerduni and 200 guests