Well when you post the complete config I can comment. Not sure why you are using VRFs at all, yet.
Assuming its 3 ISP modems into one router.
The configuration has already been posted in post #37 which is the setup we are talking about, which includes the configuration AND a diagram, no need to assume anything.
The ONLY difference between what is posted, reproducible in CHR, and the real install is that in the latter ether4 is used instead of ether8 and the vrf is called vrf4 instead of vrf8.
The three “upper” router are there in the CHR diagram just to simulate the three ISP routers, for all that matters they are three (in real installation at the moment only two) “black boxes” with 192.168.1.1 on the “inner” side and internet on the “outer” side, I had to add them to be able to do tests with disconnection of cables/power down/etc. as the “cloud” in CHR is “always” on.
As I stated, thus far no reason to use VRF has been provided and as a matter of fact it would seem NOT appropriate in this case.
Further, your recursive is incorrect.
Simple solution works:
/ip address
add address=192.168.1.241 interface=ether1 network=192.168.1.1
add address=192.168.1.242 interface=ether2 network=192.168.1.1
add address=192.168.1.243 interface=ether3 network=192.168.1.1
/ip route
add check-gateway=ping distance=2 dst-address=0.0.0.0/0 gateway=1.1.1.1 scope=10 target-scope=12
add distance=2 dst-address=1.1.1.1/32 gateway=192.168.1.1%ether1 scope=10 target-scope=11
++++++++++++
add check-gateway=ping distance=4 dst-address=0.0.0.0/0 gateway=9.9.9.9 scope=10 target-scope=12
add distance=4 dst-address=9.9.9.9/32 gateway=192.168.1.1%ether2 scope=10 target-scope=11
+++++++++++
add check-gateway=ping distance=6 dst-address=0.0.0.0/0 gateway=8.8.8.8 scope=10 target-scope=12
add distance=6 dst-address=8.8.8.8/32 gateway=192.168.1.1%ether3 scope=10 target-scope=11
/ip firewall nat
add action=src-nat chain=srcnat out-interface=ether1 to-addresses=
192.168.1.241
add action=src-nat chain=srcnat out-interface=ether2 to-addresses=
192.168.1.242
add action=src-nat chain=srcnat out-interface=ether3 to-addresses=
192.168.1.243
Then I imagine, one could assign any specific Routing rules or mangles as required using the below setup.
add fib name=use-ISP1
add dst-address=0.0.0.0/0 gateway=192.68.1.1%ether1 routing-table=use-ISP1
Well, it didn’t work when I tried it, as ether8 in the simulation (ether4 in the real setup) have the same address 192.168.1.1 (same network) as the ISP routers, that is the reason why a VRF or the other approach by mtest01 were tested, compare also with:
http://forum.mikrotik.com/t/failover-between-2-isps-using-gateways-with-same-ip-was-nat-traffic-to-vrf/170381/23
If there is a way to route on a same device between ether4 192.168.1.1 (LAN) and ether1 192.168.1.241, ether2 192.168.1.242, ether3 192.168.1.243 (WAN) with gateway for each of these three ports 192.168.1.1, without using VRF or other complex trickery, I am all ears.
Yes, there is, provided above. Probably it didnt work because your recursive was not correct.
Recursive has nothing to do with the issue I had, at least one route should have not been USHI.
And at the time, before trying with VRF and with the other method using proxy ARP, I did try the “plain” setup you propose with just one route/one ISP connection and it didn’t work.
However when I will have some spare time I will try again.
If you have the time/will can you explain what exactly is “wrong” in the recursive I used? (I have tested both in the simulation and on real setup and it worked just fine).
Your discriminatory skills are weak, hint. look at SCOPE!
Undoubtedly I miss the discriminatory skills you are gifted with or that you gained through years of experience.
Still no idea what the mistake (if any) is but - again - the setup works as is and even if the failover does not work (but It does work) at least one route should have been AS after the reboot and the other ones S, none should have been USHI.
The three routes for the DNS servers pointing to %ether1, %ether2 and %ether3 are normally all AS and should return to that state after reboot, or at least one of them should, as long as there is a cable in the interface and the modem on the other side of the cable Is working.
Man do I have to state it in writing, your SCOPES are wrong!! LOL
The config I gave works, its your config that is broken if it doesnt.
I cannot read a winbox jpeg unless its very clealry delineated
An RSC script I can read in seconds… its just a story about requirements
I cannot make up scripts for automation, aka in system scripts, or netwatch, worth a can of beans…
I can vlan wlans with eyes closed until capsman enters the picture and then I give up.
Ok, thank you anyway for the help and suggestions.
This conversation seems to have no SCOPE anymore (pun intended).
No one can force you to actually fix your config, that motivation has to come from within…
Please, do not misunderstand me.
I posted about an issue that I experienced.
I want to understand the reason why that issue happened.
Then, I am of course very open to any constructive suggestions that may help in making a better configuration.
But you started posting snippets that - without an explanation - make no sense to me, very likely your approach works with people much more experienced than myself, but not with beginners like myself.
I do understand how you may have no time/will (or patience) to teach things that - for you - are well understood and easy, but it is not at all useful to me to get partial snippets and been shout at because I don’t understand them and don’t want to blindly make changes to a working configuration before having grasped the concepts.
As such there is no point in continuing asking for clarifications/explanations that you are not going to give.
Not sure to understand the magnitude of the problem here, loosing 7 pings after a complete loss of power is no big deal in my opinion.
Maybe you could try to tune your netwatch parameters: interval=10s timeout=5s
In the worst case scenario it will take 10 seconds + 5 seconds to execute the fail-over script. And in my experience Netwatch takes much longer to react. I have the same settings as you and I noticed that it takes 45 seconds to 1 minute to switch over. My assumption is that Netwatch does not handle well the situation when there is no route to the host being monitored. as opposed to “there is a route but the ping times out”.
Well I am surprized because you went down the rabbit hole of a very complex VRF config on one hand, but then seemingly want to avoid like the plague a KISS solution ???
In any case, if there is no issue requiring solving I will move on.
The use of the VRFs is there for a different reason: both ISP1 and ISP2 routers have the same IP i.e. 192.168.1.1
@mtest001
In the configuration I had, there was no Netwatch script running, I added the described one later as a (temporary for the moment) fix for next occasion.
Pinging from a device on the network seemed to have no effect “host unreachable”.
No idea if - giving it some time - the check gateway would have done something, but that should ping every 10 seconds and more than 70-80 seconds had passed, I suspect since the routes where the check-gateway checks are were USHI there was no attempt at all from them.
BTW the 7/8/10 pings to 8.8.8.8 are “generic”, i.e. no interface is speciified in the command.
The issue is that I cannot seem to reproduce in gns3/CHR and testing on the “real” device is complicated because of the various internet users I have around that seem all unable to be disconnected for longer than a few minutes.
I will have to take the spare Ax lite I have and experiment with it on the desktop.
The timing for the recovery in itself is not a problem, as the “main” ISP1 router is slow anyway, and even the secondary one ISP2, while faster, is not a rocket, so that the reboot of the Ax lite is faster than both, hence the use - for now - of Netwatch, the cycle for that script is 30 seconds, so in case of failure there is the pinging and - provided that it restores the routes as AS - the re-connection will happen at the most in some 40 seconds (the fact that the next run of netwatch will happen some 20 seconds later is irrelevant, as that will only acknowledge that the connection has been restored.
What I don’t know (didn’t have time to test) is whether the pinging needs to actually succeed or if it is enough to “wake” the routes even if the ISP modem(s) are not (yet) ready.
Typically these modems respond to pings to them (192.168.1.1) very soon, then it takes them from several seconds to a few minutes to actually have a connection established and allowing pinging to 8.8.8.8, but the no route and the USHI should mean that not even the modem in itself is detected so maybe the pinging works even before the connection is established.
@anav
I went through this rabbit hole only because I couldn’t found any reference to a complete, working, simpler configuration, while I found endless examples/posts by people stating that routing within a same set of network addresses is not possible.
If you happen to have a reference to a complete example (or wish to produce one) please post it.
I will re-state, just in case, that the original intended use case is the following:
- I have three ISP devices (one “main”, ISP1, and two secondary and tertiary, ISP2 and ISP3) and all of them have a fixed address of 192.168.1.1(/24)
- my usual way to deal with the “main” router/modem having not internet was to unplug the RJ45 from the ISP1 and plug it in ISP2 (the procedure if also ISP2 was down and ISP3 was needed was slightly different as the LTE modem is in another room, so let’s reduce the problem to only ISP1 and ISP2)
- the devices on the network (LAN) have all a static address in the 192.168.1.0/24 range and cannot be changed, with gateway set to 192.168.1.1
- the role of the device (currently an AX lite) is to have a LAN port with address 192.168.1.1 (/24) on ether4 (LAN) and three whatever addresses on ether1/ether2/ether3 each capable of routing to a destination of 192.168.1.1 ( the three ISP modems) and from there to the internet.
As soon as I have time, I will try to pinpoint what happens in this newly discovered hiccup when a loss of power is involved, once I will have the problem more exactly described (and hopefully a better solution than the current “ugly” workaround) I will be able to try again other (possibly) simpler approaches.
The pings can be forced through a specific VRF. Try it next time to confirm if the ping fails for 1 VRF and works for the other.
Find a maintenance window and tell your users. Try somehing like very early in the morning like 5 or 6 AM or during week-ends.
There is only one VRF and it has only the ether4 (LAN) port (ether8 in the simulation).
What I was trying to say is that in “normal” operation I have:
the three routes main_ISP**n_Route: #1 AS and #2 and #3 S
the three routes vrf8_ISPn_Route: #1 AS and #2 and #3 S
the three routes vrf8_ISPn**_Ping: all three AS ← and when one or two of these go down the above routes will switch or change status.
In any case, in any situation of the modems/connections they are all either S or AS, while when the hiccup happened they were all USHI.
The generic ping restored them all (not just the set through which the ping must have gone through), this is what makes me think that it is something more like a “freeze” or “sleep” of the whole “routing stuff” and the pings acted as a wake signal.
But as said I only had a quick look before implementing the workaround and giving back the internet to the hungry, and I had/have some other things to do, I will have sone time to delve into the matter in ten days time.
And thanks, I know the “trick” of working while people sleep, but the tests/experiments I need to do may take their time, it is much better if I do them in a (this time physical) simulation, so I have all the time needed, only once I have something working I will actually put the new configuration in operation during one such maintenance slot.
Use of VRFs is warranted if if each router is serving different subnets.
VRF is NOT warranted for distribution to the same LAN ( regardless of number of subments )
The simple method presented works just fine, so I am not clear as to why you would want to attempt a complex and incorrect approach.
you maybe dont get the point ，three isp routesrs have the same ip address 192.168.1.1 ，you can not put eth1 2 3 in main at the same time ,because arp list about 192.168.1.1 only can save one entry ,that means just one isp line can work at the same time , your method gateway=192.168.1.1%eth1 %eth2 %eth3 was wrong , so only vrf can do with this situation