Community discussions

MikroTik App
 
millenium7
Member
Member
Topic Author
Posts: 386
Joined: Wed Mar 16, 2016 6:12 am

Selective routing with failover in MikroTik - How?

Thu Jun 04, 2020 6:18 am

So I have 2 scenario's that I need to find a solution for

Scenario A: 1 hop selective routing
Image

RouterA & RouterB are running OSPF to each other, the 60ghz path is set to the default cost of 10 and is running BFD, the 5ghz path is a cost of 15
This works perfectly fine for fairly rapid failover for when the 60ghz link dies. Traffic successfully fails over to the 5ghz, all good for the most part

However it never just instantly drops out and remains dropped out when there's rain or a partial obstruction. What actually happens is the link flaps, or just performs like crap, this is very noticeable and very disruptive with VoIP. So what I would rather do is have all VoIP traffic permanently routed across the 5ghz link so it never has to deal with a flapping link situation
I can accomplish this with a simple mangle rule that changes to i.e. 'BackupRoutingTable' which just has 1 route in it - a default route to the IP address on RouterA's 5ghz interface.

Great........ until the 5ghz link fails. RouterB will keep sending VoIP traffic out that interface unless the physical interface goes down, since its a local IP address it's not a loopback
So how do I solve this problem?

Scenario B: multi-hop selective routing
Image

If we can figure out Scenario A it might solve Scenario B as well, but there a potential problem especially if we are using mangle rules
At RouterC it's pretty straightforward to influence the path. If I want all traffic from Customer X to go via Internet1, it'll send to RouterB who will then send to RouterA and out the internet
And all traffic from Customer Y to go via Internet2, it'll send to RouterD->RouterE. This is very simple because RouterB just looks at its routing table and see's the closest path to the internet is via RouterA, and the same with RouterD to RouterE, easy

But what about for CustomerZ? If I have a rule on RouterE that say I want all of his traffic to go via Internet1, it will send to RouterD. But RouterD will see that the closest path to the internet is via RouterE so it'll just send the data back, and then RouterE will send the traffic out Internet2 (or cause a loop by routing it back to RouterD)
Mangle cannot be used here, because connection/packet data does not carry across so there is no way for RouterD to know for sure "oh, this traffic is supposed to go via Internet1" it's just going to look at its normal routing database, it won't use mangle

So what are the possibilities here?
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 8793
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: Selective routing with failover in MikroTik - How?

Thu Jun 04, 2020 2:20 pm

First, I am not a networking wizard so cannot help but, great explanation and diagrams.
Since no other experts have chimed in, I thought I would give my low level responses.

In the first issue, it sounds as you want the 5ghz link to switch but only after it is assured the link is hard down vice flapping.
Sounds like a script that checks over a 'period of time' that there is no connectivity and then switch.
Assuming that you want the router to keep checking the 5ghz link to go back, but with the same caveat - that the 5ghz link is up solid for some 'period of time'

The second scenario, what if Router A gives out the DHCP for customer Z (vlan zzz) , would that then change the routing ??
I'd rather manage rats than software. Follow my advice at your own risk! (Sob & mkx forced me to write that!)
MTUNA Certified, by the Ascerbic Llama!
 
solar77
Long time Member
Long time Member
Posts: 580
Joined: Thu Feb 04, 2016 11:42 am
Location: Scotland

Re: Selective routing with failover in MikroTik - How?

Thu Jun 11, 2020 5:00 pm

2nd non-routing export. I am here because anav messaged me...

scenario A, I suggest to use bounding, which respond to line failure quickly, unlike OSPF

scenario B. I think I would use OSPF and let it figure out the best path but BGP , from what I understand, would be quicker to recover in the event of topology change. not something I've done ofcourse so happy to learn something new.

Also you can use mangle rule to force routing decision on top of OSPF (that's the only way I managed to do this) so you can manually divide customers up and decided which customer goes to which gateway. not the best idea in all cases but it worked for us.
MTCNA MTCTCE UEWA
 
millenium7
Member
Member
Topic Author
Posts: 386
Joined: Wed Mar 16, 2016 6:12 am

Re: Selective routing with failover in MikroTik - How?

Wed Feb 10, 2021 9:38 am

I've managed to get this to work but its a lot of commands and a bit messy. Surely there's a cleaner and simpler way.....


The way i'm accomplishing it at the moment is to
- Create another VLAN and IP addresses on interfaces between routers
- in Route->VRF add those VLAN's with a routing mark like "SendOverBackup"
- Create another OSPF Instance, router-ID can be the same, set redistribute default route to always, routing-table set as before, use DN manually turned 'off' (otherwise routes don't advertise properly)
- Add another OSPF area 0 under that new Instance
- Add IP addresses of those VLAN's in networks
- Setup OSPF interface parameters (higher cost on the 60ghz link so its less preferred)
- Create new interface list and add those VLAN's
- Add mangle rule that marks connections that came in via those VLAN's
/ip firewall mangle add action=mark-connection chain=prerouting in-interface-list=BackupLinks new-connection-mark=fromSendOverBackup_c
- Add mangle rule to return the traffic through the same VRF, since MikroTik routing will not do this by default
/ip firewall mangle add action=mark-routing chain=prerouting connection-mark=fromSendOverBackup_c in-interface-list=!BackupLinks new-routing-mark=OtherWay
Now the objective is accomplished of having all data normally go via 60ghz, with fast fail-over thanks to BFD. And the opposite with another OSPF instance running to use the 5ghz as primary and 60ghz as backup with fast failover in the other direction
Now traffic can selectively be marked to always prefer the backup link

I just think this is messy and needlessly complicated with a lot of extra steps to achieve the goal. Does anyone know of a better/simpler method?
Remember, it CANT just be a simple mangle rule to force all traffic over backup. The goal is to have failover in BOTH paths. Just doing a simple mangle rule will mean all traffic will always go via backup path, even if its down and unreachable because they are radio links. Physical interface state will remain up (ethernet to radio) but there's no connectivity (radio to radio), hence this will never failover correctly. OSPF or BGP is needed
 
pe1chl
Forum Guru
Forum Guru
Posts: 7793
Joined: Mon Jun 08, 2015 12:09 pm

Re: Selective routing with failover in MikroTik - How?

Thu Mar 18, 2021 11:52 am

I think your only real option for routing differently depending on packet marks (e.g. based on DSCP or other kinds of SLA) is to have multiple different routing tables each maintained by a separate instance of a routing protocol (or different routing protocols), and using a selection of the routing table that is the same all through the network.
In your case: you maintain a separate routing table for VoIP and select it based on DSCP 46 or "upper 3 bits of DSCP are 5".
The routing table (also called "routing mark" in RouterOS is maintained by a routing protocol instance that is tuned differently, and emphasizes on reliable paths rather than fast paths.
To get this working OK in more complex networks than you picture it is essential that all the nodes in the network are configured the same, and that there are no nodes where e.g. the routing table selection based on DSCP is forgotten or is different. Because that would easily result in routing loops.
 
sarah
newbie
Posts: 27
Joined: Mon Feb 29, 2016 1:41 am

Re: Selective routing with failover in MikroTik - How?

Sat Mar 20, 2021 3:28 pm

I can accomplish this with a simple mangle rule that changes to i.e. 'BackupRoutingTable' which just has 1 route in it - a default route to the IP address on RouterA's 5ghz interface.

Great........ until the 5ghz link fails. RouterB will keep sending VoIP traffic out that interface unless the physical interface goes down, since its a local IP address it's not a loopback
So how do I solve this problem?
Just wondering if you add second default route to IP address of RouterA's 60ghz interface with higher distance, and add check gateway for both routes (5ghz and 60ghz). Not 'seamless' fail over though as check gateway needs about 1/2 minute to takes effect. :(

On second thought it may not work because of the return path will still prefer 1 link over the other.
RBmAPL-2nD, RBmAP2nD, RB951Ui-2HnD, RB951G-2HnD, RB2011UiAS-2HnD-IN, RBwAPG-5HacT2HnD, RB750Gr3, RBcAPGi-5acD2nD, RB4011iGS+RM, CRS326-24G-2S+RM, RB960PGS-PB, RB260GSP, CSS610-8G-2S+IN. yet, still more to come...
 
User avatar
IPANetEngineer
Trainer
Trainer
Posts: 1470
Joined: Fri Aug 10, 2012 6:46 am
Location: Jackson, MS, USA
Contact:

Re: Selective routing with failover in MikroTik - How?

Sat Mar 20, 2021 5:11 pm

We have solved similar challenges for client WISP networks using this design...

https://mum.mikrotik.com/presentations/ ... 062656.pdf

Image
Global - MikroTik Support & Consulting - English | Español | Serbian | Danish +1 855-645-7684
https://iparchitechs.com/ecosystem/mikr ... consulting mikrotiksupport@iparchitechs.com
 
millenium7
Member
Member
Topic Author
Posts: 386
Joined: Wed Mar 16, 2016 6:12 am

Re: Selective routing with failover in MikroTik - How?

Sun Mar 21, 2021 1:44 am

Thanks, I read through it. If I understand correctly, you are manipulating the traffic direction for a destination subnet. This seems like it could work if a client is given 2 IP addresses, 1 is used for regular data, the other is used for voice traffic
This way you can influence voice to use the less preferred path

is this correct?
so it's got nothing to do with traffic classes, it's strictly a destination address/subnet

it's a small step in the right direction but it's not really an optimal one. seems it adds quite a bit of administrative overhead and requires we double up on IP addresses to customers (we give out public most of the time) and also requires more configuration at clients router to split up subnets
MPLS also does not use BGP routes so not sure how it'll affect things there, it would likely break VPLS tunnels or MPLS VPNs

how's the failover time? we care more about quality of traffic flow than outright bandwidth throughput
 
pe1chl
Forum Guru
Forum Guru
Posts: 7793
Joined: Mon Jun 08, 2015 12:09 pm

Re: Selective routing with failover in MikroTik - How?

Sun Mar 21, 2021 10:54 am

Yes I agree that it would be nice to have routing dependent on class of service. I posted my reply above as a copy of the same reply in a different topic, of course here it is a bit redundant because what I wrote is mostly what you already stated as not desirable.
But I think that when you make routing decisions based on class of service only with a local viewpoint, it will fail in all but the most trivial cases (like what you pictured above) where there essentially are only two links between two endpoints and you only need to decide which link to use.
When your network is a bit less trivial than that, and is e.g. more like what IPANetEngineer's picture shows, such local next hop decisions will result in routing loops!
In that case it is unavoidable to have separated routing tables per class of service and strict adherence to selection of the correct routing table for each packet.
But then indeed you get the big burden of maintaining two differently configured routing configurations and maybe even are forced to use extra stuff like VLANs or other forms of tunnels to make the two routing configurations co-exist.
In BGP it is possible to have different instances, but I have never tried what happens when you configure 2 instances each with the same set of peers. Will the 2 instances nicely live together and keep everything separated by using 2 sets of AS numbers, or will the peerings complain about wrong AS number. Probably the latter. So that would still at least require different addresses on the endpoints for each peering.
It is a lot of work, and one small mistake will cause big mishaps.

Who is online

Users browsing this forum: No registered users and 7 guests