Community discussions

MikroTik App
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Fri Sep 30, 2022 8:10 am

Hello Everyone!

Another query here that needs your expertise. May I ask for your analysis and suggestions please on how to fix (if there is something wrong) or improve below recursive routing configuration? This sample config was made by someone and I want to understand how this config works. I don't understand why there is no target scope set on the default routes. And what are those virtual routes for? Also, with the policy based routing included, would the traffic under routing-marks "to-gw2" or "to-gw3" automatically rerouted to other ISP should the corresponding assigned ISP went down?
Note: ISP1 and ISP2 have public static IP addresses ,hence, I replaced them with xxx.xxx for security purposes.


/ip route
add comment="ISP1 route" distance=1 gateway=8.8.8.8 routing-mark=to-gw1 scope=10
add comment="ISP2 route" distance=1 gateway=10.2.2.2 routing-mark=to-gw2 scope=10
add comment="ISP3 route" distance=1 gateway=208.67.222.222 routing-mark=to-gw3 scope=10

add comment="default route ISP1" distance=1 gateway=8.8.8.8 scope=10
add comment="failover route ISP2" distance=2 gateway=10.2.2.2 scope=10
add comment="failover route ISP3" distance=3 gateway=8.8.4.4 scope=10

add comment="monitoring ip ISP1" distance=1 dst-address=9.9.9.9/32 gateway=xxx.xxx.178.121 scope=10
add comment="monitoring ip ISP2" distance=1 dst-address=1.1.1.1/32 gateway=xxx.xxx.181.89 scope=10
add comment="monitoring ip ISP3" distance=1 dst-address=8.26.56.26/32 gateway=192.168.1.1 scope=10

add check-gateway=ping comment="virtual route ISP1" distance=1 dst-address=8.8.8.8/32 gateway=\
9.9.9.9 scope=10
add check-gateway=ping comment="virtual route ISP2" distance=1 dst-address=10.2.2.2/32 gateway=\
1.1.1.1 scope=10
add check-gateway=ping comment="virtual route ISP3" distance=1 dst-address=208.67.222.222/32 \
gateway=8.26.56.26 scope=10
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Sat Oct 01, 2022 3:19 pm

anyone can help please? thank you in advance.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Sat Oct 01, 2022 3:24 pm

This sample config was made by someone
The most logical thing you need to do is write on the topic where you found those rules,
or ask whoever wrote it.
On the forum are already present dozen of examples and some are well explained.
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Sat Oct 01, 2022 4:21 pm

This sample config was made by someone
The most logical thing you need to do is write on the topic where you found those rules,
or ask whoever wrote it.
On the forum are already present dozen of examples and some are well explained.
Thank you for the response. This existing routing config in a running (production) MT was done by someone who is no longer reachable right now, hence, the non-techie owner of the network has ask me for help to check if the said config is correct or needs to improve.

Would appreciate if you can point me to specific links with almost same scenario (3 ISPs and 3 routing-mark assigned to each WAN) using PBR and Recursive Routing failover.
Thanks.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Sat Oct 01, 2022 6:46 pm

You can use search function on forum, is free
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Sat Oct 01, 2022 11:39 pm

 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Sun Oct 02, 2022 3:36 am

Thank you very much Sir.
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Sun Oct 02, 2022 10:33 am


Hello Sir,

After reading your routing guide, I was able to come up with this trial config. Would appreciate much if you can help me check for possible errors before I will implement this to the production MT and change its current routing configuration.

By the way, the expectation is that all three marked-routing traffic should maintain their internet access even if their default assigned ISP/WAN is down.

for example:

TRAFFIC#1: "to-gw1 traffic to ISP1" or "to ISP2 if ISP1 is down" or to ISP3 if both ISP1 & ISP2 are down.

TRAFFIC#2: "to-gw2 traffic to ISP2" or "to ISP1 if ISP2 is down" or "to ISP3 if both ISP1 & ISP2 are down"

TRAFFIC#3: "to-gw3 traffic to ISP3" or "to ISP1 if ISP3 is down" or "to ISP2 if both ISP3 & ISP1 are down"

Thank you.




+++TRIPLE WAN FAILOVER THREE RECURSIVE ROUTES (NESTED) WITH PBR+++

/ip firewall address-list
add address=10.100.100.0/24 list=STANDARD
add address=10.100.200.0/24 list=STANDARD
add address=10.200.123.0/24 list=VIP
add address=172.20.234.0/24 list=VIP
add address=192.168.111.0/24 list=OTHERS
add address=192.168.222.0/24 list=OTHERS

/ip firewall mangle
add action=mark-routing chain=prerouting comment=To-ISP1 new-routing-mark=to-gw1 passthrough=no src-address-list=STANDARD
add action=mark-routing chain=prerouting comment=To-ISP2 new-routing-mark=to-gw2 passthrough=no src-address-list=VIP
add action=mark-routing chain=prerouting comment=To-ISP3 new-routing-mark=to-gw3 passthrough=no src-address-list=OTHERS
++++++++++++++++++++++++++

/ip route
add comment="Virtual Route ISP1" dst-address=0.0.0.0/0
gateway=10.1.1.1 scope=10 target-scope=14
+++++++++++++++++++++
add comment="Monitoring ISP1-A" check-gateway=ping dst-address=10.1.1.1/32
gateway=8.8.8.8 scope=10 target-scope=13
add comment="Primary Route ISP1-A" distance=1 dst-address=8.8.8.8/32
gateway=XXX.XXX.181.89 scope=10 target-scope=12
add comment="Primary Route ISP1-A to-GW1" distance=1 dst-address=8.8.8.8/32
gateway=XXX.XXX.181.89 routing-mark=to-gw1 scope=10 target-scope=12
++++++++++++++++++++++
add comment="Monitoring ISP1-B" check-gateway=ping dst-address=10.1.1.1/32
gateway=76.76.2.0 scope=10 target-scope=13
add comment="Primary Route ISP1-B" distance=1 dst-address=76.76.2.0/32
gateway=XXX.XXX.181.89 scope=10 target-scope=12
add comment="Primary Route ISP1-B to-GW1" distance=1 dst-address=76.76.2.0/32
gateway=XXX.XXX.181.89 routing-mark=to-gw1 scope=10 target-scope=12
++++++++++++++++++++++++++++

add comment="Virtual Route ISP2" dst-address=0.0.0.0/0
gateway=10.2.2.2 scope=10 target-scope=14
+++++++++++++++++++++
add comment="Monitoring ISP2-A" check-gateway=ping dst-address=10.2.2.2/32
gateway=9.9.9.9 scope=10 target-scope=13
add comment="Secondary Route ISP2-A" distance=2 dst-address=9.9.9.9/32
gateway=192.168.1.1 scope=10 target-scope=12
add comment="Secondary Route ISP2-A to-GW2" distance=2 dst-address=9.9.9.9/32
gateway=192.168.1.1 routing-mark=to-gw2 scope=10 target-scope=12
++++++++++++++++++++++
add comment="Monitoring ISP2-B" check-gateway=ping dst-address=10.2.2.2/32
gateway=208.67.222.222 scope=10 target-scope=13
add comment="Secondary Route ISP2-B" distance=2 dst-address=208.67.222.222/32
gateway=192.168.1.1 scope=10 target-scope=12
add comment="Secondary Route ISP2-B to-GW2" distance=2 dst-address=208.67.222.222/32
gateway=192.168.1.1 routing-mark=to-gw2 scope=10 target-scope=12
+++++++++++++++++++++++++

add comment="Virtual Route ISP3" dst-address=0.0.0.0/0
gateway=10.3.3.3 scope=10 target-scope=14
+++++++++++++++++++++
add comment="Monitoring ISP3-A" check-gateway=ping dst-address=10.3.3.3/32
gateway=1.1.1.1 scope=10 target-scope=13
add comment="Tertiary Route ISP3-A" distance=3 dst-address=1.1.1.1/32
gateway=XXX.XXX.178.121 scope=10 target-scope=12
add comment="Tertiary Route ISP3-A to-GW3" distance=3 dst-address=1.1.1.1/32
gateway=XXX.XXX.178.121 routing-mark=to-gw3 scope=10 target-scope=12
++++++++++++++++++++++
add comment="Monitoring ISP3-B" check-gateway=ping dst-address=10.3.3.3/32
gateway=76.76.19.19 scope=10 target-scope=13
add comment="Tertiary Route ISP3-B" distance=3 dst-address=76.76.19.19/32
gateway=XXX.XXX.178.121 scope=10 target-scope=12
add comment="Tertiary Route ISP3-B to-GW3" distance=3 dst-address=76.76.19.19/32
gateway=XXX.XXX.178.121 routing-mark=to-gw3 scope=10 target-scope=12
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Sun Oct 02, 2022 8:51 pm

are the sequence of the entries/lines correct? does the order/sequence in /ip routes matters like those in /ip filter and mangle rules?

how about the distance, am I not missing something?

i hope someone can help me look and check my routing config. thank you.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Mon Oct 03, 2022 4:12 am

Will look at this tomorrow, too fried tonight to make sense of it, but good questions!!!
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Mon Oct 03, 2022 4:31 am

Will look at this tomorrow, too fried tonight to make sense of it, but good questions!!!
Thank you Sir Anav.
Aside from your guide/tutorial, i also read Chupaka's thread and other similar threads regarding recursive failover routing.
Most of the examples are dual WAN only, hence, i need to be sure if what i am doing for a triple WAN recursive failover is correct.
Unfortunately, i don't have a bench MT units to experiment right now.

In addition to my questions above, i noticed in your sample config that the target-scope of the secondary ISP (back-up) is 30. May i know what's the rational explanation why it should be target-scope=30 and not 20 or 17 or 15 or lower? In a Triple WAN recursive failover scenario, what should be the target scope of the secondary and tertiary routes? In my Triple WAN recursive (nested) config above, the scope and target-scope settings are just identical to each other for the three ISP routes. Is my config wrong?

Also, with regards to the distances, I only set the distance (based on priority 1, 2, 3) on the resolving routes and not on the monitoring (recursive) routes. Is my config correct?
What is the difference between simply using distances 1, 2 & 3 for the primary, secondary and tertiary routes respectively and by using other higher numbers for the distance like for example 5, 10, 15 or 1, 10, 20 or 100, 200, 300? does the numbers really matter? does it affect the length of time to shift from a broken route to a good route?

Looking forward to your analysis and recommendations.
Thank you and have a good sleep.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Mon Oct 03, 2022 3:13 pm

Just to confirm you have three groups of users, each should use a different wan as their primary.
There is no concern for incoming traffic externally originated, this setup is for outgoing traffic originated on the LANs?

Also , what I dont understand is why are you mangling if all the groups of users involved are whole subnets?
One should be able to avoid mangling and use routing rules etc........

Are all the addresses on the lists above local subnets behind the router ? Assuming yes.

ANother question, do any of the users need to access other subnets on the router. Since we are directing all users out a specific wan, we need to know if there any exceptions, when going outside ones own subnet. For example is their a shared printer or a local server that users will be crossing over in terms of subnets to get to............???
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Mon Oct 03, 2022 7:41 pm

Just to confirm you have three groups of users, each should use a different wan as their primary.
There is no concern for incoming traffic externally originated, this setup is for outgoing traffic originated on the LANs?

Also , what I dont understand is why are you mangling if all the groups of users involved are whole subnets?
One should be able to avoid mangling and use routing rules etc........

Are all the addresses on the lists above local subnets behind the router ? Assuming yes.

ANother question, do any of the users need to access other subnets on the router. Since we are directing all users out a specific wan, we need to know if there any exceptions, when going outside ones own subnet. For example is their a shared printer or a local server that users will be crossing over in terms of subnets to get to............???

Yes Sir, users were grouped into three and each group has assigned primary route/WAN/ISP to access the internet. And yes the concern is the traffic from LANs going to the internet.

I am sorry but i really don't fully understand yet how to use the routing rules in lieu to mangle rules. most of the video tutorials i found in youtube are using firewall address list and then mangle rules. would highly appreciate if you can teach me how to properly do it (routing rules) to be able to separate each group of users.

Yes, all addresses in the address lists are local subnets behind the router.

No shared printers nor servers, preferably each user should not be able to communicate each other locally.
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Mon Oct 03, 2022 7:49 pm

Just to confirm you have three groups of users, each should use a different wan as their primary.
There is no concern for incoming traffic externally originated, this setup is for outgoing traffic originated on the LANs?

Also , what I dont understand is why are you mangling if all the groups of users involved are whole subnets?
One should be able to avoid mangling and use routing rules etc........

Are all the addresses on the lists above local subnets behind the router ? Assuming yes.

ANother question, do any of the users need to access other subnets on the router. Since we are directing all users out a specific wan, we need to know if there any exceptions, when going outside ones own subnet. For example is their a shared printer or a local server that users will be crossing over in terms of subnets to get to............???

Yes Sir, users were grouped into three and each group has assigned primary route/WAN/ISP to access the internet. And yes the concern is the traffic from LANs going to the internet.

I am sorry but i really don't fully understand yet how to use the routing rules in lieu to mangle rules. most of the video tutorials i found in youtube are using firewall address list and then mangle rules. would highly appreciate if you can teach me how to properly do it (routing rules) to be able to separate each group of users.

Yes, all addresses in the address lists are local subnets behind the router.

No shared printers nor servers, preferably each user should not be able to communicate each other locally.
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Mon Oct 03, 2022 8:00 pm

by the way, i am using RouterOS 6.49.6

i can't really figure out where to look that /routing rules tab

what i have found is only /routing with below sub categories
BFD
BGP
Filters
MME
OSPF
Prefix List
RIP

i can't find the Rules tab under Routing.
 
Sob
Forum Guru
Forum Guru
Posts: 9119
Joined: Mon Apr 20, 2009 9:11 pm

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Mon Oct 03, 2022 8:05 pm

In v6 it's in IP->Routes on Rules tab.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Mon Oct 03, 2022 8:35 pm

You can use any numbers for distance as long as they are separated.
I setup the scope and target scope numbers such that they are also legal for Vers7 when you switch.
The only difference between V6 and V7 for this particular config would be the need to create 3 tables and
on the second IP route for each ISPX, use table= vice routing-mark=
......
VERSION 6

dst-address=0.0.0.0/0 gateway=10.10.10.10 scope=10 target-scope=14 table=main  distance=5
dst-address=0.0.0.0/0 gateway=10.10.10.10 scope=10 target-scope=14 routing-mark=ISP1_route distance=5
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
add check-gateway=ping dst-address=10.10.10.10/32 gateway=9.9.9.9 scope=10 target-scope=13
add dst-address=9.9.9.9/32 gateway=PrimaryISP-gatewayIP scope=10 target-scope=12
add check-gateway=ping dst-address=10.10.10.10/32 gateway=1.0.0.1 scope=10 target-scope=13 
add dst-address=1.0.0.1/32 gateway=PrimaryISP-gatewayIP scope=10 target-scope=12

..................................................................................................................................................

dst-address=0.0.0.0/0 gateway=10.10.10.10 scope=10 target-scope=14 table=main distance=10
dst-address=0.0.0.0/0 gateway=10.10.10.10 scope=10 target-scope=14 routing-mark=ISP2_route  distance=10
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
add check-gateway=ping dst-address=10.10.10.10/32 gateway=9.9.9.9 scope=10 target-scope=13
add dst-address=9.9.9.9/32 gateway=SecondarISP-gatewayIP scope=10 target-scope=12
add check-gateway=ping dst-address=10.10.10.10/32 gateway=1.0.0.1 scope=10 target-scope=13 
add dst-address=1.0.0.1/32 gateway=SecondaryISP-gatewayIP scope=10 target-scope=12

......................................................................................................................................................

dst-address=0.0.0.0/0 gateway=10.10.10.10 scope=10 target-scope=14 table=main distance=20
dst-address=0.0.0.0/0 gateway=10.10.10.10 scope=10 target-scope=14 routing-mark=ISP3_route  distance=20
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
add check-gateway=ping dst-address=10.10.10.10/32 gateway=9.9.9.9 scope=10 target-scope=13
add dst-address=9.9.9.9/32 gateway=TertiaryISP-gatewayIP scope=10 target-scope=12
add check-gateway=ping dst-address=10.10.10.10/32 gateway=1.0.0.1 scope=10 target-scope=13 
add dst-address=1.0.0.1/32 gateway=TertiaryISP-gatewayIP scope=10 target-scope=12

/ip routing rules
add src-address=subnetA  action=lookup table=ISP1_route comment=standard
add src-address=subnetB  action=lookup table=ISP1_route comment=standard
add src-address=subnetC  action=lookup table=ISP2_route comment=vip
add src-address=subnetD  action=lookup table=ISP2_route comment=vip
add src-address=subnetC  action=lookup table=ISP3_route comment=others
add src-address=subnetD  action=lookup table=ISP3_route comment=others
..
.....

This should do it. All standard users will follow the route rules and go out ISP1, all vip users will go out ISP2 and all others users will go out ISP3.
Case1: IF ISP1 goes down the router will look for the next available route and will find it in the main table and since ISP2 is lower in distance it will be chosen and if ISP2 is not available it will choose ISP3. IF ISP2 comes back on line the users will be directed back through ISP2 and if ISP1 comes back on line the users will go to their original ISP1.
Case2: IF ISP2 goes doen the router will look for the next available route and it will be ISP1, and then ISP3 etc.
Case3: IF ISP3 goes down, the router will look for the next available route it will e ISP1, and then ISP2 etc.


VERSION 7 Differences.

/ip tables
add name=ISP1-route fib
add name=ISP2-route fib
add name=ISP3-route fib

dst-address=0.0.0.0/0 gateway=10.10.10.10 scope=10 target-scope=14 table=ISP1_route distance=5
dst-address=0.0.0.0/0 gateway=10.10.10.10 scope=10 target-scope=14 table=ISP2_route distance=10
dst-address=0.0.0.0/0 gateway=10.10.10.10 scope=10 target-scope=14 table=ISP3_route distance=20
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Mon Oct 03, 2022 8:44 pm

In v6 it's in IP->Routes on Rules tab.
Thank you very much for pointing me to the right tab Sir. I surely overlooked it.

Now i just need to understand how to properly use the Routing Rules to replace my Mangle Rules.

By the way, may i know why Anav and the other experts here preferred to use the routing rules instead of mangling? What are the advantages and disadvantages between the two?
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Mon Oct 03, 2022 9:20 pm

In v6 it's in IP->Routes on Rules tab.
Thank you very much for pointing me to the right tab Sir. I surely overlooked it.

Now i just need to understand how to properly use the Routing Rules to replace my Mangle Rules.

By the way, may i know why Anav and the other experts here preferred to use the routing rules instead of mangling? What are the advantages and disadvantages between the two?
jajajaja I'm not an expert and that is the reason I personally avoid mangling, its more complex and since normally one cannot use fastrack with mangling, (sometimes one can work around mangling but often not) and thus performance will be slower (although probably a home user would never notice). I use it when I dont have another choice. Keep it simple!!
....
rules.jpg
You do not have the required permissions to view the files attached to this post.
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Mon Oct 03, 2022 9:34 pm

You can use any numbers for distance as long as they are separated.
I setup the scope and target scope numbers such that they are also legal for Vers7 when you switch.
The only difference between V6 and V7 for this particular config would be the need to create 3 tables and
on the second IP route for each ISPX, use table= vice routing-mark=
......
VERSION 6

dst-address=0.0.0.0/0 gateway=10.10.10.10 scope=10 target-scope=14 table=main  distance=5
dst-address=0.0.0.0/0 gateway=10.10.10.10 scope=10 target-scope=14 routing-mark=ISP1_route distance=5
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
add check-gateway=ping dst-address=10.10.10.10/32 gateway=9.9.9.9 scope=10 target-scope=13
add dst-address=9.9.9.9/32 gateway=PrimaryISP-gatewayIP scope=10 target-scope=12
add check-gateway=ping dst-address=10.10.10.10/32 gateway=1.0.0.1 scope=10 target-scope=13 
add dst-address=1.0.0.1/32 gateway=PrimaryISP-gatewayIP scope=10 target-scope=12

..................................................................................................................................................

dst-address=0.0.0.0/0 gateway=10.10.10.10 scope=10 target-scope=14 table=main distance=10
dst-address=0.0.0.0/0 gateway=10.10.10.10 scope=10 target-scope=14 routing-mark=ISP2_route  distance=10
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
add check-gateway=ping dst-address=10.10.10.10/32 gateway=9.9.9.9 scope=10 target-scope=13
add dst-address=9.9.9.9/32 gateway=SecondarISP-gatewayIP scope=10 target-scope=12
add check-gateway=ping dst-address=10.10.10.10/32 gateway=1.0.0.1 scope=10 target-scope=13 
add dst-address=1.0.0.1/32 gateway=SecondaryISP-gatewayIP scope=10 target-scope=12

......................................................................................................................................................

dst-address=0.0.0.0/0 gateway=10.10.10.10 scope=10 target-scope=14 table=main distance=20
dst-address=0.0.0.0/0 gateway=10.10.10.10 scope=10 target-scope=14 routing-mark=ISP3_route  distance=20
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
add check-gateway=ping dst-address=10.10.10.10/32 gateway=9.9.9.9 scope=10 target-scope=13
add dst-address=9.9.9.9/32 gateway=TertiaryISP-gatewayIP scope=10 target-scope=12
add check-gateway=ping dst-address=10.10.10.10/32 gateway=1.0.0.1 scope=10 target-scope=13 
add dst-address=1.0.0.1/32 gateway=TertiaryISP-gatewayIP scope=10 target-scope=12

/ip routing rules
add src-address=subnetA  action=lookup table=ISP1_route comment=standard
add src-address=subnetB  action=lookup table=ISP1_route comment=standard
add src-address=subnetC  action=lookup table=ISP2_route comment=vip
add src-address=subnetD  action=lookup table=ISP2_route comment=vip
add src-address=subnetC  action=lookup table=ISP3_route comment=others
add src-address=subnetD  action=lookup table=ISP3_route comment=others
..
.....

This should do it. All standard users will follow the route rules and go out ISP1, all vip users will go out ISP2 and all others users will go out ISP3.
Case1: IF ISP1 goes down the router will look for the next available route and will find it in the main table and since ISP2 is lower in distance it will be chosen and if ISP2 is not available it will choose ISP3. IF ISP2 comes back on line the users will be directed back through ISP2 and if ISP1 comes back on line the users will go to their original ISP1.
Case2: IF ISP2 goes doen the router will look for the next available route and it will be ISP1, and then ISP3 etc.
Case3: IF ISP3 goes down, the router will look for the next available route it will e ISP1, and then ISP2 etc.


VERSION 7 Differences.

/ip tables
add name=ISP1-route fib
add name=ISP2-route fib
add name=ISP3-route fib

dst-address=0.0.0.0/0 gateway=10.10.10.10 scope=10 target-scope=14 table=ISP1_route distance=5
dst-address=0.0.0.0/0 gateway=10.10.10.10 scope=10 target-scope=14 table=ISP2_route distance=10
dst-address=0.0.0.0/0 gateway=10.10.10.10 scope=10 target-scope=14 table=ISP3_route distance=20
Hi Sir, noted on this. Will try to implement this config ASAP once my client (the network owner) has already sent an advisory to their subscribers for a scheduled system maintenance. This is to ensure the end users are aware of a possible service downtime.

Just a little clarification, i have noticed you only used the same set of external dns or ip addresses (9.9.9.9 & 1.0.0.1) for all the three WAN routes as well as the bogus IP address (10.10.10.10). What if for some weird instances, those two dns ip addresses are not available or down? Is it ok if we use different set of dns addresses for each WAN route like what i did in my trial config posted earlier ( 2 dns addreses and 1 bogus address per WAN) What would be the disadvantage of using different sets of external ip addresses for the recursive route?
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Mon Oct 03, 2022 9:48 pm



Thank you very much for pointing me to the right tab Sir. I surely overlooked it.

Now i just need to understand how to properly use the Routing Rules to replace my Mangle Rules.

By the way, may i know why Anav and the other experts here preferred to use the routing rules instead of mangling? What are the advantages and disadvantages between the two?
jajajaja I'm not an expert and that is the reason I personally avoid mangling, its more complex and since normally one cannot use fastrack with mangling, (sometimes one can work around mangling but often not) and thus performance will be slower (although probably a home user would never notice). I use it when I dont have another choice. Keep it simple!!
....

rules.jpg
hahaha well, you wouldn't be a "forum guru" here if you are not an expert (at least on some functions of MT) yet. and second, you can't write a well organized mikrotik configuration guide if your knowledge on the subject is only limited. hehe

again, thank you very much for the help sir. will let you know once i already implemented this config.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Mon Oct 03, 2022 10:31 pm

Well its only a forum so there is no guarantee that the config will work and especially because a config relies upon all the parts working together.
I can state that implementing the config will not cause your bank accounts to empty into my bank accounts. I wish I was that good. :-)
 
Sob
Forum Guru
Forum Guru
Posts: 9119
Joined: Mon Apr 20, 2009 9:11 pm

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Tue Oct 04, 2022 3:37 am

By the way, may i know why Anav and the other experts here preferred to use the routing rules instead of mangling? What are the advantages and disadvantages between the two?
Routing rules work only with IP adresses. Mangling is more flexible, because you can use any condition allowed by firewall, so you can e.g. route all http(s) traffic (ports 80 and 443) to one ISP and everything else to another. You can't do that with routing rules. But routing rules are compatible with FastTrack, while mangling is not. Sometimes it's also good that they stay aside from firewall (if you have a lot of other stuff there). And they should be faster, but I don't have any numbers how much.
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Tue Oct 04, 2022 3:10 pm

noted on this sir. how about the cpu utilization, what's the difference between the two?
also, can i use the two (routing rules & mangle rules) at the same time? let's say for example i need to filter some packets via mangle rules from some users under one local subnet and mark them for routing and at the same time make a table via routing rules for some specific ip also under the same local subnet and route them separately to specific WANs?
 
Sob
Forum Guru
Forum Guru
Posts: 9119
Joined: Mon Apr 20, 2009 9:11 pm

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Wed Oct 05, 2022 3:05 am

About performance, see the last sentence in my previous post, sir. :)

Combination of routing rules and routing marks is possible. One unfortunate thing is that they recently changed their priorities (see viewtopic.php?p=956630#p956630), so if you configure something in v6, it may break when you upgrade to v7.
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Wed Oct 05, 2022 4:16 pm

About performance, see the last sentence in my previous post, sir. :)

Combination of routing rules and routing marks is possible. One unfortunate thing is that they recently changed their priorities (see viewtopic.php?p=956630#p956630), so if you configure something in v6, it may break when you upgrade to v7.
Sir, may i know where i can select the option to "table=main" under /ip route tab in Router0S 6.49.6? i can only see the "routing mark" dropdown menu, though when i click the dropdown i can see the route rules i made and there's also an option for "main" but it's not under the table selection menu but from the routing mark selection menu. are the "routing mark" & "table" options the same in ROS 6?

Also, could you please enlighten me what's the difference between the "main table" and the "routing mark"?

What will happen if i dont include this rule

/ip route
add dst-address=0.0.0.0/0 gateway=10.1.1.1 scope=10 target-scope=14 table=main distance=5

and only use this one?

add dst-address=0.0.0.0/0 gateway=10.1.1.1 scope=10 target-scope=14 routing-mark=ISP1_route distance=5
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Wed Oct 05, 2022 5:15 pm

table main is a v7 view, default table is the V6 view.
In other words in V7 we specifcy which table, in V6, we specify the routing mark.
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Wed Oct 05, 2022 6:35 pm

table main is a v7 view, default table is the V6 view.
In other words in V7 we specifcy which table, in V6, we specify the routing mark.
so if i am going to configure this settings

add dst-address=0.0.0.0/0 gateway=10.1.1.1 scope=10 target-scope=14 table=main distance=5
add dst-address=0.0.0.0/0 gateway=10.1.1.1 scope=10 target-scope=14 routing-mark=ISP1_route distance=5

then i will just select "main" and "ISP1_route" from the Routing Mark dropdown list under /ip route tab, right?



by the way, based from your given config template, i edited and changed the external host addresses (9.9.9.9 / 1.0.0.1) with other dns addresses to ensure recursive routing works on the other WAN if just in case the two external addresses assigned for the primary ISP are down. I also changed the virtual (bogus) external addresses 10.10.10.10 into 10.1.1.1, 10.2.2.2 and 10.3.3.3 on each WAN routes. would that be ok?

here is my draft edited config...


/ip route
add dst-address=0.0.0.0/0 gateway=10.1.1.1 scope=10 target-scope=14 routing-mark=main distance=5
add dst-address=0.0.0.0/0 gateway=10.1.1.1 scope=10 target-scope=14 routing-mark=ISP1_route distance=5
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
add check-gateway=ping dst-address=10.1.1.1/32 gateway=9.9.9.9 scope=10 target-scope=13
add dst-address=9.9.9.9/32 gateway=192.168.1.1 scope=10 target-scope=12
add check-gateway=ping dst-address=10.1.1.1/32 gateway=1.0.0.1 scope=10 target-scope=13
add dst-address=1.0.0.1/32 gateway=192.168.1.1 scope=10 target-scope=12

..................................................................................................................................................

add dst-address=0.0.0.0/0 gateway=10.2.2.2 scope=10 target-scope=14 routing-mark=main distance=10
add dst-address=0.0.0.0/0 gateway=10.2.2.2 scope=10 target-scope=14 routing-mark=ISP2_route distance=10
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
add check-gateway=ping dst-address=10.2.2.2/32 gateway=208.67.222.222 scope=10 target-scope=13
add dst-address=208.67.222.222/32 gateway=XXX.XXX.XXX.90 scope=10 target-scope=12
add check-gateway=ping dst-address=10.2.2.2/32 gateway=1.1.1.1 scope=10 target-scope=13
add dst-address=1.1.1.1/32 gateway=XXX.XXX.XXX.90 scope=10 target-scope=12

......................................................................................................................................................

add dst-address=0.0.0.0/0 gateway=10.3.3.3 scope=10 target-scope=14 routing-mark=main distance=20
add dst-address=0.0.0.0/0 gateway=10.3.3.3 scope=10 target-scope=14 routing-mark=ISP3_route distance=20
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
add check-gateway=ping dst-address=10.3.3.3/32 gateway=149.112.112.112 scope=10 target-scope=13
add dst-address=149.112.112.112/32 gateway=XXX.XXX.XXX.121 scope=10 target-scope=12
add check-gateway=ping dst-address=10.3.3.3/32 gateway=76.76.19.19 scope=10 target-scope=13
add dst-address=76.76.19.19/32 gateway=XXX.XXX.XXX.121 scope=10 target-scope=12

/ip routing rules
add src-address=10.100.100.0/24 action=lookup table=ISP1_route comment=STANDARD
add src-address=10.100.200.0/24 action=lookup table=ISP1_route comment=STANDARD
add src-address=10.200.123.0/24 action=lookup table=ISP2_route comment=VIP
add src-address=172.20.234.0/24 action=lookup table=ISP2_route comment=VIP
add src-address=192.168.111.0/24 action=lookup table=ISP3_route comment=OTHERS
add src-address=192.168.222.0/24 action=lookup table=ISP3_route comment=OTHERS


would appreciate if you can further check this config for any error.

thank you again.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Wed Oct 05, 2022 6:42 pm

Looks good from my perspective.
Now to see if it works :-)

probably should post complete config /export (minus serial number and any pubic WANIP info - seems its all private)
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Wed Oct 05, 2022 6:55 pm

Looks good from my perspective.
Now to see if it works :-)

probably should post complete config /export (minus serial number and any pubic WANIP info - seems its all private)
please pardon me but as much as i want to post an exported complete config, i can't do it for now because i don't have any desktop pc or laptop here to use. currently i only have an android phone i used to check and configure the MT via mobile app winbox. i already tried to copy the text from the terminal/CLI of the app but it can't.

nevertheless, i will post here the complete config once i get a chance to copy the exported text files.

thank you again sir for your patience in helping me.
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Thu Oct 06, 2022 7:28 am

Looks good from my perspective.
Now to see if it works :-)

probably should post complete config /export (minus serial number and any pubic WANIP info - seems its all private)
Hi Sir Anav,

I already implemented the config to my customer's MT. My initial observations are these:
1. disabling and enabling ethernet port where the Primary ISP line is is connected resulted to fast failover from primary to secondary ISP and back.
2. disabling the ethernet port where the Secondary ISP line is connected resulted to fast failover from secondary to primary ISP. However, when i enabled back the ethernet port of the secondary ISP, the original assigned traffic to Secondary ISP didn't restore back to Secondary ISP gateway.

i tried to temporarily disable these two rules

add dst-address=0.0.0.0/0 gateway=10.2.2.2 scope=10 target-scope=14 routing-mark=main distance=10
add dst-address=0.0.0.0/0 gateway=10.2.2.2 scope=10 target-scope=14 routing-mark=ISP2_route distance=10

and when i enable it again, finally, the ISP2_route shifted back from Primary ISP to Secondary ISP.

what might be lacking in this config?

/ip route
add dst-address=0.0.0.0/0 gateway=10.1.1.1 scope=10 target-scope=14 routing-mark=main distance=5
add dst-address=0.0.0.0/0 gateway=10.1.1.1 scope=10 target-scope=14 routing-mark=ISP1_route distance=5
add check-gateway=ping dst-address=10.1.1.1/32 gateway=9.9.9.9 scope=10 target-scope=13
add dst-address=9.9.9.9/32 gateway=192.168.1.1 scope=10 target-scope=12
add check-gateway=ping dst-address=10.1.1.1/32 gateway=1.0.0.1 scope=10 target-scope=13
add dst-address=1.0.0.1/32 gateway=192.168.1.1 scope=10 target-scope=12
add dst-address=0.0.0.0/0 gateway=10.2.2.2 scope=10 target-scope=14 routing-mark=main distance=10
add dst-address=0.0.0.0/0 gateway=10.2.2.2 scope=10 target-scope=14 routing-mark=ISP2_route distance=10
add check-gateway=ping dst-address=10.2.2.2/32 gateway=208.67.222.222 scope=10 target-scope=13
add dst-address=208.67.222.222/32 gateway=XXX.XXX.XXX.90 scope=10 target-scope=12
add check-gateway=ping dst-address=10.2.2.2/32 gateway=1.1.1.1 scope=10 target-scope=13
add dst-address=1.1.1.1/32 gateway=XXX.XXX.XXX.90 scope=10 target-scope=12
dst-address=0.0.0.0/0 gateway=10.3.3.3 scope=10 target-scope=14 routing-mark=main distance=20
dst-address=0.0.0.0/0 gateway=10.3.3.3 scope=10 target-scope=14 routing-mark=ISP3_route distance=20
add check-gateway=ping dst-address=10.3.3.3/32 gateway=149.112.112.112 scope=10 target-scope=13
add dst-address=149.112.112.112/32 gateway=XXX.XXX.XXX.121 scope=10 target-scope=12
add check-gateway=ping dst-address=10.3.3.3/32 gateway=76.76.19.19 scope=10 target-scope=13
add dst-address=76.76.19.19/32 gateway=XXX.XXX.XXX.121 scope=10 target-scope=12

/ip routing rules
add src-address=10.100.100.0/24 action=lookup table=ISP1_route comment=STANDARD
add src-address=10.100.200.0/24 action=lookup table=ISP1_route comment=STANDARD
add src-address=10.200.123.0/24 action=lookup table=ISP2_route comment=VIP
add src-address=172.20.234.0/24 action=lookup table=ISP2_route comment=VIP
add src-address=192.168.111.0/24 action=lookup table=ISP3_route comment=OTHERS
add src-address=192.168.222.0/24 action=lookup table=ISP3_route comment=OTHERS
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Thu Oct 06, 2022 1:13 pm

Please post the actual config, you have differences in your presentation which do not help......
/ip route
add dst-address=0.0.0.0/0 gateway=10.1.1.1 scope=10 target-scope=14 routing-mark=main distance=5
add dst-address=0.0.0.0/0 gateway=10.1.1.1 scope=10 target-scope=14 routing-mark=ISP1_route distance=5
add dst-address=0.0.0.0/0 gateway=10.2.2.2 scope=10 target-scope=14 routing-mark=main distance=10
add dst-address=0.0.0.0/0 gateway=10.2.2.2 scope=10 target-scope=14 routing-mark=ISP2_route distance=10
dst-address=0.0.0.0/0 gateway=10.3.3.3 scope=10 target-scope=14 routing-mark=main distance=20
dst-address=0.0.0.0/0 gateway=10.3.3.3 scope=10 target-scope=14 routing-mark=ISP3_route distance=20
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Thu Oct 06, 2022 5:00 pm

Please post the actual config, you have differences in your presentation which do not help......
/ip route
add dst-address=0.0.0.0/0 gateway=10.1.1.1 scope=10 target-scope=14 routing-mark=main distance=5
add dst-address=0.0.0.0/0 gateway=10.1.1.1 scope=10 target-scope=14 routing-mark=ISP1_route distance=5
add dst-address=0.0.0.0/0 gateway=10.2.2.2 scope=10 target-scope=14 routing-mark=main distance=10
add dst-address=0.0.0.0/0 gateway=10.2.2.2 scope=10 target-scope=14 routing-mark=ISP2_route distance=10
dst-address=0.0.0.0/0 gateway=10.3.3.3 scope=10 target-scope=14 routing-mark=main distance=20
dst-address=0.0.0.0/0 gateway=10.3.3.3 scope=10 target-scope=14 routing-mark=ISP3_route distance=20

i'm very sorry sir, but i dont have any pc right now to copy the config text file, hence, what can i provide now is just a screenshot from winbox mobile.

by the way, i updated some comments and names for my easy reference as well as the assigned external DNS addresses for the recursive routes. here's the screenshot from the /ip route export result of my latest config.

(i crossed out all the disabled entries, those are the original config before i implemented this new config)

please see attached jpg file for your reference.

also, i noticed the table=main ( replaced with "routing-mark=main" ) did not appear. i really don't know why. i deleted and re-entered those specific rules that should contain "routing-mark=main" in lieu of "table=main" but still it did not appear in the /ip route export command.
is it a bug or it's just normal for entries with "main" selected?
this might be the cause of why the config did not fully work but i don't know how to fix it.

seeking for your further help please. thank you.
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Thu Oct 06, 2022 5:15 pm

OMG, the attached jpg file did not able to upload.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Thu Oct 06, 2022 6:13 pm

Yes its perplexing.
If the WAN2 is up, then any new connections/sessions would be directed out WAN2.

Can you confirm that when you plug ethernet back in for WAN2, that the IP routes for WAN2 become available (goes from blue to black), never mind testing users.
If it does not, there is an issue with acquiring perhaps a new wan gateway IP???
Still removing the routes and reapplying the routes shouldnt fix the issue if that is what it is??
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Thu Oct 06, 2022 7:14 pm

Yes its perplexing.
If the WAN2 is up, then any new connections/sessions would be directed out WAN2.

Can you confirm that when you plug ethernet back in for WAN2, that the IP routes for WAN2 become available (goes from blue to black), never mind testing users.
If it does not, there is an issue with acquiring perhaps a new wan gateway IP???
Still removing the routes and reapplying the routes shouldnt fix the issue if that is what it is??
Hi Sir, yes i tried it already. when i plug back ethernet for WAN2 the two recursive routes (with real dns addresses)
assigned to WAN2 routes (see routes below)

add check-gateway=ping comment="Monitoring ISP2-B" distance=1 dst-address=\
10.2.2.2/32 gateway=94.140.14.14 scope=10 target-scope=13
add check-gateway=ping comment="Monitoring ISP2-A" distance=1 dst-address=\
10.2.2.2/32 gateway=208.67.222.222 scope=10 target-scope=13

were able to recursively resolved back to the gateway ip of WAN2 but the recursive routes with fake/bogus ip (see routes below)

add comment="Virtual Route ISP2 - (to-ISP2_primary_route)" distance=10 \
gateway=10.2.2.2 routing-mark=to-ISP2_route scope=10 target-scope=14
add comment="Virtual Route ISP2 - (main table)" distance=10 gateway=10.2.2.2 \
scope=10 target-scope=14

still stick to the gateway ip of WAN1 until i disable and enable back these two routes.



anyway, finally i was able to copy the actual export config. WAN2 & WAN3 are public static IPs (hence, i replaced it with XXX.XXX.XXX for security purposes) while WAN1 is dynamic with ISPs default modem/router gateway of 192.168.1.1


# oct/06/2022 23:16:11 by RouterOS 6.49.6
#
#
# model = CCR1009-7G-1C-1S+

/ip address
add address=XXX.XXX.XXX.122/29 interface=ether1-ISP3 network=XXX.XXX.XXX.120
add address=XXX.XXX.XXX.90/29 interface=ether2-ISP2 network=XXX.XXX.XXX.88
add address=192.168.1.99/24 interface=ether3-ISP1
network=192.168.1.0
/ip dhcp-client
add add-default-route=no disabled=no interface=ether3-ISP1
/ip route
add comment="Virtual Route ISP1 - (to-ISP1_primary_route)" distance=5 \
gateway=10.1.1.1 routing-mark=to-ISP1_route scope=10 target-scope=14
add comment="Virtual Route ISP2 - (to-ISP2_primary_route)" distance=10 \
gateway=10.2.2.2 routing-mark=to-ISP2_route scope=10 target-scope=14
add comment="Virtual Route ISP3 - (to-ISP3_route)" distance=20 gateway=\
10.3.3.3 routing-mark=to-ISP3_route scope=10 target-scope=14
add comment="Virtual Route ISP1 - (main table)" distance=5 gateway=10.1.1.1 \
scope=10 target-scope=14
add comment="Virtual Route ISP2 - (main table)" distance=10 gateway=10.2.2.2 \
scope=10 target-scope=14
add comment="Virtual Route ISP3 - (main table)" distance=20 gateway=10.3.3.3 \
scope=10 target-scope=14
add comment="Tertiary Route _ ISP3-A" distance=1 dst-address=1.1.1.1/32 \
gateway=XXX.XXX.XXX.121 scope=10 target-scope=12
add comment="Primary Route _ ISP1-A" distance=1 dst-address=9.9.9.9/32 \
gateway=192.168.1.1 scope=10 target-scope=12
add check-gateway=ping comment="Monitoring ISP1-A" distance=1 dst-address=\
10.1.1.1/32 gateway=9.9.9.9 scope=10 target-scope=13
add check-gateway=ping comment="Monitoring ISP1-B" distance=1 dst-address=\
10.1.1.1/32 gateway=76.76.19.19 scope=10 target-scope=13
add check-gateway=ping comment="Monitoring ISP2-B" distance=1 dst-address=\
10.2.2.2/32 gateway=94.140.14.14 scope=10 target-scope=13
add check-gateway=ping comment="Monitoring ISP2-A" distance=1 dst-address=\
10.2.2.2/32 gateway=208.67.222.222 scope=10 target-scope=13
add check-gateway=ping comment="Monitoring ISP3-A" distance=1 dst-address=\
10.3.3.3/32 gateway=1.1.1.1 scope=10 target-scope=13
add check-gateway=ping comment="Monitoring ISP3-B" distance=1 dst-address=\
10.3.3.3/32 gateway=76.76.2.0 scope=10 target-scope=13
add comment="Tertiary Route _ ISP3-B" distance=1 dst-address=76.76.2.0/32 \
gateway=XXX.XXX.XXX.121 scope=10 target-scope=12
add comment="Primary Route _ ISP1-B" distance=1 dst-address=76.76.19.19/32 \
gateway=192.168.1.1 scope=10 target-scope=12
add comment="Secondary Route _ ISP2-B" distance=1 dst-address=94.140.14.14/32 \
gateway=XXX.XXX.XXX.89 scope=10 target-scope=12
add comment="Secondary Route _ ISP2-A" distance=1 dst-address=\
208.67.222.222/32 gateway=XXX.XXX.XXX.89 scope=10 target-scope=12
/ip route rule
add comment="Standard Route - No ST PCQ" src-address=10.100.100.0/24 table=\
to-ISP1_route
add comment="Standard Route - with ST PCQ" src-address=10.100.200.0/24 table=\
to-ISP1_route
add comment="VIP Route - No ST PCQ" src-address=10.200.123.0/24 table=\
to-ISP2_route
add comment="VIP Route - with ST PCQ" src-address=172.20.234.0/24 table=\
to-ISP2_route
add comment="Other Routes - No ST PCQ" src-address=192.168.111.0/24 table=\
to-ISP3_route
add comment="Other Routes - with ST PCQ" src-address=192.168.222.0/24 table=\
to-ISP3_route
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Thu Oct 06, 2022 7:27 pm

The question was not about what the users were experiencing.
I wanted to know what IP routes showed in terms of being active or inactive.

When you pulled the plug on the WAN2 etherport, all its associated routes should have turned from black to blue, and when you plugged ether2 back on they should have turned
from blue to black.

In addition in IP DHCP client settings, you would see the status of the connection at any given point in time and can monitor what happens before you pull the plug, after you pull the plug, and when you put the plug back in.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Thu Oct 06, 2022 8:14 pm

On a quick look at your routes......... one can do a quick cross check. 3 wans with two recursives per WAN means for IP routes(2x3) =6 and for Recursives(4x3) =12, there should be 18 routes in total. You have 18 so that passes a quick check.

From:
/
ip route
add comment="Virtual Route ISP1 - (to-ISP1_primary_route)" distance=5 \
gateway=10.1.1.1 routing-mark=to-ISP1_route scope=10 target-scope=14
add comment="Virtual Route ISP2 - (to-ISP2_primary_route)" distance=10 \
gateway=10.2.2.2 routing-mark=to-ISP2_route scope=10 target-scope=14
add comment="Virtual Route ISP3 - (to-ISP3_route)" distance=20 gateway=\
10.3.3.3 routing-mark=to-ISP3_route scope=10 target-scope=14
add comment="Virtual Route ISP1 - (main table)" distance=5 gateway=10.1.1.1 \
scope=10 target-scope=14
add comment="Virtual Route ISP2 - (main table)" distance=10 gateway=10.2.2.2 \
scope=10 target-scope=14
add comment="Virtual Route ISP3 - (main table)" distance=20 gateway=10.3.3.3 \
scope=10 target-scope=14
add comment="Tertiary Route _ ISP3-A" distance=1 dst-address=1.1.1.1/32 \
gateway=XXX.XXX.XXX.121 scope=10 target-scope=12
add comment="Primary Route _ ISP1-A" distance=1 dst-address=9.9.9.9/32 \
gateway=192.168.1.1 scope=10 target-scope=12
add check-gateway=ping comment="Monitoring ISP1-A" distance=1 dst-address=\
10.1.1.1/32 gateway=9.9.9.9 scope=10 target-scope=13
add check-gateway=ping comment="Monitoring ISP1-B" distance=1 dst-address=\
10.1.1.1/32 gateway=76.76.19.19 scope=10 target-scope=13
add check-gateway=ping comment="Monitoring ISP2-B" distance=1 dst-address=\
10.2.2.2/32 gateway=94.140.14.14 scope=10 target-scope=13
add check-gateway=ping comment="Monitoring ISP2-A" distance=1 dst-address=\
10.2.2.2/32 gateway=208.67.222.222 scope=10 target-scope=13
add check-gateway=ping comment="Monitoring ISP3-A" distance=1 dst-address=\
10.3.3.3/32 gateway=1.1.1.1 scope=10 target-scope=13
add check-gateway=ping comment="Monitoring ISP3-B" distance=1 dst-address=\
10.3.3.3/32 gateway=76.76.2.0 scope=10 target-scope=13
add comment="Tertiary Route _ ISP3-B" distance=1 dst-address=76.76.2.0/32 \
gateway=XXX.XXX.XXX.121 scope=10 target-scope=12
add comment="Primary Route _ ISP1-B" distance=1 dst-address=76.76.19.19/32 \
gateway=192.168.1.1 scope=10 target-scope=12
add comment="Secondary Route _ ISP2-B" distance=1 dst-address=94.140.14.14/32 \
gateway=XXX.XXX.XXX.89 scope=10 target-scope=12
add comment="Secondary Route _ ISP2-A" distance=1 dst-address=\
208.67.222.222/32 gateway=XXX.XXX.XXX.89 scope=10 target-scope=12
.....

TO: (and organized so clear, you have the order of rules so confusing hard to read.)
add comment="Virtual Route ISP1 - (main table)" distance=5 gateway=10.1.1.1 \
scope=10 target-scope=14
add comment="Virtual Route ISP1 - (to-ISP1_primary_route)" distance=5 \
gateway=10.1.1.1 routing-mark=to-ISP1_route scope=10 target-scope=14
add check-gateway=ping comment="Monitoring ISP1-A" distance=1 dst-address=\
10.1.1.1/32 gateway=9.9.9.9 scope=10 target-scope=13
add comment="Primary Route _ ISP1-A" distance=1 dst-address=9.9.9.9/32 \
gateway=192.168.1.1 scope=10 target-scope=12
add check-gateway=ping comment="Monitoring ISP1-B" distance=1 dst-address=\
10.1.1.1/32 gateway=76.76.19.19 scope=10 target-scope=13
add comment="Primary Route _ ISP1-B" distance=1 dst-address=76.76.19.19/32 \
gateway=192.168.1.1 scope=10 target-scope=12

add comment="Virtual Route ISP2 - (main table)" distance=10 gateway=10.2.2.2 \
scope=10 target-scope=14
add comment="Virtual Route ISP2 - (to-ISP2_primary_route)" distance=10 \
gateway=10.2.2.2 routing-mark=to-ISP2_route scope=10 target-scope=14
add check-gateway=ping comment="Monitoring ISP2-A" distance=1 dst-address=\
10.2.2.2/32 gateway=208.67.222.222 scope=10 target-scope=13
add comment="Secondary Route _ ISP2-A" distance=1 dst-address=\
208.67.222.222/32 gateway=XXX.XXX.XXX.89 scope=10 target-scope=12
add check-gateway=ping comment="Monitoring ISP2-B" distance=1 dst-address=\
10.2.2.2/32 gateway=94.140.14.14 scope=10 target-scope=13
add comment="Secondary Route _ ISP2-B" distance=1 dst-address=94.140.14.14/32 \
gateway=XXX.XXX.XXX.89 scope=10 target-scope=12

add comment="Virtual Route ISP3 - (main table)" distance=20 gateway=10.3.3.3 \
scope=10 target-scope=14
add comment="Virtual Route ISP3 - (to-ISP3_route)" distance=20 gateway=\
10.3.3.3 routing-mark=to-ISP3_route scope=10 target-scope=14
add check-gateway=ping comment="Monitoring ISP3-A" distance=1 dst-address=\
10.3.3.3/32 gateway=1.1.1.1 scope=10 target-scope=13
add comment="Tertiary Route _ ISP3-A" distance=1 dst-address=1.1.1.1/32 \
gateway=XXX.XXX.XXX.121 scope=10 target-scope=12
add check-gateway=ping comment="Monitoring ISP3-B" distance=1 dst-address=\
10.3.3.3/32 gateway=76.76.2.0 scope=10 target-scope=13
add comment="Tertiary Route _ ISP3-B" distance=1 dst-address=76.76.2.0/32 \
gateway=XXX.XXX.XXX.121 scope=10 target-scope=12
...

On a positive note it all checks out.
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Thu Oct 06, 2022 9:21 pm

The question was not about what the users were experiencing.
I wanted to know what IP routes showed in terms of being active or inactive.

When you pulled the plug on the WAN2 etherport, all its associated routes should have turned from black to blue, and when you plugged ether2 back on they should have turned
from blue to black.

In addition in IP DHCP client settings, you would see the status of the connection at any given point in time and can monitor what happens before you pull the plug, after you pull the plug, and when you put the plug back in.

ok sir, for easier reference let's call each route based on the comments/names i wrote in my config to avoid confusion.

although i can't see the color differences of the routes when active or not in mobile winbox but i think i can still determine what route is active and what not by looking into its "route flags".

NOTE: for now WAN3 has no internet connection due to the line is not available (has been temporarily disconneted by the ISP due to upgrade of facilities). Hence, WAN3 has failover to WAN1 for its internet access.


Here are the routes status:

SCENARIO #1: all WANs are connected into the ethernet port of MT except WAN3.

-Virtual Route ISP1 - (main table) = active,static
-Virtual Route ISP1 - (to-ISP1_primary_route) = active,static
-Monitoring ISP1-A = active,static
-Monitoring ISP1-B = static
-Primary Route _ ISP1-A = active,static
-Primary Route _ ISP1-B = active,static
All routes resolved to WAN1 gateway IP

-Virtual Route ISP2 - (main table) = static
-Virtual Route ISP2 - (to-ISP2_primary_route) = active,static
--Monitoring ISP2-A = static
-Monitoring ISP2-B = active,static
-Secondary Route _ ISP2-A = active,static
-Secondary Route _ ISP2-B = active,static
All routes resolved to WAN2 gateway IP

-Virtual Route ISP3 - (main table) = static
-Virtual Route ISP3 - (to-ISP3_primary_route) = active,static
-Monitoring ISP3-A = active,static
-Monitoring ISP3-B = static
-Tertiary Route _ ISP3-A = active,static
-Tertiary Route _ ISP3-B = active,static
All routes resolved to WAN1 gateway IP because WAN3 ethernet port has no cable plugged in (physical internet line from ISP is under repair)

========================================
SCENARIO #2: WAN1 ethernet cable pulled out, WAN3 still no ethernet cable plugged in (line under repair).

-Virtual Route ISP1 - (main table) = active,static
-Virtual Route ISP1 - (to-ISP1_primary_route) = active,static
-Monitoring ISP1-A = active,static
-Monitoring ISP1-B = static
-Primary Route _ ISP1-A = active,static
-Primary Route _ ISP1-B = active,static
All routes resolved to WAN2 gateway IP

-Virtual Route ISP2 - (main table) = static
-Virtual Route ISP2 - (to-ISP2_primary_route) = active,static
-Monitoring ISP2-A = static
-Monitoring ISP2-B = active,static
-Secondary Route _ ISP2-A = active,static
-Secondary Route _ ISP2-B = active,static
All routes resolved to WAN2 gateway IP

-Virtual Route ISP3 - (main table) = static
-Virtual Route ISP3 - (to-ISP3_primary_route) = active,static
-Monitoring ISP3-A = active,static
-Monitoring ISP3-B = static
-Tertiary Route _ ISP3-A = active,static
-Tertiary Route _ ISP3-B = active,static
All routes resolved to WAN2 gateway IP because WAN3 ethernet port has no cable plugged in (physical internet line from ISP is under repair)

===========================================
SCENARIO #3: WAN1 cable plugged back into the ethernet port of MT.
WAN3 still no ethernet cable plugged in (under repair).

-Virtual Route ISP1 - (main table) = active,static
-Virtual Route ISP1 - (to-ISP1_primary_route) = active,static
-Monitoring ISP1-A = active,static
-Monitoring ISP1-B = static
-Primary Route _ ISP1-A = active,static
-Primary Route _ ISP1-B = active,static
All routes resolved back to WAN1 gateway IP after WAN1 cable has been plugged back into the ethernet port of MT.

-Virtual Route ISP2 - (main table) = static
-Virtual Route ISP2 - (to-ISP2_primary_route) = active,static
--Monitoring ISP2-A = static
-Monitoring ISP2-B = active,static
-Secondary Route _ ISP2-A = active,static
-Secondary Route _ ISP2-B = active,static
All routes resolved to WAN2 gateway IP

-Virtual Route ISP3 - (main table) = static
-Virtual Route ISP3 - (to-ISP3_primary_route) = active,static
-Monitoring ISP3-A = active,static
-Monitoring ISP3-B = static
-Tertiary Route _ ISP3-A = active,static
-Tertiary Route _ ISP3-B = active,static
All routes still resolved to WAN2 gateway IP even if WAN1 is already available.
WAN3 ethernet port has no cable plugged in (physical internet line from ISP is under repair)

===================================
SCENARIO #4: WAN2 cable pulled out from MT ethernet port. WAN3 has still no ethernet cable plugged in.

-Virtual Route ISP1 - (main table) = active,static
-Virtual Route ISP1 - (to-ISP1_primary_route) = active,static
-Monitoring ISP1-A = active,static
-Monitoring ISP1-B = static
-Primary Route _ ISP1-A = active,static
-Primary Route _ ISP1-B = active,static
All routes resolved to WAN1 gateway IP

-Virtual Route ISP2 - (main table) = static
-Virtual Route ISP2 - (to-ISP2_primary_route) = active,static
--Monitoring ISP2-A = static
-Monitoring ISP2-B = active,static
-Secondary Route _ ISP2-A = active,static
-Secondary Route _ ISP2-B = active,static
All routes resolved to WAN1 gateway IP after pulling out WAN2 ethernet cable.

-Virtual Route ISP3 - (main table) = static
-Virtual Route ISP3 - (to-ISP3_primary_route) = active,static
-Monitoring ISP3-A = active,static
-Monitoring ISP3-B = static
-Tertiary Route _ ISP3-A = active,static
-Tertiary Route _ ISP3-B = active,static
All routes now resolved to WAN1 gateway IP from WAN2 gateway IP after pulling out WAN2 ethernet cable.
WAN3 ethernet port still has no cable plugged in (physical internet line from ISP is under repair)

=============================
SCENARIO #5: WAN2 cable plugged back into the ethernet port of MT.
WAN3 still has no ethernet cable plugged in.

-Virtual Route ISP1 - (main table) = active,static
-Virtual Route ISP1 - (to-ISP1_primary_route) = active,static
-Monitoring ISP1-A = active,static
-Monitoring ISP1-B = static
-Primary Route _ ISP1-A = active,static
-Primary Route _ ISP1-B = active,static
All routes resolved to WAN1 gateway IP

-Virtual Route ISP2 - (main table) = static
-Virtual Route ISP2 - (to-ISP2_primary_route) = active,static
-Monitoring ISP2-A = static
-Monitoring ISP2-B = active,static
-Secondary Route _ ISP2-A = active,static
-Secondary Route _ ISP2-B = active,static
ERROR: Not all routes resolved back to WAN2 gateway IP after plugging back in WAN2 ethernet cable into MT.
Monitoring ISP2-A & B and Secondary Route _ ISP2-A & B routes all resolved back to WAN2 gateway IP after plugging in the cable BUT the Virtual Route ISP - (main table) & (to-ISP2_primary_route) routes still remained/resolved to WAN1 gateway IP causing all dedicated routes going out to WAN2 internet still routing to WAN1.


-Virtual Route ISP3 - (main table) = static
-Virtual Route ISP3 - (to-ISP3_primary_route) = active,static
-Monitoring ISP3-A = active,static
-Monitoring ISP3-B = static
-Tertiary Route _ ISP3-A = active,static
-Tertiary Route _ ISP3-B = active,static
All routes still resolved to WAN1 gateway IP after plugging back in WAN2 ethernet cable. WAN3 ethernet port has still no cable plugged in (physical internet line from ISP is under repair)


NOTE: The routes under WAN2 only returns back to its original gateway after disabling and enabling the two "Virtual Route ISP2 - (main table) and (to-ISP2_primary route)" routes.
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Thu Oct 06, 2022 9:36 pm

Based on the various scenario testing conducted above, i initially observed at least two errors.
1. WAN2 routes failed to return back to its original route after WAN2 physical line has been restored.
2. WAN3 failed to follow priority failover route. When WAN1 was down it resolved to WAN2 gateway but when WAN1 was restored, WAN3 traffic still remain at WAN2 gateway. The expectation should be WAN3 must choose WAN1 over WAN2 if both WAN1 & WAN2 are available. WAN3 must only choose WAN2 as failover route if WAN1 is down.

Since WAN3 physical connection is not available, i was not able to test if same failover behavior will happen to WAN3 like what happened to WAN2.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Thu Oct 06, 2022 11:03 pm

Your assumptions are probably correct wrt WAN3.

In terms of colours, active routes are black, INACTIVE routes are light blue.
If something is not reachable it states it clearly in the text line under column gateway.
See example from winbox doc.
kullers.JPG
You do not have the required permissions to view the files attached to this post.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Thu Oct 06, 2022 11:06 pm

Two possibilities.
a. your interpreting information as to what is up or down or available incorrectly or MORE LIKELY
b. there is something wrong on the config format, nothing you have done as there are no errors I can see, so it must be the design............
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Fri Oct 07, 2022 6:56 am

Two possibilities.
a. your interpreting information as to what is up or down or available incorrectly or MORE LIKELY
b. there is something wrong on the config format, nothing you have done as there are no errors I can see, so it must be the design............
With regards to interpreting what is available, i was able to confirm it by creating three PPPoE accounts with each under different subnet based on the three groups of subnets i made in the /ip route rules tab.
I was able to confirm that the routes always follow what the gateway ip the route "Virtual Route ISP*******" is currently resolving to.
Best example is when WAN2 ethernet cable was plugged in back to MT port, the "Secondary Route _ ISP2*****" and "Monitoring ISP2*****" routes resolved back to WAN2 gateway ip
but the "Virtual Route ISP2******" routes still remained resolving to WAN1 gateway ip.
And when i conducted speed test and traceroute coming from behind the ONU with PPPoE account IP add assigned to "to-ISP2_route" group, the traffic indeed passes to the gateway ip of WAN1 instead of WAN2.
Hence, i can conclude that whenever a specific WAN is down and gets back, its assigned traffic won't be automatically routed back to is original gateway.

Also, i was able to observed that in the case of WAN3 route where it resolved to WAN2 gateway when WAN1 was down, it did not resolve back to WAN1 gateway when WAN1 went available again. WAN3 route only resolved back to WAN1 gateway when WAN2 was down which should not be the case because WAN1 has the lower distance than WAN2, hence, WAN1 should be the first priority among the three different routes.

Maybe there is/are still lacking route/s in my config?

What do you think Sir i need to add/revise?
Could you please enlighten me what are the purpose of those "Virtual Route ISP* - (main table)" routes in my config for? What will happen to the routes if i remove them and only retain the "Virtual Route ISP* - (to-ISP********)" routes together with the "Monitoring ISP*****" and "Primary/Secondary/Tertiary Route _ ISP******" routes? Would the failover function still work?
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Fri Oct 07, 2022 1:00 pm

What you have looks good to me so there must be a flaw in the design.................. ( back to the drawing board )
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Fri Oct 07, 2022 4:39 pm

What you have looks good to me so there must be a flaw in the design.................. ( back to the drawing board )


I temporarily disabled all WAN3 routes to isolate the cause of error but unfortunately it yielded the same results.
When I unplugged and plugged in back the ethernet cable of WAN1, the traffic smoothly switch from WAN1 to WAN2 and back to WAN1.
When I unplugged the ethernet cable of WAN2 the traffic of WAN2 smoothly switches to WAN1 but when I plugged it back the traffic did not switch back from WAN1 to WAN2, although the two monitoring routes already resolved back to WAN2 gateway IP but the Virtual Routes for WAN2 still remained at WAN1 gateway IP.

Maybe there must be a need to fix on the second recursive routes (Virtual Routes)?
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Fri Oct 07, 2022 7:42 pm

i tried to copy Chupaka's workaround by putting a type=blackhole routes to the recursive dns addresses and somehow it has solved the error in WAN2 unable to resolve back to its gateway. Unfortunately, the problem shifted to WAN3.

Maybe i was not able to properly configure the distances and scopes of the blackhole routes? Honestly, i don't understand what is the blackhole for? How does it affects the route?
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Tue Oct 11, 2022 10:38 am

What you have looks good to me so there must be a flaw in the design.................. ( back to the drawing board )
Hi Sir Anav!

Finally, i was able to make the recursive failover config works by modifying and adding failover routes for each routing rules with priority distance. Here's my current working config.

/ip route
add comment="Virtual Route ISP1 - (to-ISP1_primary_route)" distance=1 \
gateway=10.1.1.1 routing-mark=to-ISP1_route scope=10 target-scope=14
add comment="Virtual Route ISP1 - (failover-to-ISP2)" distance=2 gateway=\
10.2.2.2 routing-mark=to-ISP1_route scope=10 target-scope=14
add comment="Virtual Route ISP1 - (failover-to-ISP3)" distance=3 gateway=\
10.3.3.3 routing-mark=to-ISP3_route scope=10 target-scope=14
add comment="Virtual Route ISP2 - (to-ISP2_primary_route)" distance=1 \
gateway=10.2.2.2 routing-mark=to-ISP2_route scope=10 target-scope=14
add comment="Virtual Route ISP2 - (failover-to-ISP1)" distance=2 gateway=\
10.1.1.1 routing-mark=to-ISP2_route scope=10 target-scope=14
add comment="Virtual Route ISP2 - (failover-to-ISP3)" distance=3 gateway=\
10.3.3.3 routing-mark=to-ISP2_route scope=10 target-scope=14
add comment="Virtual Route ISP3 - (to-ISP3_route)" distance=1 gateway=\
10.3.3.3 routing-mark=to-ISP3_route scope=10 target-scope=14
add comment="Virtual Route ISP3 - (failover-to-ISP1)" distance=2 gateway=\
10.1.1.1 routing-mark=to-ISP3_route scope=10 target-scope=14
add comment="Virtual Route ISP3 - (failover-to-ISP2)" distance=3 gateway=\
10.2.2.2 routing-mark=to-ISP3_route scope=10 target-scope=14
add comment="Virtual Route ISP1 - (main table)" distance=1 gateway=10.1.1.1 \
scope=10 target-scope=14
add comment="Virtual Route ISP2 - (main table)" distance=2 gateway=10.2.2.2 \
scope=10 target-scope=14
add comment="Virtual Route ISP3 - (main table)" distance=3 gateway=10.3.3.3 \
scope=10 target-scope=14
add check-gateway=ping comment="Monitoring ISP1-A" distance=1 dst-address=\
10.1.1.1/32 gateway=9.9.9.9 scope=10 target-scope=13
add check-gateway=ping comment="Monitoring ISP1-B" distance=1 dst-address=\
10.1.1.1/32 gateway=76.76.19.19 scope=10 target-scope=13
add check-gateway=ping comment="Monitoring ISP2-A" distance=1 dst-address=\
10.2.2.2/32 gateway=208.67.222.222 scope=10 target-scope=13
add check-gateway=ping comment="Monitoring ISP2-B" distance=1 dst-address=\
10.2.2.2/32 gateway=94.140.14.14 scope=10 target-scope=13
add check-gateway=ping comment="Monitoring ISP3-A" distance=1 dst-address=\
10.3.3.3/32 gateway=1.1.1.1 scope=10 target-scope=13
add check-gateway=ping comment="Monitoring ISP3-B" distance=1 dst-address=\
10.3.3.3/32 gateway=76.76.2.0 scope=10 target-scope=13
add comment="Primary Route _ ISP1-A" distance=1 dst-address=9.9.9.9/32 \
gateway=192.168.1.1 scope=10 target-scope=12
add comment="Primary Route _ ISP1-B" distance=1 dst-address=76.76.19.19/32 \
gateway=192.168.1.1 scope=10 target-scope=12
add comment="Secondary Route _ ISP2-A" distance=1 dst-address=\
208.67.222.222/32 gateway=XXX.XXX.181.89 scope=10 target-scope=12
add comment="Secondary Route _ ISP2-B" distance=1 dst-address=94.140.14.14/32 \
gateway=XXX.XXX.181.89 scope=10 target-scope=12
add comment="Tertiary Route _ ISP3-A" distance=1 dst-address=1.1.1.1/32 \
gateway=XXX.XXX.178.121 scope=10 target-scope=12
add comment="Tertiary Route _ ISP3-B" distance=1 dst-address=76.76.2.0/32 \
gateway=XXX.XXX.178.121 scope=10 target-scope=12
/ip route rule
add comment="Standard Route - No ST PCQ" src-address=10.100.100.0/24 table=\
to-ISP1_route
add comment="Standard Route - with ST PCQ" src-address=10.100.200.0/24 table=\
to-ISP1_route
add comment="VIP Route - No ST PCQ" src-address=10.200.123.0/24 table=\
to-ISP2_route
add comment="VIP Route - with ST PCQ" src-address=172.20.234.0/24 table=\
to-ISP2_route
add comment="Other Routes - No ST PCQ" src-address=192.168.111.0/24 table=\
to-ISP3_route
add comment="Other Routes - with ST PCQ" src-address=192.168.222.0/24 table=\
to-ISP3_route


By the way, I have a new problem I can no longer ping and access the customers PPPoE modem/router for maintenance purposes. Before I applied this recursive routing with specific WAN routing rules for each local subnet, I was able to access any connected customer's PPPoE modem/router as long as I am connected to one of the local subnets but now i can't.
I understand with this new routing rules, each local subnet traffic will be forcely directed to WAN gateway, hence, inter-subnet communication is no longer possible.
Can anyone help me how to be able to access any connected devices within the local subnets?
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Wed Oct 12, 2022 1:08 am

One step at a time, I have to figure what you did first, and why it works and then my guess is that we will need additional routing rules to allow such traffic.
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Wed Oct 12, 2022 5:54 am

One step at a time, I have to figure what you did first, and why it works and then my guess is that we will need additional routing rules to allow such traffic.

what i did is i just add these two failover routes and replicate them to the rest of the WANs.

add comment="Virtual Route ISP1 - (failover-to-ISP2)" distance=2 gateway=\
10.2.2.2 routing-mark=to-ISP1_route scope=10 target-scope=14
add comment="Virtual Route ISP1 - (failover-to-ISP3)" distance=3 gateway=\
10.3.3.3 routing-mark=to-ISP3_route scope=10 target-scope=14


i also changed the distance from the initial config 5,10,20 to simply 1,2,3. but i think distance numbers doesn't matter right as long as you set the priority route with a lower number than the less priority route?
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Wed Oct 12, 2022 3:32 pm

Finally, i was able to figure out what has blocked the LAN access to customer's PPPoE modem/router. When I disable any of the rules below, i can now access any modem/router that belongs to the local subnet with its routing rules being disabled.


/ip route rule
add comment="Standard Route - No ST PCQ" src-address=10.100.100.0/24 table=\
to-ISP1_route
add comment="Standard Route - with ST PCQ" src-address=10.100.200.0/24 table=\
to-ISP1_route
add comment="VIP Route - No ST PCQ" src-address=10.200.123.0/24 table=\
to-ISP2_route
add comment="VIP Route - with ST PCQ" src-address=172.20.234.0/24 table=\
to-ISP2_route
add comment="Other Routes - No ST PCQ" src-address=192.168.111.0/24 table=\
to-ISP3_route
add comment="Other Routes - with ST PCQ" src-address=192.168.222.0/24 table=\
to-ISP3_route



I tried to add some additional routes to bypass those routing rules and allow some LAN-to-LAN access but to no avail. Maybe i was not able to configure the correct route or rule.

Would appreciate if someone could help me build additional routes/config to allow me to access any of the devices within the local subnets without disabling the routing rules listed above. Will prefer if I can also specify the MAC address of the device (ex: 1A:BC:E2:3F:BC:09) that will be given access to LAN.

Thank you in advance.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Wed Oct 12, 2022 7:36 pm

Can you be clearer?
Do you mean you specifically?
What are the requirements..................

Is it a case of .......
user with IP X, on subnet B needs access to ??
user with IP Y, on subnet C needs access to ??

Subnet B needs access to Subnet Y??????
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Thu Oct 13, 2022 6:15 am

Can you be clearer?
Do you mean you specifically?
What are the requirements..................

Is it a case of .......
user with IP X, on subnet B needs access to ??
user with IP Y, on subnet C needs access to ??

Subnet B needs access to Subnet Y??????
Hi Sir Anav, what i meant is that any LAN IP under any LAN subnet can assess to any LAN IP under any LAN subnet. In short, it's an open communication within the entire LAN but access is only granted to a specific device (MAC address).

For now the actual scenario should be like this:
- Admin's smartphone when connected to any of the customer's PPPoE ONT/ONU wifi can ping & access the GUI of other customers' PPPoE ONT/ONU regardless of whether they are under the same local subnet or not.
- same scenario applies to Admin's Laptop when connected to the LAN or any of the customers' ONT/ONU ethernet ports.
- same scenario applies when Admin's device is connected to the wifi hotspot.

The above scenarios can be successfully done when those /route rules were not yet implemented but after implementation i can no longer access any of the devices connected to the LAN even with devices under the same subnet where my smartphone or PC is connected. When i temporarily disabled those routing rules (under /route rules tab) i was able to access all LAN devices again.

The MAC address whitelisting is for security reasons so that any unauthorized device won't be able to access other connected devices within the LAN.

Looking forward to your additional config suggestions. Thank you.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Thu Oct 13, 2022 6:19 pm

Can you please post your latest........ cheers
Alex
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Thu Oct 13, 2022 6:44 pm

Can you please post your latest........ cheers
Alex
as requested sir, here is my current config with recursive failover routes and routing rules implemented.


/ip route
add comment="Virtual Route ISP1 - (to-ISP1_primary_route)" distance=1 \
gateway=10.1.1.1 routing-mark=to-ISP1_route scope=10 target-scope=14
add comment="Virtual Route ISP1 - (failover-to-ISP2)" distance=2 gateway=\
10.2.2.2 routing-mark=to-ISP1_route scope=10 target-scope=14
add comment="Virtual Route ISP1 - (failover-to-ISP3)" distance=3 gateway=\
10.3.3.3 routing-mark=to-ISP3_route scope=10 target-scope=14
add comment="Virtual Route ISP2 - (to-ISP2_primary_route)" distance=1 \
gateway=10.2.2.2 routing-mark=to-ISP2_route scope=10 target-scope=14
add comment="Virtual Route ISP2 - (failover-to-ISP1)" distance=2 gateway=\
10.1.1.1 routing-mark=to-ISP2_route scope=10 target-scope=14
add comment="Virtual Route ISP2 - (failover-to-ISP3)" distance=3 gateway=\
10.3.3.3 routing-mark=to-ISP2_route scope=10 target-scope=14
add comment="Virtual Route ISP3 - (to-ISP3_route)" distance=1 gateway=\
10.3.3.3 routing-mark=to-ISP3_route scope=10 target-scope=14
add comment="Virtual Route ISP3 - (failover-to-ISP1)" distance=2 gateway=\
10.1.1.1 routing-mark=to-ISP3_route scope=10 target-scope=14
add comment="Virtual Route ISP3 - (failover-to-ISP2)" distance=3 gateway=\
10.2.2.2 routing-mark=to-ISP3_route scope=10 target-scope=14
add comment="Virtual Route ISP1 - (main table)" distance=1 gateway=10.1.1.1 \
scope=10 target-scope=14
add comment="Virtual Route ISP2 - (main table)" distance=2 gateway=10.2.2.2 \
scope=10 target-scope=14
add comment="Virtual Route ISP3 - (main table)" distance=3 gateway=10.3.3.3 \
scope=10 target-scope=14
add check-gateway=ping comment="Monitoring ISP1-A" distance=1 dst-address=\
10.1.1.1/32 gateway=9.9.9.9 scope=10 target-scope=13
add check-gateway=ping comment="Monitoring ISP1-B" distance=1 dst-address=\
10.1.1.1/32 gateway=76.76.19.19 scope=10 target-scope=13
add check-gateway=ping comment="Monitoring ISP2-A" distance=1 dst-address=\
10.2.2.2/32 gateway=208.67.222.222 scope=10 target-scope=13
add check-gateway=ping comment="Monitoring ISP2-B" distance=1 dst-address=\
10.2.2.2/32 gateway=94.140.14.14 scope=10 target-scope=13
add check-gateway=ping comment="Monitoring ISP3-A" distance=1 dst-address=\
10.3.3.3/32 gateway=1.1.1.1 scope=10 target-scope=13
add check-gateway=ping comment="Monitoring ISP3-B" distance=1 dst-address=\
10.3.3.3/32 gateway=76.76.2.0 scope=10 target-scope=13
add comment="Primary Route _ ISP1-A" distance=1 dst-address=9.9.9.9/32 \
gateway=192.168.1.1 scope=10 target-scope=12
add comment="Primary Route _ ISP1-B" distance=1 dst-address=76.76.19.19/32 \
gateway=192.168.1.1 scope=10 target-scope=12
add comment="Secondary Route _ ISP2-A" distance=1 dst-address=\
208.67.222.222/32 gateway=XXX.XXX.181.89 scope=10 target-scope=12
add comment="Secondary Route _ ISP2-B" distance=1 dst-address=94.140.14.14/32 \
gateway=XXX.XXX.181.89 scope=10 target-scope=12
add comment="Tertiary Route _ ISP3-A" distance=1 dst-address=1.1.1.1/32 \
gateway=XXX.XXX.178.121 scope=10 target-scope=12
add comment="Tertiary Route _ ISP3-B" distance=1 dst-address=76.76.2.0/32 \
gateway=XXX.XXX.178.121 scope=10 target-scope=12
/ip route rule
add comment="Standard Route - No ST PCQ" src-address=10.100.100.0/24 table=\
to-ISP1_route
add comment="Standard Route - with ST PCQ" src-address=10.100.200.0/24 table=\
to-ISP1_route
add comment="VIP Route - No ST PCQ" src-address=10.200.123.0/24 table=\
to-ISP2_route
add comment="VIP Route - with ST PCQ" src-address=172.20.234.0/24 table=\
to-ISP2_route
add comment="Other Routes - No ST PCQ" src-address=192.168.111.0/24 table=\
to-ISP3_route
add comment="Other Routes - with ST PCQ" src-address=192.168.222.0/24 table=\
to-ISP3_route


with that config i can no longer access any connected device within the LAN.
when i disable the rules under /ip route rules tab i can access again all connected devices within the LAN regardless of what subnet they belong to as long as i am connected to the LAN or behind any ONT/ONU that is connected to the MT via PPPoE or from the hotspot.

now, what are the additional config i need to do in order to be able to access any connected devices within the LAN without disabling the routing rules under the /ip route rules tab?
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Thu Oct 13, 2022 9:52 pm

I am not really caring about white listing etc.......

So you have three IP addresses that require access to other LANs and this is the only cross LAN commmunication required by those behind the router?

Are the three IPs within the same subnet and if so which one.

Are the three IPs set statically in DHCP leases (wont change)?
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Fri Oct 14, 2022 2:51 am

I am not really caring about white listing etc.......

So you have three IP addresses that require access to other LANs and this is the only cross LAN commmunication required by those behind the router?

Are the three IPs within the same subnet and if so which one.

Are the three IPs set statically in DHCP leases (wont change)?

hi sir anav, i am sorry i don't fully understand your question. where did you get that notion of "having only three IP addresses that require access to the other LANs"? i did not say that. what i said is i need to have access to all devices connected to the LAN whenever my device is connected to anywhere within the LAN without the need to disable the routing rules i made under "/ip route rule" tab, otherwise, it will just defeat the purpose of having a designated WAN route for internet traffic coming from specific subnets.

by the way, i am sorry for the confusion by mentioning the "hotspot". what i meant is before these routing rules was implemented i was able to access any connected devices in the LAN (regardless ot what subnet they belong to) as long as i am also connected (either via MT ether port or behind customer's PPPoE ONT/ONU) to the LAN and even through the hotspot service (my device is connected to the AP of the hotspot) i can also access all devices in the LAN.
(SPECIAL NOTE: i don't need to access devices connected via the hotspot service nor to devices of the hotspot customers).

okay, to avoid confusion and to make it simpler, let's just forget now the Hotspot connected devices and just focus on the devices (ONT/ONU) connected via PPPoE service.
also, let's just focus on the three subnets as an example.

from these rules

/ip route rule
add comment="Standard Route - No ST PCQ" src-address=10.100.100.0/24 table=\
to-ISP1_route
add comment="Standard Route - with ST PCQ" src-address=10.100.200.0/24 table=\
to-ISP1_route
add comment="VIP Route - No ST PCQ" src-address=10.200.123.0/24 table=\
to-ISP2_route
add comment="VIP Route - with ST PCQ" src-address=172.20.234.0/24 table=\
to-ISP2_route
add comment="Other Routes - No ST PCQ" src-address=192.168.111.0/24 table=\
to-ISP3_route
add comment="Other Routes - with ST PCQ" src-address=192.168.222.0/24 table=\
to-ISP3_route

let's temporarily eliminate the other subnets from the LAN and just retain these three subnets and rename the rules to

/ip route rule
add comment="Standard Route - PPPoE-A" src-address=10.100.100.0/24 table=\
to-ISP1_route
add comment="VIP Route - PPPoE-B " src-address=10.200.123.0/24 table=\
to-ISP2_route
add comment="Other Routes - PPPoE-C" src-address=192.168.111.0/24 table=\
to-ISP3_route

NOTE: all PPPoE clients ip addresses are set to static (not changing) all PPPoE servers reside on a single bridge eversince the MT was deployed to production.

Hope these make my requirement clearer to you.

Looking forward to your expertise on this routing problem. Thank you.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Fri Oct 14, 2022 4:42 am

What I meant was plainly stated.
You as the admin require to access different LAN devices, if we are to make special rules we need to know how many IPs they are and where are they coming from?

If its just you the admin or its an entire subnett that needs access to other LAN devices its probably doable within routing rules otherwise mangling comes into the picture.
So please answer the question

who needs what from where to where................. then we can design accordingly.
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Fri Oct 14, 2022 7:42 pm

What I meant was plainly stated.
You as the admin require to access different LAN devices, if we are to make special rules we need to know how many IPs they are and where are they coming from?

If its just you the admin or its an entire subnett that needs access to other LAN devices its probably doable within routing rules otherwise mangling comes into the picture.
So please answer the question

who needs what from where to where................. then we can design accordingly.
Hi Sir Anav, I as the admin needs to access the LAN devices (customers PPPoE ONT/ONU devices as well as some AP devices (used for the hotspot). And i need to have the access wherever i go as long as there is an available ONT/ONU or AP that is connected to the LAN in which my mobile device (laptop or smartphone) can connect to.
(NOTE: PPPoE and Hotspot customers are located in different areas and they are connected to the LAN via EPON OLT and PTP-Radios).
Hence, I need to have access to LAN devices (regardless of its subnet) whether i am at the physical location of the MT or i am in the field at any of the customers' location.

Anyway, I am now able to access any device connected to the LAN (except those mobile devices connected to the hotspot service) after adding below additional routing rules.

/ip route rule

+++++added routing rules to enable remote access of LAN devices++++++

add comment="Standard Route - PPPoE-A" dst-address=10.100.100.0/24 table=\
main
add comment="VIP Route - PPPoE-B" dst-address=10.200.123.0/24 table=\
main
add comment="Other Routes - PPPoE-C" dst-address=192.168.111.0/24 table=\
main

+++++original routing rules (dedicated WAN traffic)+++++

add comment="Standard Route - PPPoE-A" src-address=10.100.100.0/24 table=\
to-ISP1_route
add comment="Standard Route - Hotspot-A" src-address=10.100.200.0/24 table=\
to-ISP1_route
add comment="VIP Route - PPPoE-B" src-address=10.200.123.0/24 table=\
to-ISP2_route
add comment="VIP Route - Hotspot-B" src-address=172.20.234.0/24 table=\
to-ISP2_route
add comment="Other Routes - PPPoE-C" src-address=192.168.111.0/24 table=\
to-ISP3_route
add comment="Other Routes - Hotspot-C" src-address=192.168.222.0/24 table=\
to-ISP3_route

Now, my remaining problem is how can i also use the hotspot connection to be able to access those LAN devices. In other words, I will connect to any of the hotspot APs and log in to the hotspot portal and then i can now be able to remotely access those LAN devices.
I tried replicating same routing rules with the subnet IPs of the hotspot service as destination address and set the table to main but it did not work out, i can't still access the LAN devices.

Second remaining problem is how will i configure to allow just only my PCs (desktop and laptop) and smartphone to access the LAN devices (point of entry is anywhere within the LAN). My initial idea is to whitelist the MAC addresses of my PCs and smartphone so that any other devices (PCs and smartphones) that are not listed cannot be used to access the LAN devices, but i don't know how to configure it yet.
If you have a better way to implement the LAN access restriction aside from whitelisting of MACs, would be most welcome.

Thank you.
 
AzDsL
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 61
Joined: Sun May 22, 2022 4:17 am

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Sun Oct 16, 2022 3:06 am

What I meant was plainly stated.
You as the admin require to access different LAN devices, if we are to make special rules we need to know how many IPs they are and where are they coming from?

If its just you the admin or its an entire subnett that needs access to other LAN devices its probably doable within routing rules otherwise mangling comes into the picture.
So please answer the question

who needs what from where to where................. then we can design accordingly.


Hi Sir Anav, i am wondering why below rule did not work on my hotspot subnet. I can't still access the LAN devices when i connect to the hotspot service.

add comment="Standard Route - Hotspot-A" dst-address=10.100.200.0/24 table=\
main

But it did work on the PPPoE subnet, i was able to access the LAN devices after implementing below rule.

add comment="Standard Route - PPPoE-A" dst-address=10.100.100.0/24 table=\
main


While both of these PBR rules below (for PPPoE-A and Hotspot-A subnets) to direct the traffic coming from the specific subnet going out to a specific WAN did work.

add comment="Standard Route - PPPoE-A" src-address=10.100.100.0/24 table=\
to-ISP1_route
add comment="Standard Route - Hotspot-A" src-address=10.100.200.0/24 table=\
to-ISP1_route


Both the PPPoE Server and Hotspot Server reside on the same bridge without any VLAN.
I also just used the default auto config tab when i created the hotspot server, hence, no special or additional config made on the hotspot service.
The only difference of the two is that PPPoE ONT/ONU devices IP address are set into static while the Hotspot has DHCP.
And as I previously stated, i don't need to access the devices (customers mobile phones and laptops) contected to the hotspot service. I only need to use the hotspot service as gateway to be able to access the LAN devices (in this case the PPPoE ONT/ONU and some APs). In other words, when i am connected to the hotspot service, i should be able to access those LAN devices located at different physical locations and under different subnets.

Do i need to put a rule at firewall to make it work? If yes, what rule it is? Can you give me some sample rule/s?

Also, what is the relationship between the routing rules (/ip route rule) and the firewall rules (/ip firewall filter & /ip firewall mangle)? Can the three works together? Or one overwites the other? I read somewhere in this forum that any mangle rule disables the filter fasttrack rule, but i think fasttrack is not my concern now.

I badly need to fix my config ASAP because access to the LAN devices is very important especially when i am in the field, otherwise, i will be forced to disable the dedicated WAN routing rules.

Hope someone can help me on this. Thank you in advance.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Wed Feb 15, 2023 4:23 am

Been awhile but revisiting this thread.......
The way to access LAN resources when using routing rules is by using MORE routing rules, prior in order, to those rules forcing users out a particular WAN.
Typically

/routing rule add dst-address=SUBNETX action=lookup-only-in table table=main

This would ensure all the subnets behind the router would be able to reach subnet X.
One could make it more specific to only a single subnet Y vice all of them

/routing rule add dst-address=SUBNETX src-address=subnet Y action=lookup-only-in table table=main

++++++++++++++++++++++++++++++++++++

My question, which I was remiss not to ask earlier is about your additions to ensure failover worked properly.
You use routing-mark (which in this case doesnt refer to any mangling) so assuming it refers to them in the routing rules and to clarify in ver7 firmware to the tables..........
Last edited by anav on Wed Feb 15, 2023 2:39 pm, edited 2 times in total.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 18958
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: ROS 6.49.6 How to fix or improve config for Recursive Routing with PBR?

Wed Feb 15, 2023 5:02 am

Further I found the error in your config in the new rules you added.
The pattern detected was
dst 1 - routing mark isp1 virtual gwy (set to the primary gateway) = 10.1.1.1
dst 2 - routing mark isp 1 virtual gwy (set to the secondary gateway) = 10.2.2.2
dst 3 - routing mark isp1 virtual gwy (set to the tertiary gateway) = 10.3.3.3

dst 1 - routing mark isp2 virtual gwy (set to the secondary gateway) = 10.2.2.2
dst 2 - routing mark isp2 virtual gwy (set to the primary gateway) = 10.1.1.1
dst 3 - routing mark isp2 virtual gwy (set to the tertiary gateway) = 10.3.3.3

dst 1 - routing mark isp3 virtual gwy (set to the tertiary gateway) = 10.3.3.3
dst 2 - routing mark isp3 virtual gwy (set to the primary gateway) = 10.1.1.1
dst 3 - routing mark isp3 virtual gwy (set to the secondary gateway) = 10.2.2.2

However in the first block you have an error!
dst 1 - routing mark isp1 virtual gwy (set to the primary gateway) = 10.1.1.1
dst 2 - routing mark isp 1 virtual gwy (set to the secondary gateway) = 10.2.2.2
dst 3 - routing mark isp3 virtual gwy (set to the tertiary gateway) = 10.3.3.3

Who is online

Users browsing this forum: JDF, qatar2022 and 44 guests