Community discussions

MikroTik App
 
bcmdevtl
just joined
Topic Author
Posts: 18
Joined: Sat Mar 27, 2021 2:40 am

Multiple WAN, PPPoE and routing challenge

Mon Feb 26, 2024 12:02 pm

I have an RB5009 ROS v7.12.1 which I intend to use for up to 3 WAN connections and with client/guest network on the LAN side.
I will use PPPoE connections to keep track of the data usage per carrier.

ether 1-3 configured as WAN sources with DHCP client configured for each, connected a source on ether1 with IP 192.168.100.1
ether 4 configured as a hotspot bridge (service hotspot+ppp service), configured to use our shore Radius server
ether 5 is physically connected to ether4 with eth cable, pppoe client named pppoe-client is configured establishes via ether5
ether 6 is configured for the client network bridge (10.1.10.0/24)

WAN1 (ether1) is the priority link which will be used for the client data, if WAN1 is down the client data will be switched to WAN2 or WAN3

I'm only able to get the data to flow if I mangle the client traffic and route the traffic directly via the WAN1 gateway, if I try to set the gateway to the pppoe-client interface there is no traffic flow.

In my test setup:
WAN1 gw is 192.168.100.1
PPPoE gw is 192.168.99.1 (pppoe-client establishes with 192.168.99.254)
Business network 10.1.10.0/24

/ip firewall mangle
add action=mark-routing chain=prerouting in-interface=pppoe-business \
new-routing-mark=wan1 passthrough=yes
add action=mark-routing chain=prerouting in-interface=business \
new-routing-mark=business_traffic passthrough=yes src-address=10.1.10.0/24

/routing rule
add action=lookup-only-in-table disabled=no routing-mark=business_traffic table=business_traffic
add action=lookup-only-in-table disabled=no routing-mark=wan1 table=wan1

/ip route
add comment="business client" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=pppoe-vsat pref-src="" \
routing-table=business_traffic scope=30 suppress-hw-offload=no target-scope=10
add comment=wan1 disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.100.1 pref-src="" routing-table=wan1 \
scope=30 suppress-hw-offload=no target-scope=1

Any pointers would be great.
 
Mesquite
Member
Member
Posts: 420
Joined: Tue Jan 23, 2024 9:16 pm

Re: Multiple WAN, PPPoE and routing challenge

Mon Feb 26, 2024 1:26 pm

Draw a diagram and nothing is clear.
Do you have three pppoe connections, and if so why are you using IP DHCP CLIENT?
What type of WAN connections.
Is each give you a private or public IP.........
Is each from a different provider.
Do you have any vpns coming into the router
Do you have any servers on the LAN.
Which is the first backup WAN2 or WAN3
 
bcmdevtl
just joined
Topic Author
Posts: 18
Joined: Sat Mar 27, 2021 2:40 am

Re: Multiple WAN, PPPoE and routing challenge

Thu Feb 29, 2024 12:12 pm

This is for use in a maritime setup , the PPPoE server is on shore and is used for data usage tracking and forms a part of our billing platform.
WAN1 is a low latency satellite link (40ms), private IP received
WAN2 is a high latency satellite link (650+ms), private IP received
WAN3 is a mobile/4G link (low latency), private IP received
All links are from different providers

RADIUS server is on a remote(shore) servers so we're authenticating through the available WAN sources, we're establishing one pppoe-client per WAN, we have two user sources which is business and crew, business traffic should be allowed on all WAN's, priority order is WAN1, WAN2, WAN3. Crew traffic should only be able to go over WAN1, if it is not available this network would not get internet access. On the crew networks we're running hotspot for data allocation.

The key things is to be able to route the traffic via the PPPoE connections so the traffic is accounted for and we can report on the usage.

Running Wireguard for our remote connection only, there may be servers on the LAN side (customer equipment) but we have no control over that.
You do not have the required permissions to view the files attached to this post.
 
Mesquite
Member
Member
Posts: 420
Joined: Tue Jan 23, 2024 9:16 pm

Re: Multiple WAN, PPPoE and routing challenge

Thu Feb 29, 2024 11:03 pm

So can I conclude that
-both sets of users (business and private) are to use WAN1 as their primary source of internet?
-if WAN1 is not available, then WAN2 should be used next by business users ONLY
-if WAN2 is not available, then WAN3 should be used next by business users ONLY.

In other words there is no fail over for private users.

+++++++++++++++++++++++++++++
When you say private IPs.
All you get is a private Gateway IP and a private IP,
Does the gateway change or is it fixed.
 
bcmdevtl
just joined
Topic Author
Posts: 18
Joined: Sat Mar 27, 2021 2:40 am

Re: Multiple WAN, PPPoE and routing challenge

Thu Feb 29, 2024 11:44 pm

That's correct Mesquite
 
Mesquite
Member
Member
Posts: 420
Joined: Tue Jan 23, 2024 9:16 pm

Re: Multiple WAN, PPPoE and routing challenge

Fri Mar 01, 2024 12:33 am

First things is that we need to make all three WANS available on the Main Table. I have no clue about your pppoe connections and dont care, going to focus on the WAN connections.
Dont think any mangles required ??

/ip route
add check-gateway=ping distance=1 dst-address=0.0.0.0 gateway=current-private-gateway-ISP1 routing-table=main comment=WAN1-Update
add check-gateway=ping distance=2 dst-address=0.0.0.0 gateway=current-private-gateway-ISP2 routing-table=main comment=WAN2-Update
add check-gateway=ping distance=3 dst-address=0.0.0.0 gateway=current-private-gateway-ISP3 routing-table=main comment=WAN3-Update


Now we have established as all three being available with WAN1 as primary. I dont see any IP DHCP client settings.
You should have them indicated for each WAN and set so that the settings are, and that the selection for add default route = NO,

However, if the private IP and more importantly, the ISP private gateway you get can change, then you need to do more.
You need to go back to IP DHCP client for each WAN and add default route=yes and set the distance to 255 and add this script to the script area in advanced tab.

:if ($bound=1) do={
:local gw [ /ip dhcp-client get $"gateway-address" ]
/ip route set [ find comment="WANX-Update" gateway!=$gw ] gateway=$gw
}

In this manner, when the gateway IP changes, your manual route will also be updated.

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

So far we have established routes and a failover so that all can access 1, then 2 and then 3
However we only want business users to access WAN2 and WAN3 in the failover scenario (where WAN1 is not working).

My mind is going blank and the only thing I can think of is NOT to mangle but simply to add a routing rules to force those users to wan1 only......
In this manner, the traffic will simply be dropped.
This should not be a problem assuming there are no local addresses ( other local subnets the non business users need to access ).

/routing table add fib name=StickToWAN1
/routing rule
add src-address=10.7.70.0/24 action=lookup-only-in-table table=StickToWAN1
add src-address=10.8.80.0/24 action=lookup-only-in-table table=StickToWAN1

/ip route
add check-gateway=ping distance=1 dst-address=0.0.0.0 gateway=current-private-gateway-ISP1 routing-table=main comment=WAN1-Update
add check-gateway=ping distance=2 dst-address=0.0.0.0 gateway=current-private-gateway-ISP2 routing-table=main comment=WAN2-Update
add check-gateway=ping distance=3 dst-address=0.0.0.0 gateway=current-private-gateway-ISP3 routing-table=main comment=WAN3-Update

add distance=1 dst-address=0.0.0.0 gateway=current-private-gateway-ISP1 routing-table=StickToWAN1 comment=WAN1-Update
 
bcmdevtl
just joined
Topic Author
Posts: 18
Joined: Sat Mar 27, 2021 2:40 am

Re: Multiple WAN, PPPoE and routing challenge

Tue Mar 05, 2024 3:47 am

Thanks Mesquite, adding the routes and that routing is ok, but the main challenge remains to force the traffic to go via the pppoe so the traffic is recorded. Working on a solution using VRF's to achieve this, hopefully get over the finish line soon.

Thanks for the script to bind the gw-ip.

Who is online

Users browsing this forum: Ahrefs [Bot], anav and 24 guests