How to merge two link for more throughput

Hi All,

I have one pppoe server and I need more internet for it. I have two wireless link to pppoe_server one of them have 100mb capacity and one of them is 130Mb. How can i use both of link to share bandwith ? I’m sharing small diagram and settings.
please help me to solve this query
IMG_1537.PNG
Router A
/routing ospf instance
set [ find default=yes ] router-id=10.255.255.1
/routing ospf interface
add authentication=simple authentication-key=somekey cost=10 interface=VLANtoRouterB network-type=broadcast
/routing ospf network
add area=backbone network=10.255.255.1/32
add area=backbone network=10.0.0.172/30

Router B
/routing ospf instance
set [ find default=yes ] redistribute-connected=as-type-1 router-id=10.255.255.24
/routing ospf interface
add authentication=simple authentication-key=somekey cost=10 interface=bridge1-Radius network-type=broadcast
add authentication=simple authentication-key=somekey cost=10 interface=ether9-SomeP2PLink network-type=broadcast
add authentication=simple authentication-key=somekey cost=10 interface=ether4-SomeP2PLink network-type=broadcast
add authentication=simple authentication-key=somekey cost=10 interface=“ether10-Router C” network-type=broadcast
add authentication=simple authentication-key=somekey cost=10 interface=“ether6-1100Ahx4 DudeServer” network-type=broadcast
add authentication=simple authentication-key=somekey cost=10 interface=ether5-Main_PPPoE_Server network-type=broadcast
add authentication=simple authentication-key=somekey cost=10 interface=“ether1-AF24 P2P Link For RouterD” network-type=broadcast
add authentication=simple authentication-key=somekey cost=10 interface=VLANfromRouterA network-type=broadcast
/routing ospf network
add area=backbone network=10.10.0.0/23
add area=backbone network=10.0.0.88/29
add area=backbone network=10.0.0.112/29
add area=backbone network=10.255.255.24/32
add area=backbone network=10.0.0.72/30
add area=backbone network=10.0.0.76/30
add area=backbone network=10.0.0.108/30
add area=backbone network=10.0.0.160/29
add area=backbone network=10.0.0.172/30
/ip route
add distance=1 gateway=10.0.0.173

Router C
/routing ospf instance
set [ find default=yes ] redistribute-connected=as-type-1 router-id=10.255.255.26
/routing ospf interface
add authentication=simple authentication-key=somekey cost=10 interface=11GhzLink-toPPoeServer-B network-type=broadcast
add authentication=simple authentication-key=somekey cost=10 interface=ether1-Link from RouterB network-type=broadcast
/routing ospf network
add area=backbone network=10.0.0.44/30
add area=backbone network=10.255.255.26/32
add area=backbone network=10.0.0.72/30
/ip route
add distance=1 gateway=10.0.0.73

Router D
/routing ospf instance
set [ find default=yes ] router-id=10.255.255.12
/routing ospf interface
add authentication=simple authentication-key=somekey cost=15 interface=ether4-NetMetaltoPPoeServer-B network-type=broadcast
add authentication=simple authentication-key=somekey cost=10 interface=ether3-SomeP2PLink network-type=broadcast
add authentication=simple authentication-key=somekey cost=10 interface=ether2-SomeDevice network-type=broadcast
add authentication=simple authentication-key=somekey cost=10 interface=ether1-AF24 P2P Link From RouterB network-type=broadcast
add authentication=simple authentication-key=somekey cost=10 interface=ether5-SomeP2PLink network-type=broadcast
/routing ospf network
add area=backbone network=10.0.0.136/29
add area=backbone network=10.0.0.152/29
add area=backbone network=10.0.0.16/30
add area=backbone network=10.0.0.168/30
add area=backbone network=10.0.0.160/29
add area=backbone network=10.255.255.12/32
add area=backbone network=10.0.0.176/29
/ip route
add distance=1 gateway=10.0.0.161



PPPoEServer-B
/routing ospf area
add area-id=0.0.0.1 default-cost=1 inject-summary-lsas=yes name=pppoe1 type=stub
/routing ospf instance
set [ find default=yes ] redistribute-connected=as-type-1 router-id=10.255.255.7
/routing ospf area range
add area=pppoe1 range=172.31.12.0/24
add area=pppoe1 range=172.31.1.0/26
/routing ospf interface
add authentication=simple authentication-key=somekey cost=10 interface=11GhzLink-from RouterC network-type=broadcast
add authentication=simple authentication-key=somekey cost=20 interface=“ether4-p2pLink From RouterD” network-type=broadcast
add passive=yes
/routing ospf network
add area=backbone network=10.0.0.44/30
add area=backbone network=10.255.255.7/32
add area=backbone network=10.0.0.136/29
add area=pppoe1 network=172.31.12.0/24
add area=pppoe1 network=172.31.1.0/26

/ip route
add check-gateway=ping distance=1 gateway=10.0.0.45
add check-gateway=ping distance=2 gateway=10.0.0.137

You could potentially EoIP tunnel from PPPoE server to Router A taking both paths (as thats where the separate paths converge) then using a bonding interface to bond the 2.

This would likely mean you lose the 30mbit on the 130mbit leg of the link, but being wireless you could run into more overhead loss with out of time packets hitting Router A (if the latency differs on one link)

Another option is to split your userbase using source based routing, i.e. some PPPoE users go via Link A, some PPPoE users go via Link B and try manually manage the load.

Thanks for your help joegoldman

I think this one is not better

can you explain this way to me ?

No one to help ?

No any solution for this topology ?

amt, I can’t specifically state what he means but here are some options.

  • Perform routing based on the source IP address assigned on the PPPoE server (Policy Based Routing)

Let’s say hypothetically your PPPoE assigns your customers IP addresses from the 192.168.254.0/24 network. If this is the case you can tell RouterOS to send packets sourced from 192.168.254.0/25 to the 130 MB link and packets sourced from 192.168.254.128/25 to the 100 MB link.

  • Create multiple PPPoE servers or termination points

Right now, all of your users connect to a single PPPoE server. All traffic destined for your customers goes to and from it. This has the advantage of a single point to manage your IP addressing and user information. It has the down side of being a single point of failure and a bottleneck for traffic.

A way to work with this is split your pool of addresses up and create PPPoE servers in different places of your network. Maybe 1 per tower. This would allow normal routing to take over at the tower with all of it’s benefits like link costing and pathing at layer 3 via a dynamic routing protocol like OSPF. Along with splitting your addressing pool up you can move your current PPPoE server closer to the upstream Internet and allow the tunnels to it be the ones that are diversified.

I’m not ultra familiar with the RouterOS PPPoE server but you should be able to target it at either a central RouterOS UserMan server or another RADIUS server like freeradius or MS NPS. This would keep your user management centralized for but your routing more distributed and resilient.

  • Drop PPPoE Altogether

This is likely the least feasible approach as most tooling is built around PPP but in theory you can deploy other technologies to manage customers and their connections. I think the wireless industry has a unique advantage in this field by having a CPE on almost all of their customers networks. Scripting can be leveraged to automate swapping a non-paying customer’s CPE to a remediation VLAN and back again. You can use other technologies like 802.1x paired with an existing RADIUS back-end, with or without certificates.

Personally, option 3 would be what I would do but I understand that’s not a situation everyone can roll with. As far as your particular question of “merging” your two links. You can either create a pair of EoIP tunnels and bond them like the first user said or use PBR like I indicated to achieve some cross link utilization. Other than that you need to look at the more architectural fixes like #2 and #3.

Hi idlemind,

first off all very thanks for your answer.


this is what I’m trying to do but I could not success … I tried to mangle and route /25 traffic over the other link but its not worked. What can it be the problem ? Or Mangle and routing mark is correct way for this ?

/ip firewall mangle
add action=mark-routing chain=prerouting new-routing-mark=Link-2 passthrough=yes src-address=172.31.1.0/26
/ip route
add distance=1 gateway=10.0.0.137 routing-mark=Link-2

Thanks

Seems right to me. Maybe drop the pass through to ensure it isn’t getting any other routing marks assigned?

There is no any other mangle rule. But when i done this only some upload traffic start over second link.

Only traffic within the /26 would. PBR won’t get you a 50/50 balance.

Yes exactly, but Im testing /26 to see effect
Also /26 enough for 20mb traffic on this pppoe_server. if this solution work with /26 I should see minimum 10mb on second link ether. But I see only few upload..

If you run torch on a router upstream towards “Link-1” do you see traffic sourced within 172.31.1.0/26? If not then PBR is working correctly and the users within the /26 are currently only generating the load you see on “Link-2.”

Just as a catch-all. I imagine the 172.31.1.0/26 network is you obfuscating the addressing. If it’s not and you’re an ISP that is handing out RFC1918 addressing due to limited public IPv4 addressing I’d encourage you to look at RFC6598 and the 100.64.0.0/10 address space. The reason I state that is you don’t want to be giving your customers addressing that could potentially overlap with their own networks if you don’t have to.

Yes I see trafffic on Link-1, I changed the mangle rule as below to test, I add all range to see what happen;

/ip firewall mangle
add action=mark-routing chain=prerouting new-routing-mark=Link-2 passthrough=yes src-address=172.31.1.0/24
/ip route
add distance=1 gateway=10.0.0.137 routing-mark=Link-2

but after this I see that all upload traffic start to go over Link-2. But I need download :slight_smile:

using 172.31.0.x for only example, Yes Im using 100.64.0.0/10 range for customers.

thanks

If traffic is assymetrical you’ll need tell upstream routers which path to send the destination traffic. Start with static routes for simplicity.

If we look at a hypothetical 172.31.1.0/24 and for ease just split it in half, 172.31.1.0/25 through Link-1 and 172.31.1.128/25 through Link-2.

Outbound traffic from PPPoE Customers:

We do the source based routing on the PPPoE server.

Inbound traffic to PPPoE Customers:

Start with static routes on RtrB. Send 172.31.1.0/25 to RtrC and send 172.31.1.128/25 to RtrD. Apply additional static routes to RtrC and RtrD only if necessary to stop it from flowing back up to RtrB.

Ok..
I done as you said.
this is router b;

/ip route
add distance=1 gateway=10.0.0.173
add  distance=1 dst-address=172.31.1.128/25 gateway=10.0.0.74
add  distance=1 dst-address=172.31.1.0/25 gateway=10.0.0.164

On router C

/ip route
add distance=1 gateway=10.0.0.73
add  distance=1 dst-address=172.31.1.128/25 gateway=10.0.0.46

On router D

/ip route
add distance=1 gateway=10.0.0.161
add  distance=1 dst-address=172.31.1.0/25 gateway=10.0.0.164

But theese boards talk each other with ospf, can it be any problem if i add any static route to routers while they talk eachother with ospf. and after this settings do i still need mangle rule ?

Thanks

The PBR component (mangle and route with mark) is for outbound traffic.

The static routes on RtrB and optionally RtrC and RtrD are for inbound traffic.

You need both in reality.

The static routes shouldn’t cause any harm within your environment. That said you may want to look at something that may handle the splitting of the prefixes cleaner. I’d have to think about if we can do it within the constraints of OSPF but iBGP should work without a problem for sure.

I will test it now but before test;
1- create mangle for 172.31.1.0/25 which will come from link-1 and mark route for making output from link-1 ?
2- create one mangle for 172.31.1.128/25 which will be come from link-2, to making output from link-2. ?
3- Further more, do i need to redistribute-static routes as-type-1 in ospf instance ?
are thesee correct ways ?

Hi idlemind,
Should i do this settings ?

No to redistributing the statics.

The mangles should be on the PPPoE server and apply only to traffic sourced from the two /25s

Hi idlemind,
thanks for your help, I did job as your said and it seems work. but I have some questions.
I add static route 172.31.1.0/24 at router B to show 172.31.1.0/24 rane on Router D(cause Link-2 to pppoe_server on routerD) and when i check /ip route on router D 172.31.1.0/24 range already dynamically added by ospf and shows me interface of Link-2(that’s what we need) so I’m not going to add new static route on Router D. should i still need to add static route at router D ?
On PPPoE server there is some public ip’s assigned by radius and one more /26 range pppoe pool, I create only one mangle for /24 ip range which is comes from Link-2( we route them manually) do I need magle rule for others?


Thanks