Load Balance 2 WAN's with Two local iface's

alex_rhys-hurn · December 15, 2006, 3:12pm

Hello all,

This is an overview of my network:

WAN1 = ISP1
WAN2 = ISP2
LAN1 = Wisp Customers
LAN2 = Cybercafe

I have WAN1 and WAN2 successfully load balancing with persistent connections as per this on the wiki: http://wiki.mikrotik.com/wiki/Load_Balancing_Persistent

I have LAN1 as the only interface that is being subjected to the Mangle rules and therefore traffic in on LAN1 is the only traffic that is being load balanced across the two ISP’s

LAN2 Traffic is not being marked in anyway and so is following the third default route for non marked traffic and is going out on WAN1 only.

You can see then that I want to add LAN2 to be marked by the mangle rules and so also to use the benefit of the load balancing on the two WAN’s.

My Question is then: How do I add a second in-interface argument to the existing mangle rules. For e.g. I have this rule:

add chain=prerouting src-address-list=odd in-interface=LAN1 action=mark-connection new-connection-mark=odd passthrough=yes

You can see that it says in-interface=LAN1.

I want it to say in-interface=LAN1 AND LAN2 or some such thing.

I suppose that I could simply duplicate the whole shebang chaning the in-interfaces for LAN2 but will that work?

Its a production system and unfortunately it is not easy to simulate this for me, so if you guys reckon one of these will work, tell me and I will have faith and try…

For more info my configs are below.

Thanks to all,

Alex

Current Configs:
Mikrotik RouterOS 2.9.34, on x86 PC Server

/ ip firewall mangle
add chain=prerouting src-address-list=odd in-interface=LAN1 action=mark-connection new-connection-mark=odd passthrough=yes comment=“1-Persistent” disable=yes
add chain=prerouting src-address-list=odd in-interface=LAN1 action=mark-routing new-routing-mark=odd comment=“2-Persistent” disable=yes
add chain=prerouting src-address-list=even in-interface=LAN1 action=mark-connection new-connection-mark=even passthrough=yes comment=“3-Persistent” disable=yes
add chain=prerouting src-address-list=even in-interface=LAN1 action=mark-routing new-routing-mark=even comment=“4-Persistent” disable=yes
add chain=prerouting in-interface=LAN1 connection-state=new nth=1,1,0 action=mark-connection new-connection-mark=odd passthrough=yes comment=“5-Persistent” disabled=yes
add chain=prerouting in-interface=LAN1 action=add-src-to-address-list address-list=odd address-list-timeout=1d connection-mark=odd passthrough=yes comment=“6-Persistent” disabled=yes
add chain=prerouting in-interface=LAN1 connection-mark=odd action=mark-routing new-routing-mark=odd passthrough=no comment=“7-Persistent” disabled=yes
add chain=prerouting in-interface=LAN1 connection-state=new nth=1,1,1 action=mark-connection new-connection-mark=even passthrough=yes comment=“8-Persistent” disabled=yes
add chain=prerouting in-interface=LAN1 action=add-src-to-address-list address-list=even address-list-timeout=1d connection-mark=even passthrough=yes comment=“9-Persistent” disabled=yes
add chain=prerouting in-interface=LAN1 connection-mark=even action=mark-routing new-routing-mark=even passthrough=no comment=“10-Persistent” disabled=yes

/ ip firewall nat
add chain=srcnat connection-mark=odd action=src-nat to-addresses=xxx.xxx.xxx.xxx to-ports=0-65535
add chain=srcnat connection-mark=even action=src-nat to-addresses=yyy.yyy.yyy.yyy to-ports=0-65535

/ ip route
add dst-address=0.0.0.0/0 gateway=xxx.xxx.xxx.xxx scope=255 target-scope=10 routing-mark=odd
add dst-address=0.0.0.0/0 gateway=yyy.yyy.yyy.yyy scope=255 target-scope=10 routing-mark=even
add dst-address=0.0.0.0/0 gateway=xxx.xxx.xxx.xxx scope=255 target-scope=10

Diatheke · December 15, 2006, 9:02pm

I would try exactly what you already suggested. Do the whole /ip nat mangle for the next in-interface. Though I will admit I have never tried this I have no reason to believe it wouldn’t work.

/ ip firewall mangle
add chain=prerouting in-interface=LAN1 connection-state=new nth=1,1,0 \
    action=mark-connection new-connection-mark=odd passthrough=yes comment="" \
    disabled=no 
add chain=prerouting in-interface=LAN1 connection-mark=odd action=mark-routing \
    new-routing-mark=odd passthrough=no comment="" disabled=no 
add chain=prerouting in-interface=LAN1 connection-state=new nth=1,1,1 \
    action=mark-connection new-connection-mark=even passthrough=yes comment="" \
    disabled=no 
add chain=prerouting in-interface=LAN1 connection-mark=even action

/ ip firewall mangle
add chain=prerouting in-interface=LAN2 connection-state=new nth=1,1,0 \
    action=mark-connection new-connection-mark=odd passthrough=yes comment="" \
    disabled=no 
add chain=prerouting in-interface=LAN2 connection-mark=odd action=mark-routing \
    new-routing-mark=odd passthrough=no comment="" disabled=no 
add chain=prerouting in-interface=LAN2 connection-state=new nth=1,1,1 \
    action=mark-connection new-connection-mark=even passthrough=yes comment="" \
    disabled=no 
add chain=prerouting in-interface=LAN2 connection-mark=even action

Trying to get src-nat and dst-nat with load balancing is a trick I haven’t been able to figure out though http://forum.mikrotik.com/t/multiple-wan-gateways-with-src-and-dst-nat/10557/1 The outbound stuff passing through the router works pretty well but I ultimately switched back to WRAP w/ Debian.

alex_rhys-hurn · December 16, 2006, 1:18pm

Thanks for the resposne diatheke,

With your brain agreeing with mine that it should work in principle, I have gone ahead and tried it.

First positive thing is that I haven killed everything and locked myself out

Second, it doesnt seem quite stable. I mean traffic is passing and accorting to conntracking odds and evens are being handed out correctly.

I just have a feeling that something is sluggish. I do think that there are more even connections than odd ones.

Its early days, I will look at it.

On the SCR-NAT and DST-NAT issue, I agree with you this is not working well. Luckily for me I dont have much need for the incoming and so it is all coming in through only one link. Outgoing no problems though.

You do have to do the persistent sessions one though for peoples online banking to work…

Thanks, Alex.

Diatheke · December 18, 2006, 5:06pm

In general I’ve decided I don’t like the ROS solution of load balancing and abandoned using ROS for this type of application. I have been decidedly happier using Debian GNU/Linux with a custom kernel and things like the iproute package.

In ROS you mark up sessions and route on those marks. I prefer to hash out the destination routing table and then use fast dead gateway detection. This way you don’t need to worry about persistent connections simply because all traffic uses a particular gateway.

Some might argue that you don’t get as good of load balancing using this method over checking for new sessions, marking them and alternating them across your gateways and that may be true for some.

My experience is my WAN gateways are very well balanced and I can do src-nat and dst-nat on my router/gateway interfaces. I also do some custom DNS views from a BIND process routing on my “router” to give inbound redundancy as well.

You might want to check your CPU and make sure its not running high. I also find it helpful to look at the connection table to make sure everything is going where you expect it to.

alex_rhys-hurn · December 18, 2006, 9:58pm

Presumably your debian solution also plays well with web-cache? Sounds like a good solution.

Unfortunately I am a consultant supporting my client on the East African Coast, and am not actually on site… THis makes it difficult to re-develop a solution. Let me think about it a while.

ROS is also not easy to get to work well with the load balancing and web-proxy.

Seems ROS is more suited to some wireless and basic core router tasks…

Salaams,

Alex

Diatheke · December 19, 2006, 12:28am

Yes, plays well with web cache. Reason being its nothing more than something like:

Destination
1.1.1.1 gateway1
1.1.1.2 gateway2
1.1.1.3 gateway1
1.1.1.4 gateway2

The trick is you don’t really add all 4+ billion destinations. You simply look for a destination in cache if its not there you put it in the cache and alternate between destinations; the cache ages out in some interval (I like 1 hour). This has worked much better for me than marking sessions and then matching sessions to routes. This solution doesn’t even require connection tracking to load balance with this method.

I agree, I really like these MTs for extremely flexible APs and routing (let something else do your load balancing). For me core router requires MPLS so that I can do MPLS TE + VPN. Since ROS doesn’t have any MPLS support I can’t use it for core.

gustkiller · December 28, 2006, 1:22am

i have a friend that did something like that destinations - gateway load balance like you did in debian, but he is a dumb that only sell the script with all that configuration for mt :@