Dual WAN routing, no failover

I have attached a quick paint image of this particular network. I don’t need to failover the 2 ISP’s if one goes down, I really want to route a set of subnets out 1 ISP and obviously the rest out the other ISP. I’m more concerned with getting my Mangle rules and route marking/table setup properly. I cannot user WAN1 interface since the routed traffic will be going out the same interface. Anyone have a similar setup, or point me in the right direction.

Thanks,
network.png

Why do you have switch between your ISPs and your router?

several locations coming in via fiber directly attached to my noc, each location on its own router for various reasons.

I don’t understand. What are the various reasons?

I have several finger networks that come in via fiber and hook in to their respective x86 RouterOS machines, from each of the RouterOS machines they hit the switch, go into the Core Router (Virtual machine in ESXi). This is beside the point however, I am now using 2 ISP’s (went from 500Mbpsx500Mbps to 800Mbpsx800Mbps). I am now going out a 2nd core for the second ISP in a similar fashion as the first.

It looks is if 4 entries would get you there:

A mangle rule with in-interface set to mark routing for traffic from ether4 as “ISP1”.
A mangle rule with SRC address list set to mark routing for traffic from four ISP1 subnets on ether3 as “ISP1”.
An address list containing the four “ISP1” subnets on ether3.
A default route to ISP1 with routing mark “ISP1”

You could also do it the other way around - say just make an address list containing all ISP2 subnets and mangle to “ISP2” for that traffic and set a default route to ISP2 with routing mark “ISP2”.

Whichever way you do it make sure that you have routes back to the source/client networks with the same routing mark as they will be using for outbound traffic.

Routes;

0 A S  dst-address=0.0.0.0/0 gateway=2.2.2.1 
       gateway-status=2.2.2.1 reachable ether2 outbound distance=1 scope=30 
       target-scope=10 routing-mark=isp_2 

1 A S  dst-address=0.0.0.0/0 gateway=1.1.1.1 
       gateway-status=1.1.1.1 reachable ether2 outbound distance=1 
       scope=30 target-scope=10

Mangle (I went from 4 subnets to ISP2 to 14 - hence the marks)

2   chain=input action=mark-connection new-connection-mark=isp2 
     passthrough=no dst-address=2.2.2.2 in-interface=ether2 outbound 

 3   chain=output action=mark-routing new-routing-mark=isp_2 
     passthrough=no connection-mark=isp2
 
 4   ;;; new-location-a-isp2
     chain=prerouting action=mark-routing new-routing-mark=isp_2 
     passthrough=no src-address=10.81.11.20-10.81.11.254 

 5   ;;; new-location-b-isp2
     chain=prerouting action=mark-routing new-routing-mark=isp_2 
     passthrough=no src-address=10.81.12.20-10.81.12.254 

 6   ;;; new-location-c-isp2
     chain=prerouting action=mark-routing new-routing-mark=isp_2 
     passthrough=no src-address=10.81.13.20-10.81.13.254 

 7   ;;; new-location-d-isp2
     chain=prerouting action=mark-routing new-routing-mark=isp_2 
     passthrough=no src-address=10.81.14.20-10.81.14.254 

 8   ;;; a-location-isp2
     chain=prerouting action=mark-routing new-routing-mark=isp_2 
     passthrough=no src-address=10.82.11.32-10.82.11.254 

 9   ;;; b-location-isp2
     chain=prerouting action=mark-routing new-routing-mark=isp_2 
     passthrough=no src-address=10.82.12.32-10.82.12.254 

10   ;;; c-location-isp2
     chain=prerouting action=mark-routing new-routing-mark=isp_2 
     passthrough=no src-address=10.82.13.32-10.82.13.254 

11   ;;; d-location-isp2
     chain=prerouting action=mark-routing new-routing-mark=isp_2 
     passthrough=no src-address=10.82.14.32-10.82.14.254 

12   ;;; e-location-isp2
     chain=prerouting action=mark-routing new-routing-mark=isp_2 
     passthrough=no src-address=10.82.15.32-10.82.15.254 

13   ;;; f-location-isp2
     chain=prerouting action=mark-routing new-routing-mark=isp_2 
     passthrough=no src-address=10.82.16.32-10.82.16.254 

14   ;;; g-location-isp2
     chain=prerouting action=mark-routing new-routing-mark=isp_2 
     passthrough=no src-address=10.82.17.32-10.82.17.254 

15   ;;; h-location-isp2
     chain=prerouting action=mark-routing new-routing-mark=isp_2 
     passthrough=no src-address=10.82.18.32-10.82.18.254 

16   ;;; i-location-isp2
     chain=prerouting action=mark-routing new-routing-mark=isp_2 
     passthrough=no src-address=10.82.19.32-10.82.19.254 

17   ;;; j-location-isp2
     chain=prerouting action=mark-routing new-routing-mark=isp_2 
     passthrough=no src-address=10.82.20.30-10.82.20.254 

18   ;;;queue marking download
     chain=postrouting action=mark-packet new-packet-mark=down passthrough=no 
     dst-address=10.0.0.0/8 

19   ;;;queue marking upload
     chain=prerouting action=mark-packet new-packet-mark=up passthrough=no 
     src-address=10.0.0.0/8