Use same gw/route for outbound traffic

Hello,

I’m trying to implement some sort of policy routing by using connection- and routing marks in order to return incoming traffic on the same outbound path. Basically I’m having two upstreams ISP A and ISP B, which both send me a default-route via BGP.

ISP A is main-uplink (local-pref=200 and no bgp prepend) and handles most in- and outbound traffic.
ISP B (local-pref=100 and bgp prepend) is mostly for fallback and only has very little inbound traffic.

Now I would like to force all inbound traffic coming from ISP B to return back outbound via ISP B route and not via ISP A. All other outbound traffic should be routed via ISP A. The reason for this is that ISP B has a better route for a few, directly connected networks, but which have a very bad route over ISP A. Since I only receive default routes, obviously my router can’t make routing decisions himself. For now I do not want to add those affected networks to my routing table manually nor I want to switch towards receiving a partial or full table by on of the two ISPs. So the only feasible solution would be some sort of policy-based routing.

Somehow it seems I can’t get this working, even after looking at a few other examples here on the forums. Running on latest ROS 6.40.1.
Is there something wrong with my mangle rules or am I missing something else?

Thanks in advance

> ip firewall mangle print

chain=forward action=mark-connection new-connection-mark=ISP_A passthrough=yes in-interface=eth1-isp-A log=no log-prefix="" 
chain=forward action=mark-connection new-connection-mark=ISP_B passthrough=yes in-interface=eth1-isp-B log=no log-prefix="" 

chain=prerouting action=mark-routing new-routing-mark=transit_ISP_A passthrough=no connection-mark=ISP_A log=no log-prefix="" 
chain=prerouting action=mark-routing new-routing-mark=transit_ISP_B passthrough=no connection-mark=ISP_B log=no log-prefix="" 

chain=output action=mark-routing new-routing-mark=transit_ISP_A passthrough=no connection-mark=ISP_A log=no log-prefix="" 
chain=output action=mark-routing new-routing-mark=transit_ISP_B passthrough=no connection-mark=ISP_B log=no log-prefix=""


> routing filter print

chain=isp-A-in address-family=ip invert-match=no action=accept set-routing-mark="transit_ISP_A" set-bgp-local-pref=200 set-bgp-prepend-path="" set-bgp-med=0 

chain=isp-B-in address-family=ip invert-match=no action=accept set-routing-mark="transit_ISP_B" set-bgp-local-pref=100 set-bgp-prepend-path="" set-bgp-med=0


> ip route print

 0 ADb  dst-address=0.0.0.0/0 gateway=A.A.A.A gateway-status=A.A.A.A reachable via  eth1-isp-A distance=20 scope=40 target-scope=10 routing-mark=transit_ISP_A bgp-as-path="65534" bgp-local-pref=200 bgp-med=0 bgp-origin=igp received-from=AS_ISP_A
 
 1 ADb  dst-address=0.0.0.0/0 gateway=B.B.B.B gateway-status=B.B.B.B reachable via  eth1-isp-B distance=20 scope=40 target-scope=10 routing-mark=transit_ISP_B bgp-as-path="65535" bgp-local-pref=100 bgp-med=0 bgp-origin=igp received-from=AS_ISP_B

Since you already do BGP just solve it with that.

For ISP A, accept the same routes you prefer ISP B for but use AS path prepending and local preference to make them less preferred than the ISP B path.

For ISP B, accept only the routes you want and prefer that link for those routes with normal BGP methods (AS path prepending and local preference).

No policy routing “magic” required.

Yes, but here’s the point. I’m getting default routes only from both ISPs.

The first and most practical is to receive more routes from each. Usually an upstream provider can send you a default, a default plus their routes, full table or any combination.

Trying to do tricks with connection tracking shouldn’t be necessary and definitely isn’t the first solution you should be looking at while entirely possible.

A last note, the Internet was designed to not care about traffic symmetry so under normal circumstances nothing bad will happen from sending the return traffic out one of the two paths. It may not be ideal for your situation of trying to make better use of an otherwise ideal connection.

Connection tracking is a way higher cost (CPU / RAM) fix than to just use routing.