Hi All

I have a level 6 RouterOS server connected to two different ISP networks (X and Y) via BGP using our own ASN and IP blocks. The BGP is up and running between both ISP networks and is working great.

The problem I have encountered is that certain traffic coming in from network X gets replied to via network Y. The rx and tx traffic are on separate interfaces.

My questions are:
How do I set the reply traffic to go out over the same interface in came in on?
Does this ever happen out in the general Internet?

Thanks
Garth

How do I set the reply traffic to go out over the same interface in came in on?

You identify the networks it happens to and write a BGP policy (using routing filters) that assigns a weight or local preference to the route you want traffic to take. You basically have two routes to a given destination and you don't like the way traffic is going 'naturally', so you need to manually intervene and assign better metrics to the path you want traffic to take.

Does this ever happen out in the general Internet?

Yes.
Let's say, for example, that your network MyNetwork is trying to get to resource BrilliantSite. Resource BriliantSite has two uplinks: Provider BigGuy and provider LittleGuy. Your network has uplinks to provider LittleGuy and provider MediumGuy, which in turn has an uplink to provider BigGuy. The path to BrilliantSite from your point of view is shortest through LittleGuy, so that's how your router will send traffic. But resource BrilliantSite has a contract where sending through BigGuy is way cheaper for them, and their network engineers unfortunately only configured a policy where all traffic goes through BigGuy as long as that provider is up without also making their inbound path via LittleGuy less desirable by prepending the AS path. As a result you get traffic from BrilliantSite through BigGuy, but send it out via LittleGuy.

Realistically, though, it often doesn't matter if traffic is perfectly symmetric. Internet routing happens via BGP, and BGP specifically wasn't written to find the shortest or fastest routing path. BGP was explicitly written to find the BEST path, where best is defined as the path that most reflects the routing policy people want to enforce on traffic paths - that could be short, fast, cheap, whatever. Though of course it's absolutely valid to say "I want symmetric routing" and and to try and enforce that. But similarly it's also valid for two end points to choose the cheapest path every time, which could well lead to asymmetric routing.

Thanks Fewi. Very informative.

I am concerned on using asymmetric routing as we will be using a lot of voice traffic over one of the routes. From what I have read, to force outgoing traffic to use the same interface it was initiated on I would need to use policy based routing and mark incoming packets to use one of two routing tables. Would this be the best / only option to force symmetric routing? Is it possible to do this without nat?

Garth