BGP working on both interfaces. But TX only goes back out a single interface.
Is it possible to route the traffic received on AT&T circuit to go back out using the AT&T GW and route the traffic received on the COGENT circuit to go back out using the COGENT GW?
Maybe using MARK-ROUTING action for all traffic on a specific IN INTERFACE?
Then set the Routing Mark on the 0.0.0.0/0 route for each provider?
I attempted a prerouting mangle for all traffic on the Cogent Interface to Mark Routing ‘cogent’, then I adjusted the 0.0.0.0/0 route that goes to the Cogent gateway to use routing mark ‘cogent’ but the CCR1072 went from 41% cpu to 100% cpu
I don’t know whether you’ve taken a shortcut only when describing the configuration here or when implementing it. To assign a routing-mark X directly to anything that came in via interface Y while a default route with that routing-mark X goes via that interface Y effectively causes anything that came in through interface Y to be looped back out that interface.
What you actually need is to remember the in-interface of the download direction of a particular traffic stream and to assign a corresponding routing-mark to the corresponding traffic in the upload direction; to do that, you need to assign a connection-mark based on in-interface while handling the download direction, and assign a routing-mark based on the connection mark and in-interface (which must be “any but the one referred to in the first step”) in the upload direction.
If the above is what you’ve actually done, but you hadn’t used any rule which would rely on connection tracking in the router’s firewall ever before doing that, that’s been a quantum leap because as long as there is nothing in the firewall that would rely on connection state (or create one), the connection tracking module is not activated. And once you activate it by adding the first rule which needs it, the amount of CPU spent on handling a packet raises substantially. So if the machine was at 40% without connection tracking, it may easily need 120% of its available capacity now to deal with the same traffic volume. The very essence of connection tracking is to compare every single packet to the complete list of existing connections. Of course the process is highly optimized but it still takes quite some CPU.
And there is no way out, you cannot connection-track only the incoming traffic from one of the WANs, because you need the context provided by the connection tracking to determine the outgoing WAN for the mirror traffic.