Hi there!
While debugging some unrelated issues I re-discovered a problem in my setup that I was and still am stuck at. Here is my setup (relevant config export in attachment):
- I have two ISPs, ISP1 and ISP2
- ISP1 gives dst-nats me a public IP to 10.11.33.1 (let's call iIP1 the public IP address dst-natted here).
- ISP2 gives me public IP 1.2.3.82 (let's IP2 the public IP address) anonymized, don't want even more bots knocking on my door)
- ISP1 is the primary ISP, ISP2 is my backup. The failover is realized by marking connections in the mangle table and setting up routes appropriately. This works perfectly.
I have DNS names to IP1 and IP2 to access my NAS machine. From the outside, everything works well. From the inside, accessing IP1 works well too. However, accessing IP2 does not. After analyzing the rules, I reached the conclusion to what's happening, that is pretty close:
1) Machine from LAN tries to initiate a connection to the NAS, sends out TCP SYN to IP2.
2) Router sees the packet, sends it over ISP1 (primary ISP).
3) Router sees the packet again returning from ISP2 (backup ISP), but it does not mark it at this time (at least I don't see the mark in the table, and I believe the marking is done after SYN-ACK is seen).
4) Target machine doing the NAS responds SYN-ACK, but for some reason it is probably sent out using the default rule to ISP1.
Now, if I hardcoded the IP1 and IP2 into to router, I'm sure I can come up with a few firewall rules to handle this specific issue. If those IPs would be directly on the interfaces, I could also build a shortcut using srcnat (I think). But I wonder if there is a standard solution for this kind of configuration? I think it's pretty common. Ideas and suggestions appreciated.
Cheers,
Krakonos
PS: I'm not so desperate to fix it, so I won't be cooking crazy solutions - we mostly don't need the backup interface to be accessible, but I'm curious about it.