First important thing to understand is that there isn’t the perfect universal solution. There can be various differences in setups and each may require something slightly different (e.g things like static or dynamic address can influence a lot). Even when it’s same, it’s often possible to use different ways to reach the goal. You can also use various shortcuts and there’s nothing wrong with them, it’s useless to create “perfect” config that covers everything, when you won’t really need some of that at all.
Other thing that helps is understanding how it works. At least rought idea about packet flow is a must. Then there’s connection tracking, which is part of stateful firewall. Router examines every single packet and tries to put them together, so it knows what belongs to what. When there’s e.g. packet from client:12345 to server:80 and then response from server:80 to client:12345, router understands that they belong to same connection. Connection tracking happens regardles of you using any marks or not, because it’s required for stateful firewall (connection-state option) or NAT. You can optionally use connection marks, if you need to work with some connections. What happens is that router then automatically assigns same mark to all packets of same connection.
About your points:
-
There is no need for passthrough for incoming packets, because marking connection is all you need for them (actually just the first packet). They then go to internal server and nothing special is required for that.
-
It won’t work with in-interface=bridge2, because on IP level the VLANs are incoming interfaces. You’d need two rules for each WAN and that twice, because you also have two LANs, so you’d need to match connections:
- from LAN1 with WAN1 mark
- from LAN1 with WAN2 mark
- from LAN2 with WAN1 mark
- from LAN2 with WAN2 mark
You can see that it’s getting complicated. Imagine that you’d have even more LANs or WANs. So it may actually be better to use what you had before, i.e. don’t care about source, just match connections:
- with WAN1 mark
- with WAN2 mark
As I wrote before, this would break routing for incoming packets, but if you have routing rules telling router that route to local subnets should be looked up in main routing table, it fixes the problem. It’s a compromise, one part may look less clean, but other part can make the whole thing simpler. It’s up to you, what you find better, there’s not just one proper solution.
And you know what’s the most funny part? It’s possible that you don’t need any mangle rules at all. At the beginning you wrote that everything works, even when you don’t have them. So it depends on what exactly you do. The point of this whole thing is to allow to use port forwarding with multiple WANs. But default simple example has only one LAN and one server. It uses one WAN as default for outgoing traffic and you need to override it for connections that came from another WAN.
This applies in your case if you need to forward port from WANx to 192.168.0.x and also port from other WANy to same 192.168.0.x. If you do this, you need this connection and route marking. But if you forward port from WANx to 192.168.0.x and don’t forward any port from WANy to it, and forward another port from WANy to 192.168.254.x but no port from WANx to it, i.e. forwarded ports are only from WAN which target server uses as default, then you don’t need this at all, because your previous routing rules are enough.
