What happens is that the client request comes to the IP address of the WAN2, is dst-nated to the IP address of the server on the LAN and delivered there. However, the response of the server uses the routing rules which do not use WAN2 as gateway, so the response is sent out via WAN1. I’m not sure whether it is sent out with the source address of WAN2 and thus the ISP doesn’t let it through, or whether it is sent with the source address of WAN1 so the client doesn’t accept it, but the result is effectively the same in both cases.
So what you need to do is to use connection-marking and routing-marking, so that requests which came from outside to WAN2’s address would be responded via WAN2 and requests which came to WAN1’s address would be responded via WAN1.
Look here for details.