Network engineer here. I've been stuck on this for a few days now. I've made a little bit of progress but ultimately I am stuck. I'm running Router OS v6.48.6 on a hEX S.
I have two WANs at home (no load balancing or anything fancy, they just have different metrics). One WAN is for work, so I have a bunch of static routes to send traffic out that link. The other WAN is for normal home networking needs and is the best default route in my routing table.
I have a couple of servers at home, and some DNS records that point to my work WAN IP. So, I am attempting to dst-nat (port forward) HTTPS traffic that hits my work WAN (via the A record I've published). The traffic is getting dst-nat'd to my server just fine; I can see it show up on when using tcpdump on the server. The server is in one of my LANs with the rest of the clients that normally get routed out the personal WAN, not the work WAN. So, if traffic comes in on the work WAN, having it go back out the personal WAN won't work. To prevent this, I'm using the mangle feature to apply routing, packet, and connection marks on the incoming dst-nat'd traffic so that I can catch them on their way back out (assuming the "connection" mark will apply to both the incoming and outgoing traffic). Here are my mangle rules:
Code: Select all
/ip firewall mangle
add action=mark-routing chain=prerouting comment="Tag packets going back to IPs in the NAS-Web-Access list AND in through the work circuit with the routing mark \"route-to-work-circuit\"" connection-mark=\
nas-web-conn-inbound dst-address-list=NAS-Web-Access new-routing-mark=route-to-work-circuit passthrough=yes
add action=mark-connection chain=prerouting in-interface="ether4 (work)" new-connection-mark=nas-web-conn-inbound passthrough=yes src-address-list=NAS-Web-Access
add action=mark-packet chain=prerouting in-interface="ether4 (work)" new-packet-mark=nas-web-packet-inbound passthrough=yes src-address-list=NAS-Web-Access
add action=mark-routing chain=prerouting comment="Mark route - traffic destined to addresses in Work-Mgmt-Networks" dst-address-list=work-Mgmt-Networks new-routing-mark=route-to-work-circuit passthrough=yes
The IPs of remote clients that I want to allow to hit the server on my LAN are defined in the NAS-Web-Access address list. I am new to MikroTik's approach on route-maps, but my intent here was to tag both the "packet" and the "connection" coming in from the remote client to catch them on their way in, and then apply a route mark on the reply packets that would go back out that is based off the connection mark applied to the incoming packet.
Here are my routes. Note that the default route out the personal WAN is missing from this list because it's a dynamic route from a DHCP assigned interface. However, if I were to represent it statically, it would look like this:
add distance=1 gateway="Personal (VLAN 201)"
Code: Select all
/ip route
add distance=1 gateway="ether4 (Work)" routing-mark=route-to-work-circuit
add distance=5 gateway="ether4 (Work)"
And my NAT rules... I am testing with port 80 right now, and then will switch over to 443 once I iron out the configs. So,
Code: Select all
/ip firewall nat
add action=dst-nat chain=dstnat comment="Test - redo forwarding port 80 from Work WAN to 192.186.10.105" connection-mark=nas-web-conn-inbound dst-port=80 in-interface="ether4 (Work)" protocol=tcp to-addresses=\
192.168.10.105 to-ports=80
add action=masquerade chain=srcnat comment="NAT packets that have been route tagged with \"route-to-work-circuit\" AND are going to an address from the \"NAS-Web-Access\"" out-interface="ether4 (Work)" routing-mark=\
route-to-work-circuit
add action=masquerade chain=srcnat comment="Masquerade VPN traffic if outgoing interface is Work Circuit" out-interface="ether4 (Work)" src-address=192.168.175.0/27
add action=masquerade chain=srcnat comment="Masquerade VPN traffic to go out Personal Circuit" out-interface="Personal (VLAN 201)" src-address=192.168.175.0/27
add action=masquerade chain=srcnat comment="Masquerade VLAN 20 out Personal interface." out-interface="Personal (VLAN 201)" src-address=192.168.20.0/25
add action=masquerade chain=srcnat comment="Masquerade VLAN 5 if outgoing interface is Work Circuit" out-interface="ether4 (Work)" src-address=192.168.10.0/25
add action=masquerade chain=srcnat comment="NAT LAN networks out Personal if they are a part of the LAN_Networks_NAT address list." out-interface="Personal (VLAN 201)" \
src-address-list=LAN_Networks_NAT
The problem is the return traffic back to the client. I have been capturing packets on the destination server in my LAN, on my Mikrotik router, and on the remote client that's trying to access the server.
On the server in my LAN at home, I see packets coming in and replies going back out.
On the remote client, I do not see ANY return traffic. I only see the requests going out with a destination address of my work WAN.
On the MikroTik, I see the reply packets coming from the server, hitting the MikroTik's LAN interface (assuming it's getting re-translated), and then I see a packet go out the work WAN interface destined to the remote client's external IP. The remote client is also behind NAT and I manage that end of the network too. However, many of the packets leaving the work WAN interface on the Mikrotik are retransmits for sequence 0, which confirms the remote client is not getting anything at all back.
So, I'm pretty sure this is happening because the replies from the server on my LAN are not being NAT'd back out the TCP port they came in on. In other words, I think the MikroTik is NAT'ing them out the correct interface, but not as part of an existing connection. But, in that case, I would at least expect to see the traffic hit the remote client (and subsequently ignore it), but it's not. Plus, I don't see any src-nat rules incrementing on the MikroTik interface.
Here is the packet capture from the Mikrotik that shows packets leaving my WAN interface and going to the remote client's external IP.
Any ideas on what I can try? I hope I've shared enough of the details. Thank you for your time!