i've what seems like a simple little routing situation here that's really kicking me to the curb.
our gateway router, 10.0.0.2, is advertising a default route to devices through a gre tunnel to a different site who's routers we don't control. when the traffic from those devices comes in, i need it sent through an inspection appliance (10.1.1.69) before being allowed out on the internet, or to resources behind 10.0.0.1.
i'm open to suggestions if i have the best implementation here. i used the gre tunnel between 10.0.0.2 and 10.1.1.69 so i can have a single policy route to control all the traffic right as it appears through the gre tunnel from the other site.
i've blown multiple days into trying to make this work with mark routing, so i'd really appreciate the help. mark routing is new to me, so details would be appreciated.
when doing a packet capture on 10.0.0.2, it seems the traffic is being sent out to the internet, but has no path back? it's a real head scratcher to me..
i would also be fine with letting the traffic go from 10.1.1.69 directly to 10.0.0.1 and then out to the internet via 10.0.0.2 if that's the more correct thing to do. but allegedly asymmetric routing is generally best avoided.
you mean on 10.0.0.2? no i haven't. i don't quite know which packets to mark.
there was a post on here a while about just such a scenario that i can't find anymore. and some routeros packet flow ninja solved it with some kind of firewall accept rule. since the packets are traveling through 10.0.0.2 twice, it somehow messes with connection tracking. i'm sure it's just that simple, i'm just not quite smart enough.
It would be easier if the 10.1.1.69 box is able to apply SRCNAT on the packets after inspection before sending them further. It doesn't have to be a "masquerade" kind of SRCNAT to a single address, you can also "netmap" to a new source address range to still be able to identify the real source.
With the source address changed, you avoid the issues with connection tracking on 10.0.0.2 due to the same packet (dst/src) going through it twice.
If 10.1.1.69 is not capable of doing NAT by itself, place an additional router between 10.0.0.2 and 10.1.1.69.
Let's says the sending host is 10.1.20.5. There is only one conntrack entry in the table for SRC 10.1.20.5 DST 8.8.8.8. Once the 1st packets is done with #3, this conntrack entry is updated to be SRCNAT'ed, including the new mapped source address. EDIT: see my post below.
The 2nd packet that 10.1.20.5 sends to 8.8.8.8, at #2 before being forwarded to 10.1.1.69, will have this SRCNAT applied, and 10.1.1.69 no longer sees a packet with source 10.1.20.5.
At #310.1.1.69 now sends a packet with SRC == local address of router DST == 8.8.8.8 back to 10.0.0.2, this will be registered as a new conntrack entry with the router being the source.
Later responses at #4 will be directed to the input chain (if not blocked by firewall) and never sent back to 10.1.20.5 anymore.
Thinking more about that, I think it could be that SRCNAT is not applied at all. I've to do some tests to verify.
Because the NAT rules are normally processed only once per connection (when the first packet goes through the chains). So, after the 1st packet exists at #2 (with no NAT), the conntrack entry is done with checking NAT rules, which means the packet that comes back at #3 from 10.1.1.69 will have connection-state=established and no NAT rule will be consulted for it anymore.
Which means the real problem will be that when the packet exists towards WAN, no SRCNAT will be applied. Which is equally as bad because response will not come back from the internet. Unless we have another router between 10.0.0.2 and WAN that does the SRCNAT.
i think something about exactly this, if i recall correctly. i'm quite certain that on that one forum post that got this to work they just had conntrack ignore the first pass through 10.0.0.2 or something..
Indeed, very good idea. I just did a test configuration with notrack and it does work!
Test CHR has default route to WAN via 172.20.80.1 on ether3, with masquerade on out-interface=ether3.
Clients connected to ether2 is in 192.168.88.0/24 subnet, using the routers' 192.168.88.1 as gateway. In this case, 192.168.88.1 plays the role of 10.0.0.2 in the setup above, and client 192.168.88.254 is 10.1.20.25.
On ether1 router obtains address 10.12.1.247 via DHCP and has remote router at 10.12.1.1. 10.12.1.1 plays the roles of the appliance 10.1.1.69. It logs the packets received from 192.168.88.1 and return them back to 10.12.1.247.
Client on ether2, for example 192.168.88.254 sends packets to the internet, for example to forum.mikrotik.com at 159.148.147.244, using the CHR at 192.168.88.1 as gateway.
Raw notrack rule makes sure this is not tracked.
Routing rule directs this packet to use via-rb routing table
Routing table via-rb has default route using 10.12.1.1 (on ether1) as gateway.
Packet is directed to 10.12.1.1, this router logs and sends the packet unchanged back to 10.12.1.247 (address of CHR on ether1.
Packets is processed normally by firewall with connection tracking, because in-interface=ether1, both RAW rule and routing rule no longer match this packet.
Packet uses default main table, which has 172.20.80.1 on ether3 as gateway, NAT is applied correctly.
Response packet arrived via ether3 are processed normally by conntrack and are un-NAT-ed and send back to 192.168.88.254 via ether2.
Connection works normally. We can see in the traceroute the two times the packet reaches 192.168.88.1 (which has the role of 10.0.0.2). The 2nd hop is the 10.12.1.1 "appliance".
