Community discussions

MikroTik App
 
User avatar
Taner
just joined
Topic Author
Posts: 12
Joined: Thu Nov 22, 2018 11:48 am

Issue with Source NAT over IPSEC

Thu Nov 10, 2022 3:30 pm

Hello All,

I have an issue with implementation of source nat over IPSEC tunnel.

These addresses are used for IPSEC VPN between my site and remote:
52.0.14.116/32 Remote public address
62.216.146.250 My public address

This address is for LAN bridge interface of the Mikrotik:
10.10.10.1/24

This is my server address:
10.10.20.20

This is the remote server:
192.168.1.21/32

This is the address with which I should nat my server 10.10.20.20:
115.249.109.170

My server 10.10.20.20 is routed to the Mikrotik LAN network 10.10.10.0/24 from another internal router in my network.
set routing-options static route 192.168.1.21/32 next-hop 10.10.10.254. (This router has L3 interface with address 10.10.10.254)
My server is VM in vMware Esxi host, the esxi host is connected to switch on the switch is defined the layer 3 interface for gw of the network 10.10.20.0/24 (10.10.20.254).
Between switch and another router, I have OSPF.
The IP for OSPF on the router is for 10.10.100.1 and on the switch is 10.10.100.2.

This router and Mikrotik see each other directly as directly connected.

You can skip this detail but for more information I will provide it:
The Mikrotik Router is also VM machine under vMware Esxi host.
The Mikrotik Router receive his WAN and LAN over VLAN from the Juniper router with IP 10.10.10.254.
The Juniper router is core router with zone-based firewall.
The Mikrotik router is putted in Zone A.
My server 10.10.20.20 is on the Zone B.
Between zones A and B the traffic is accepted from 10.10.20.20 to 10.10.10.0/24.
Also I have Zone for WAN communications.
The zone is untrust to untrust.
And there I accepted the traffic between the WAN IP address of the Mikrotik and the remote site for IPSEC VPN.


In the end:
The goal is to forward the traffic from my server 10.10.20.20 to remote server 192.168.1.21 through VPN IPSEC with the source nat of my server 10.10.20.20 with IP address 115.249.109.170.

For now, the situation is that the IPSEC VPN is established between the remote site 52.0.14.116/32 and my site 62.216.146.250..
When I initiate traffic from my server 10.10.20.20 to the remote server behind the VPN IPSEC I see only how my packets reach to Mikrotik router on its local interface 10.10.10.1 and then I see time out.
I see also in the logs for source nat this:
srcnat: in:LAN out:WAN, connection-state:new src-mac 00:10:db:ff:10:00, proto TCP (SYN), 10.10.20.20:65355->192.168.1.21:1313, len 52
And here I suspect my server is not src nat-ed with 115.249.109.170 through IPSEC VPN.

The remote site expects me to address 115.249.109.170.
This is my configuration:

# nov/10/2022 13:44:24 by RouterOS 7.4.1
# software id =
#
/interface bridge
add name=Remotesite
add name=LAN
add name=WAN

/interface ethernet
set [ find default-name=ether2 ] disable-running-check=no name=ether1
set [ find default-name=ether1 ] disable-running-check=no name=ether2

/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik

/ip ipsec profile
add dh-group=ecp521 enc-algorithm=aes-256 hash-algorithm=sha256 lifetime=8h \
name=IPSEC prf-algorithm=sha256
/ip ipsec peer
add address=52.0.14.116/32 comment="To remote Site" exchange-mode=ike2 \
local-address=62.216.146.250 name=To remote site profile=IPSEC

/ip ipsec proposal
add auth-algorithms=sha256 enc-algorithms=aes-256-cbc lifetime=23h name=\
Default pfs-group=ecp521

/port
set 0 name=serial0
set 1 name=serial1

/interface bridge port
add bridge=WAN interface=ether1
add bridge=LAN interface=ether2

/ipv6 settings
set disable-ipv6=yes

/ip address
add address=62.216.146.250/29 interface=WAN network=62.216.146.0
add address=10.10.10.1/24 interface=LAN network=10.10.10.0
add address=115.249.109.170 interface=SourceNATLoo network=115.249.109.170

/ip dhcp-client
add interface=*1

/ip dns
set servers=8.8.8.8,8.8.4.4

/ip firewall filter
add action=accept chain=input dst-address=62.216.146.250 src-address=\
52.0.14.116

/ip firewall nat
add action=src-nat chain=srcnat comment="Source NAT to Remote Site" log=yes \
src-address=10.10.20.20 to-addresses=115.249.109.170

/ip ipsec identity

add auth-method=digital-signature certificate=\
letsencrypt-autogen_2022-11-10T13:03:00Z comment="Identity for Remote site" \
peer=To remote Site remote-id=fqdn:MYVPN.example-example.example.us

/ip ipsec policy
add comment="For traffic between me and remote site through VPN Tunnel " dst-address=192.168.1.21/32 \
level=use peer=To remote Site proposal=Default src-address=115.249.109.170/32 \
tunnel=yes
/ip route
add comment="Route all networks trough WAN address 62.216.146.240 of our current \
Mikrotik" disabled=no distance=1 dst-address=0.0.0.0/0 gateway=\
62.216.146.240 pref-src="" routing-table=main scope=30 suppress-hw-offload=\
no target-scope=10

add comment="Route to my server trough GW 10.10.10254 (SRX L3 address v\
lan 624)" disabled=no distance=1 dst-address=10.10.20.20/32 gateway=\
10.10.10.254 pref-src="" routing-table=main scope=30 suppress-hw-offload=\
no target-scope=10

/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set www-ssl certificate=letsencrypt-autogen_2022-11-10T13:03:00Z
set api disabled=yes
set api-ssl disabled=yes
/system clock
set time-zone-name=Europe/Sofia


Could you share your opinion regarding the topic?
Any guidance for troubleshooting?

Thank you in advance.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Issue with Source NAT over IPSEC

Sun Nov 13, 2022 10:25 am

Could you share your opinion regarding the topic?
Any guidance for troubleshooting?
The log row is misleading - it only shows the original src-address (and dst-address) of the packet but not the new reply-dst-address:

With a rule action=src-nat chain=srcnat dst-address=192.168.227.22 log=yes to-addresses=192.168.227.23 in place, when I ping 192.168.227.22 from the router itself (192.168.227.21), the log shows the following:
04:38:41 firewall,info srcnat: in:(unknown 0) out:ether1, connection-state:new proto ICMP (type 8, code 0), 192.168.227.21->192.168.227.22, len 56
As srcnat is the last firewall chain where you can log the packet, there is no way in your case to see the change of the source address - while sniffing shows the payload packets just extracted from received IPsec transport packets, it does not show the payload packets about to be encapsulated into sent IPsec transport packets.

But in my test case where no IPsec policy is in use, I can see the packets to leave with a changed source address when sniffing:
[me@myTik] > tool/sniffer/quick ip-protocol=icmp ip-address=192.168.227.23
Columns: INTERFACE, TIME, NUM, DIR, SRC-MAC, DST-MAC, SRC-ADDRESS, DST-ADDRESS, PROTOCOL, SIZE, CPU
INTERFACE  TIME   NUM  DIR  SRC-MAC            DST-MAC            SRC-ADDRESS     DST-ADDRESS     PROTOCOL  SIZE  CPU
ether1     4.892    1  ->   00:15:5D:FC:E9:15  00:15:5D:FC:E9:16  192.168.227.23  192.168.227.22  ip:icmp     70    0
ether1     5.895    2  ->   00:15:5D:FC:E9:15  00:15:5D:FC:E9:16  192.168.227.23  192.168.227.22  ip:icmp     70    0
As for troubleshooting the issue: as the tunnel is only there for communication between the servers, you can sniff for the IPsec transport packets (in your case, they should be bare ESP ones unless the remote peer is behind a NAT), and you should be able to see one ESP packet to be sent from your CHR to the remote peer per each ping request sent from your server. If you do, it means the transmit direction works on your CHR, and the issue is either at the remote peer or in the receive direction.

If indeed bare ESP is used, it is possible that the remote peer's firewall blocks incoming ESP.
 
User avatar
Taner
just joined
Topic Author
Posts: 12
Joined: Thu Nov 22, 2018 11:48 am

Re: Issue with Source NAT over IPSEC

Mon Nov 14, 2022 9:33 am

Hi Sindy,

Thank you for the information.

[me@myTik] > tool/sniffer/quick ip-protocol=ipsec-esp ip-address=(my server or remote server)

I don't see anything.

Could you clarify how I can check the ESP packet from Mikrotik to see TX and RX traffic in IPSEC tunnel?
Thank you in advance.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Issue with Source NAT over IPSEC

Mon Nov 14, 2022 3:23 pm

In the output of /ip ipsec installed-sa print, do the src-address and dst-address items contain only IP addresses or also ports (after a colon)?
 
User avatar
Taner
just joined
Topic Author
Posts: 12
Joined: Thu Nov 22, 2018 11:48 am

Re: Issue with Source NAT over IPSEC

Tue Nov 15, 2022 9:26 am

In the output of /ip ipsec installed-sa print, do the src-address and dst-address items contain only IP addresses or also ports (after a colon)?

Hi, I see port 4500 on both sides:

[MyMikrotik] > ip ipsec/installed-sa/print
Flags: S - SEEN-TRAFFIC; E - ESP
Columns: SPI, STATE, SRC-ADDRESS, DST-ADDRESS, AUTH-ALGORITHM, ENC-ALGORITHM, ENC-KEY-SIZE
# SPI STATE SRC-ADDRESS DST-ADDRESS AUTH-ALGORITHM ENC-ALGORITHM ENC-KEY-SIZE
0 E 0xF4E95E5 mature remote_IP site:4500 mysiteIP:4500 sha256 aes-cbc 256
1 SE 0x5A4F0CF5 mature mysite_IP:4500 remotesite_IP:4500 sha256 aes-cbc 256
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Issue with Source NAT over IPSEC

Tue Nov 15, 2022 10:36 am

I see port 4500 on both sides:
OK, so there is NAT somewhere between your peers. Nothing in your configuration suggests that, although the WAN configuration is weird - see the note below. So there is probably NAT on the remote end. Nothing bad about that, it just explains why the SAs use UDP-encapsulated ESP rather than bare one, for the cost of a few extra header bytes per packet.

The /ip/ipsec/installed-sa/print shows that some traffic is being sent from your site to the remote one (see the S - SEEN-TRAFFIC flag in the second column of that SA), but nothing comes back (the S flag is missing for the SA from the remote site to you).

You can run /ip/ipsec/installed-sa/print detail interval=1s to see the byte and packet counters of the SAs to increase live, so that you could compare the state when you ping the remote server and when you don't - if no other traffic than the pings matches the traffic selector of the policy, the counters should be steady while not pinging.

Note 1: the own WAN IP address has a /29 mask and ends with .250, so the corresponding subnet address is .248, but the network item says .0. Also the gateway of the default route is .240 which is outside that /29 (.248-.255). So either you have obfuscated the export without paying attention to details, or it only works thanks to multiple side effects.

Note 2: unless it's a consequence of obfuscation again, there is no actual firewall in your Mikrotik configuration, as the default handling in all firewall chains is accept. So the single selective accept rule in your /ip firewall filter changes nothing as everything else is accepted too.
 
User avatar
Taner
just joined
Topic Author
Posts: 12
Joined: Thu Nov 22, 2018 11:48 am

Re: Issue with Source NAT over IPSEC

Wed Nov 16, 2022 12:09 am

I see port 4500 on both sides:
OK, so there is NAT somewhere between your peers. Nothing in your configuration suggests that, although the WAN configuration is weird - see the note below. So there is probably NAT on the remote end. Nothing bad about that, it just explains why the SAs use UDP-encapsulated ESP rather than bare one, for the cost of a few extra header bytes per packet.

The /ip/ipsec/installed-sa/print shows that some traffic is being sent from your site to the remote one (see the S - SEEN-TRAFFIC flag in the second column of that SA), but nothing comes back (the S flag is missing for the SA from the remote site to you).

You can run /ip/ipsec/installed-sa/print detail interval=1s to see the byte and packet counters of the SAs to increase live, so that you could compare the state when you ping the remote server and when you don't - if no other traffic than the pings matches the traffic selector of the policy, the counters should be steady while not pinging.

Note 1: the own WAN IP address has a /29 mask and ends with .250, so the corresponding subnet address is .248, but the network item says .0. Also the gateway of the default route is .240 which is outside that /29 (.248-.255). So either you have obfuscated the export without paying attention to details, or it only works thanks to multiple side effects.

Note 2: unless it's a consequence of obfuscation again, there is no actual firewall in your Mikrotik configuration, as the default handling in all firewall chains is accept. So the single selective accept rule in your /ip firewall filter changes nothing as everything else is accepted too.

Hi,

The source nat is in my mikrotik only for NAT of my Server.
/ip firewall nat
add action=src-nat chain=srcnat comment="Source NAT to Remote Site" log=yes \
src-address=10.10.20.20 to-addresses=115.249.109.170

Between LAN and WAN I don't do nat.
I establish from WAN to remote site IPSEC.
With traffic selector in IPsec policy, I define when Mikrotik receives packets with the source address of the source nat-ed address of my server and the destination address of the remote server behind VPN then put this traffic on the IPSEC.

[My Mikrotik] > /ip/ipsec/installed-sa/print detail interval=1s
Flags: S - seen-traffic; H - hw-aead; A - AH, E - ESP
0 E spi=0xB1B3D71 src-address=Remote_Site:4500
dst-address=Local_site:4500 state=mature auth-algorithm=sha256
enc-algorithm=aes-cbc enc-key-size=256
auth-key="wkwwlwlwlw"
enc-key="wowowoowow"
add-lifetime=19h12m8s/1d10s replay=128

1 S E spi=0x5A4F7F53 src-address=Local_site:4500
dst-address=Remote_site:4500 state=mature auth-algorithm=sha256
enc-algorithm=aes-cbc enc-key-size=256
auth-key="opsoopsppsppp"
enc-key="ospsp[s[ps[[sp"
addtime=nov/15/2022 17:19:53 expires-in=17h45m30s

After testing telnet session from my server to the remote server these values changed:
add-lifetime=19h12m8s/1d10s current-bytes=468 current-packets=9
replay=128
-- [Q quit|D dump|C-z pause]

Regarding WAN IP address the IP and Mask are incorrect only here on my post, in the real configuration it's correct.
I posted here not correct addresses to hide sensitive information.

Regarding Note 2:
I have a few rules:
Accept input protocol 50 ipsec-esp in interface wan
Accept input source address my wan address, dst address remote site in interface wan
Accept input
Drop input in interface WAN


My WAN interface is protected by another router with a zone-based firewall.
And there I accepted incoming traffic only from the remote site to my Mikrotik wan interface.
For Outcoming traffic from my WAN IP address, I accepted any traffic.

I tried to disable all rules in IP Filter Rules in my Mikrotik.
But the result is the same, I see 3 x TX packets and 0 RX packets through IPSEC during one telnet session attempt.
 
User avatar
nichky
Forum Guru
Forum Guru
Posts: 1275
Joined: Tue Jun 23, 2015 2:35 pm

Re: Issue with Source NAT over IPSEC

Wed Nov 16, 2022 1:58 am

OK, so there is NAT somewhere between your peers. Nothing in your configuration suggests that, although the WAN configuration is weird - see the note below. So there is probably NAT on the remote end. Nothing bad about that, it just explains why the SAs use UDP-encapsulated ESP rather than bare one, for the cost of a few extra header bytes per packet.
quick one ,in my case i got src-port and dst ports=0, what does it mean?
the tunnel it does work ,also i can see SA
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Issue with Source NAT over IPSEC

Wed Nov 16, 2022 9:49 am

The source nat is in my mikrotik only for NAT of my Server.
/ip firewall nat
add action=src-nat chain=srcnat comment="Source NAT to Remote Site" log=yes \
src-address=10.10.20.20 to-addresses=115.249.109.170

Between LAN and WAN I don't do nat.
I establish from WAN to remote site IPSEC.
I do understand you use this NAT rule for the payload traffic, that's the goal, but it's not what I am talking about.

When IPsec peers negotiate the connection using IKE (or IKEv2), each peer sends its own address and port and the address and port from which it has received the other peer's packet in the IKE or IKEv2 message, and the other peer compares these values to what it can see on its end. If they differ, it means there is NAT at least in one direction, and this makes both peers switch over from using UDP port 500 to using UDP port 4500 and use the same UDP flow they use for IKE (IKEv2) connection also to send the ESP packets encapsulated into UDP ones (because bare ESP has no notion of ports and thus it is not possible to NAT multiple ESP flows to the same remote destination).

So there is NAT somewhere on the path between your Mikrotik and the remote peer, or the remote peer intentionally forces the NAT traversal behavior to avoid problems with bare ESP, as some ISPs handle it incorrectly. The presence of NAT is not the reason of your issue, it just explains why you cannot see bare ESP packets.

After testing telnet session from my server to the remote server these values changed:
add-lifetime=19h12m8s/1d10s current-bytes=468 current-packets=9
replay=128
...
But the result is the same, I see 3 x TX packets and 0 RX packets through IPSEC during one telnet session attempt.
It means everything is correct at Mikrotik side and you have to search for the issue at the remote end.
 
User avatar
Taner
just joined
Topic Author
Posts: 12
Joined: Thu Nov 22, 2018 11:48 am

Re: Issue with Source NAT over IPSEC

Fri Dec 23, 2022 12:01 am

Hi guys.

Sorry for the delayed answer.
My configuration is okay.
The issue was in the other end.

The tunnel with source nat through IPSEC VPN has been established successfully.

Thank you for the attention regarding the topic.
Best Regards

Who is online

Users browsing this forum: ccrsxx, Google [Bot] and 63 guests