Community discussions

MikroTik App
 
User avatar
mdd
newbie
Topic Author
Posts: 46
Joined: Mon Oct 02, 2017 4:25 pm
Location: Klaipeda, Lithuania

P2P IPSEC strange behavere

Wed May 04, 2022 9:09 am

Hi All,

Have strange issue with IPSEC P2P setup. All works for a while and after random time tunnel traffic just not forwarding any more.
The ipsec says connection established but no traffic getting in/out. And if i try disable peers and policy and after re-enable them - traffic starts forwarding, but not everythime.
 
User avatar
mdd
newbie
Topic Author
Posts: 46
Joined: Mon Oct 02, 2017 4:25 pm
Location: Klaipeda, Lithuania

Re: P2P IPSEC strange behavere

Fri May 06, 2022 8:28 am

Any advise at least how to troubleshoot or look for the issues ?
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: P2P IPSEC strange behavere

Sat May 07, 2022 9:36 pm

Are both peers on public IP addresses, i.e. is bare ESP used as transport protocol? If so, does the input chain of the firewall at both devices accept ESP packets?
 
User avatar
memelchenkov
Member Candidate
Member Candidate
Posts: 202
Joined: Sun Oct 11, 2020 12:00 pm
Contact:

Re: P2P IPSEC strange behavere

Sat May 07, 2022 9:40 pm

It may be due wrong PFS. Try 'none' to make sure it is not related.
It may be due to 7.x firmware if you use it, they have some bugs, depends on configuration.
 
User avatar
mdd
newbie
Topic Author
Posts: 46
Joined: Mon Oct 02, 2017 4:25 pm
Location: Klaipeda, Lithuania

Re: P2P IPSEC strange behavere

Mon May 09, 2022 11:34 am

2Sindy - yes they are both on pub IPs. Input chaings are added to accept ESP.
2memelchenkov - select "none" in profiles PFS. I will wait until other side will change this too to see if this helps. No i am using 6.45.9 version, but other side use newer version - this what worries me more as this could be the issue maybe just guessing :(

UPDATED:
Did not helped selecting "none" in profiles PFS :(
 
User avatar
mdd
newbie
Topic Author
Posts: 46
Joined: Mon Oct 02, 2017 4:25 pm
Location: Klaipeda, Lithuania

Re: P2P IPSEC strange behavere

Wed Jun 22, 2022 3:21 pm

ok guys found what the issue was - hw could not coupe with high encryption levels. Now when we change encryption to lover settings it works.
 
User avatar
mdd
newbie
Topic Author
Posts: 46
Joined: Mon Oct 02, 2017 4:25 pm
Location: Klaipeda, Lithuania

Re: P2P IPSEC strange behavere

Tue Aug 09, 2022 3:34 pm

Unfortunately the issue still exis :(

Traffics goes one way (seems to me by counters on rules incress)
RAW.PNG
when pingin remote. Again were working for few days after disabled/enable peers tunnel is up but no traffic passing.
Pinging from 10.254.0.0/24.
Any suggestion how to trace this traffic by logging or else ?
You do not have the required permissions to view the files attached to this post.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: P2P IPSEC strange behavere

Tue Aug 09, 2022 4:32 pm

When this happens, what does /ip ipsec statistics print show at both ends?
 
User avatar
mdd
newbie
Topic Author
Posts: 46
Joined: Mon Oct 02, 2017 4:25 pm
Location: Klaipeda, Lithuania

Re: P2P IPSEC strange behavere

Wed Aug 10, 2022 9:43 am

Have two ipsec tunels setups with clients: (clients side can not check, but they say that all other existing tunnels works fine for them only with mine are problems) Can find what is the issue on my side :(

On my side
> /ip ipsec statistics print
in-errors: 0
in-buffer-errors: 0
in-header-errors: 0
in-no-states: 233
in-state-protocol-errors: 0
in-state-mode-errors: 0
in-state-sequence-errors: 0
in-state-expired: 0
in-state-mismatches: 0
in-state-invalid: 0
in-template-mismatches: 0
in-no-policies: 0
in-policy-blocked: 0
in-policy-errors: 0
out-errors: 0
out-bundle-errors: 0
out-bundle-check-errors: 0
out-no-states: 3012
out-state-protocol-errors: 8
out-state-mode-errors: 0
out-state-sequence-errors: 0
out-state-expired: 8
out-policy-blocked: 0
out-policy-dead: 0
out-policy-errors: 0
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: P2P IPSEC strange behavere

Wed Aug 10, 2022 10:49 am

they say that all other existing tunnels works fine for them only with mine are problems
This bit may be very important - IPsec matches packets to traffic selectors of the individual policies until the first match, so if the traffic selector of your tunnel gets shadowed by a traffic selector of some other tunnel, the traffic the client sends to you may end up at another peer.
On my side
/ip ipsec statistics print
                  in-errors: 0
           in-buffer-errors: 0
           in-header-errors: 0
               in-no-states: 233
   in-state-protocol-errors: 0
       in-state-mode-errors: 0
   in-state-sequence-errors: 0
           in-state-expired: 0
        in-state-mismatches: 0
           in-state-invalid: 0
     in-template-mismatches: 0
             in-no-policies: 0
          in-policy-blocked: 0
           in-policy-errors: 0
                 out-errors: 0
          out-bundle-errors: 0
    out-bundle-check-errors: 0
              out-no-states: 3012
  out-state-protocol-errors: 8
      out-state-mode-errors: 0
  out-state-sequence-errors: 0
          out-state-expired: 8
         out-policy-blocked: 0
            out-policy-dead: 0
          out-policy-errors: 0
It's not the kind of error I was expecting to see, but the non-0 values do indicate something is wrong. When it is broken, what does /ip ipsec installed-sa print show? If you want to obfuscate, it is only necessary to obfuscate the IP addresses, all the keys are ephemeral so the eventual attacker would have to use them while they are still valid. But their actual values would only be interesting if you had data from both devices from the same time (to compare them to see whether the rekeying result was the same at both peers, there used to be a bug in this a few years ago) anyway.

Have you ever had enough patience to wait 30+ minutes after the tunnel broke? If there is a rekeying issue, chances are high that the result of the next rekey will be correct, which would be a confirmation that the issue is related to rekeying and not to something else.

Debugging is hard when you don't have information from both peers. You can run /tool sniffer quick ip-protocol=ipsec-esp and see whether your router is sending anything when you try to generate some traffic towards the peer, and whether your router is receiving anything when the remote side generates some traffic towards you. If something is coming from the remote side, it is likely a rekeying issue (but in such case, the statistics should show non-0 values in another row, I just don't remember which one); if nothing is coming, the remote side may have that policy shadowing issue - or some other one.
 
User avatar
mdd
newbie
Topic Author
Posts: 46
Joined: Mon Oct 02, 2017 4:25 pm
Location: Klaipeda, Lithuania

Re: P2P IPSEC strange behavere

Fri Aug 12, 2022 10:51 am

Hi Sindy,

From that behave and all forums i have read about king of this isues maybe NAT rules affecting the traffic (just presuming). But i have made RAW rules that NAT not be involved in translation. But counters of these rules never grow when no traffic go in/out. Tunnel allways eshablished.
/ip firewall raw print
Flags: X - disabled, I - invalid, D - dynamic 
 0    ;;; iii
    chain=prerouting action=notrack log=no log-prefix="" src-address=10.254.0.0/24_MYLAN dst-address=172.16.3.0/30_CLIENT_LAN 
    chain=prerouting action=notrack log=no log-prefix="" src-address=10.251.0.0/24_MYLAN dst-address=10.14.0.0/16_CLIENT_LAN 
    chain=prerouting action=notrack log=no log-prefix="" src-address=172.16.3.0/30 dst-address=10.254.0.0/24 
    chain=prerouting action=notrack log=no log-prefix="" src-address=10.14.0.0/16 dst-address=10.251.0.0/24 


At the moment no traffic to/from both tunnels.
/ip ipsec installed-sa print
Flags: H - hw-aead, A - AH, E - ESP 
 0 HE spi=0xF0D0A4F src-address=CLIENT_IP dst-address=MY_WANstate=mature auth-algorithm=sha1 
      enc-algorithm=aes-cbc enc-key-size=256 auth-key="1de0" 
      enc-key="e7e23da6a4b2bcb94f17922d3970e1915d5ba3cf49c6f34" add-lifetime=48m/1h 
      replay=128 

 1 HE spi=0x9884EAF6 src-address=MY_WANdst-address=CLIENT_IP state=mature auth-algorithm=sha1 
      enc-algorithm=aes-cbc enc-key-size=256 auth-key="26a66ed094f8ee0ef5cb" 
      enc-key="5eb3259s8f487c3b1661d1c67e9fb46" add-lifetime=48m/1h 
      replay=128 

 2 HE spi=0x637A3BB src-address=CLIENT_IP2 dst-address=MY_WANstate=mature auth-algorithm=sha1 
      enc-algorithm=aes-cbc enc-key-size=128 auth-key="7b89486ad9c61183d0992b54cc8" 
      enc-key="5d0765sc5c3385ee481" add-lifetime=24m/30m replay=128 

 3 HE spi=0x85F90A9 src-address=MY_WANdst-address=CLIENT_IP2 state=mature auth-algorithm=sha1 
      enc-algorithm=aes-cbc enc-key-size=128 auth-key="573608f9ss59b5cddcaee01c13e" 
      enc-key="00bbads27cb33a26" addtime=aug/12/2022 10:25:43 expires-in=16m15s 
      add-lifetime=24m/30m current-bytes=2048 current-packets=40 replay=128 
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: P2P IPSEC strange behavere

Sun Aug 14, 2022 7:03 pm

From that behave and all forums i have read about king of this isues maybe NAT rules affecting the traffic (just presuming).
NAT may be necessary so that IPsec could work or it may prevent it from working, it depends on the use case. But it would require something more to make inappropriate NAT rules cause this random behaviour - for example, routes in the regular routing would have to change because some interface would go down (or up), causing the same traffic to use a different route than usually, and consequently to hit another NAT rule.

counters of these rules never grow when no traffic go in/out. Tunnel allways eshablished.
That's an important observation, it may indicate that the problem is not with IPsec and that the traffic that should be sent through the tunnel actually doesn't reach your Mikrotik at all. But this is only the case if all of your traffic is a response one (i.e. your side doesn't actively send anything to the remote one).

In another words, not enough information to help at the moment. I need a configuration export and some information about the application topology - servers at both ends, servers only on your side, servers only on their side, how frequently the traffic is sent when it works... there may be a firewall at their side not doing NAT but blocking ESP traffic in incoming direction if there was a long enough period of silence.

At the moment no traffic to/from both tunnels.
Actually, there was a small bit of traffic from you to them:
3 HE spi=0x85F90A9 src-address=MY_WANdst-address=CLIENT_IP2 state=mature auth-algorithm=sha1
enc-algorithm=aes-cbc enc-key-size=128 auth-key="573608f9ss59b5cddcaee01c13e"
enc-key="00bbads27cb33a26" addtime=aug/12/2022 10:25:43 expires-in=16m15s
add-lifetime=24m/30m current-bytes=2048 current-packets=40 replay=128
 
User avatar
mdd
newbie
Topic Author
Posts: 46
Joined: Mon Oct 02, 2017 4:25 pm
Location: Klaipeda, Lithuania

Re: P2P IPSEC strange behavere

Tue Aug 16, 2022 4:17 pm

When we do ping on tunnel's end point we are getting only counters increased from our side to remote side on RAW rules.

I have added some of configuration here ips/subnets randomly created here for public view.
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip ipsec profile
add dh-group=modp1024 enc-algorithm=3des name=profile_1
add dh-group=modp1024 enc-algorithm=3des name=profile_2
add dh-group=modp1024 enc-algorithm=aes-128 name=CLIENT1 nat-traversal=no
add dh-group=modp1536 enc-algorithm=aes-256 lifetime=2h10m name=CLIENT2 \
nat-traversal=no
/ip ipsec peer
add address=3.3.3.3/32 local-address=2.2.2.2 name=CLIENT1 profile=CLIENT1 \
send-initial-contact=no
add address=1.1.1.1/32 local-address=2.2.2.2 name=CLEINT2 profile=CLIENT2 \
send-initial-contact=no
add disabled=yes name=peer2 passive=yes profile=profile_2
add disabled=yes name=peer1 passive=yes profile=profile_1
/ip ipsec proposal
set [ find default=yes ] disabled=yes enc-algorithms=3des pfs-group=none
add enc-algorithms=aes-128-cbc name=CLIENT1
add enc-algorithms=aes-256-cbc lifetime=1h name=CLEINT2 pfs-group=modp1536
/ip dhcp-client
add disabled=no interface=ether1
/ip firewall filter
add action=accept chain=input comment="ALLOW IPSEC" protocol=ipsec-esp
add action=accept chain=input comment="ALLOW IPSEC" protocol=ipsec-ah
add action=accept chain=input comment="ALLOW IPSEC" dst-port=500 log=yes \
protocol=udp
add action=accept chain=input comment="ALLOW IPSEC" dst-port=4500 protocol=\
udp
add action=accept chain=input dst-port=1701 protocol=udp
add action=accept chain=input comment="ALLOW IPSEC L2TP" dst-port=1723 \
protocol=tcp
add action=accept chain=input comment="ALLOW GRE" protocol=gre
add action=accept chain=input comment="ALLOW PING" protocol=icmp
add action=accept chain=input comment="ALLOW CLIENT1" dst-address=\
10.254.0.0/24 src-address=172.16.3.0/30
add action=accept chain=forward dst-address=172.16.3.0/30 src-address=\
10.254.0.0/24
add action=accept chain=forward dst-address=10.254.0.0/24 src-address=\
172.16.3.0/30
add action=accept chain=input comment="ALLOW CLIENT2" dst-address=\
10.251.0.0/24 src-address=10.14.0.0/16
add action=accept chain=forward dst-address=10.251.0.0/24 src-address=\
10.14.0.0/16
add action=accept chain=input comment="ALLOW SSTP" dst-port=443 protocol=tcp
add action=accept chain=input comment="ALLOW trafic from E1 (established)" \
connection-state=established in-interface=ether1
add action=drop chain=input comment="DROP trafic from E1 (all)" in-interface=\
ether1
add action=drop chain=input comment="DROP trafic from PPP (all)" \
in-interface=all-ppp
/ip firewall nat
add action=accept chain=srcnat comment="CLIENT1 " dst-address=172.16.3.0/30 \
src-address=10.254.0.0/24
add action=accept chain=srcnat comment="CLIENT1 " dst-address=10.254.0.0/24 \
src-address=172.16.3.0/30
add action=accept chain=srcnat comment=CLIENT2 dst-address=10.251.0.0/24 \
src-address=10.14.0.0/16
add action=accept chain=srcnat comment=CLIENT2 dst-address=10.14.0.0/16 \
src-address=10.251.0.0/24
add action=src-nat chain=srcnat comment=COMMENTS out-interface=ether1 \
src-address=192.168.111.0/24 to-addresses=2.2.2.2
add action=masquerade chain=srcnat comment=GW out-interface=ether1 \
src-address=!10.240.240.250
add action=masquerade chain=srcnat comment=COMMENTS dst-address=10.240.0.0/20
add action=masquerade chain=srcnat comment=COMMENTS dst-address=\
10.240.18.0/23
add action=masquerade chain=srcnat comment=COMMENTS dst-address=\
10.240.16.0/24 src-address=192.168.5.0/24
add action=masquerade chain=srcnat comment=COMMENTS dst-address=\
10.240.16.0/24 src-address=192.168.99.0/24
add action=masquerade chain=srcnat comment=COMMENTS dst-address=\
10.240.16.0/24 src-address=10.10.10.0/24
add action=masquerade chain=srcnat comment=COMMENTS dst-address=\
172.16.55.0/24 src-address=10.10.10.0/24
add action=masquerade chain=srcnat comment=COMMENTS dst-address=\
172.16.66.0/24
add action=masquerade chain=srcnat comment=COMMENTS dst-address=10.254.0.0/16 \
src-address=10.10.10.14
/ip firewall raw
add action=notrack chain=prerouting comment=CLIENT1 dst-address=172.16.3.0/30 \
src-address=10.254.0.0/24
add action=notrack chain=prerouting dst-address=10.254.0.0/24 src-address=\
172.16.3.0/30
add action=notrack chain=prerouting comment=CLIENT2 dst-address=10.14.0.0/16 \
src-address=10.251.0.0/24
add action=notrack chain=prerouting dst-address=10.251.0.0/24 src-address=\
10.14.0.0/16
/ip ipsec identity
# Suggestion to use stronger pre-shared key or different authentication method
add peer=CLEINT2 secret=secret
# Suggestion to use stronger pre-shared key or different authentication method
add peer=CLIENT1 secret=secret
add generate-policy=port-override peer=peer2 remote-id=ignore secret=\
verysecret
# Suggestion to use stronger pre-shared key or different authentication method
add generate-policy=port-override peer=peer1 remote-id=ignore secret=test
/ip ipsec policy
set 0 disabled=yes
add dst-address=172.16.3.0/30 level=unique peer=CLIENT1 proposal=CLIENT1 \
src-address=10.254.0.0/24 tunnel=yes
add dst-address=10.14.0.0/16 level=unique peer=CLEINT2 proposal=CLEINT2 \
src-address=10.251.0.0/24 tunnel=yes
/ip route
add comment="comment" distance=15 gateway=99.99.99.99 \
pref-src=99.99.99.91 routing-mark=DSL1
add comment="comment" distance=16 gateway=88.88.88.88 \
pref-src=88.88.88.81 routing-mark=DSL3

/ip route rule
add action=lookup-only-in-table dst-address=0.0.0.0/0 src-address=\
192.168.99.252/32 table=DSL1
add action=lookup-only-in-table dst-address=0.0.0.0/0 src-address=\
192.168.77.23/32 table=DSL
add action=lookup-only-in-table dst-address=0.0.0.0/0 src-address=\
10.240.14.24/32 table=DSL
add action=lookup-only-in-table dst-address=0.0.0.0/0 src-address=\
10.240.14.21/32 table=DSL
add action=lookup-only-in-table dst-address=0.0.0.0/0 src-address=\
10.240.14.202/32 table=DSL
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: P2P IPSEC strange behavere

Tue Aug 16, 2022 6:07 pm

If the pinging causes the corresponding raw rules to count, it means the traffic does reach your Mikrotik.

If the the corresponding installed-sa (from your public address to client's one) also counts as you ping, the IPsec policy matching also does its job.

But there is one more possibility, I can see you have a multi-WAN arrangement. What can happen, and I admit it requires multiple factors to coincide:
  • the payload traffic from both sides ceases for a certain period of time (10 minutes by default) so the ESP connection disappears from connection tracking
  • the primary WAN fails
  • some payload from your side is sent while the primary WAN is down, so an ESP transport packet is sent, but since the primary WAN is down and the tracked connection doesn't exist, the new tracked connection created by the ESP packet gets src-nated to the address of the backup WAN; if payload packets keep coming, this connection with the src-nat handling active keeps getting refreshed, so the ESP packets keep arriving to the destination with an unexpected source address even if the primary WAN recovers. As Mikrotik doesn't support MOBIKE, the recipient ignores them rather than redirecting the SA to the new address.
You can use /tool sniffer ip-protocol=ipsec-esp ip-address=ip.of.the.client to double-check through which interface and with what source address the transport ESP packets are leaving when you ping.
 
User avatar
mdd
newbie
Topic Author
Posts: 46
Joined: Mon Oct 02, 2017 4:25 pm
Location: Klaipeda, Lithuania

Re: P2P IPSEC strange behavere

Wed Aug 17, 2022 1:42 pm

If the pinging causes the corresponding raw rules to count, it means the traffic does reach your Mikrotik.
Ping echo packets are send from my Router, but never gets back. Just to clear.
If the the corresponding installed-sa (from your public address to client's one) also counts as you ping, the IPsec policy matching also does its job.
When pingins SA counters bytes are incresing only from my side too
But there is one more possibility, I can see you have a multi-WAN arrangement. What can happen, and I admit it requires multiple factors to coincide:
  • the payload traffic from both sides ceases for a certain period of time (10 minutes by default) so the ESP connection disappears from connection tracking
    How to check this one ???
  • the primary WAN fails

    Monitoring all the time. DSL comments are just left overs, all now opic fibre connections. Never failed so far. Backups are just for fail over in case. For IPSEC we use only one like this
    IPSEC P2P OUTSIDE:1.1.1.3/30
    Main route 0.0.0.0 1.1.1.1 / distance 1/ pref source 1.1.1.3
  • some payload from your side is sent while the primary WAN is down, so an ESP transport packet is sent, but since the primary WAN is down and the tracked connection doesn't exist, the new tracked connection created by the ESP packet gets src-nated to the address of the backup WAN; if payload packets keep coming, this connection with the src-nat handling active keeps getting refreshed, so the ESP packets keep arriving to the destination with an unexpected source address even if the primary WAN recovers. As Mikrotik doesn't support MOBIKE, the recipient ignores them rather than redirecting the SA to the new address.
You can use /tool sniffer ip-protocol=ipsec-esp ip-address=ip.of.the.client to double-check through which interface and with what source address the transport ESP packets are leaving when you ping.

I have tried to capture, but none packets captured even from IP romete or local IP, or fithout filter :( What this could be. Tried to to flush, to disable and enable IPSEC peers but no packets were captured. Connections still showing establised. ????
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: P2P IPSEC strange behavere

Wed Aug 17, 2022 1:54 pm

I have tried to capture, but none packets captured even from IP romete or local IP, or fithout filter :( What this could be. Tried to to flush, to disable and enable IPSEC peers but no packets were captured. Connections still showing establised. ????
What was the exact command line you used for sniffing (substitute just the public IP addresses if you use them on that command line)?
 
User avatar
mdd
newbie
Topic Author
Posts: 46
Joined: Mon Oct 02, 2017 4:25 pm
Location: Klaipeda, Lithuania

Re: P2P IPSEC strange behavere

Thu Aug 18, 2022 2:12 pm

I have tried to capture, but none packets captured even from IP romete or local IP, or fithout filter :( What this could be. Tried to to flush, to disable and enable IPSEC peers but no packets were captured. Connections still showing establised. ????
What was the exact command line you used for sniffing (substitute just the public IP addresses if you use them on that command line)?
/tool sniffer
set filter-ip-protocol=ipsec-eso
set filter-ip-address=.163 or .177
.163 is MY GW
.177 is CLIENTS GW

Ok seems that i was too tired. Now packets have been captured. And seems that ESP packets moving. But again looks like one way... for my while pinging from my tunnel subnet.
caps_d.png
And opened packets interface/public ips are right?
caps_d.png
You do not have the required permissions to view the files attached to this post.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: P2P IPSEC strange behavere

Thu Aug 18, 2022 2:32 pm

I am confused. The packet dissection (bottommost picture) shows a packet from the client to your Mikrotik (src-mac-address Cisco, dst-mac-address Mikrotik). That would mean that the ESP traffic is only coming from the client to your Mikrotik as no ESP sent from your address. In such case, it would mean that the ESP traffic from them is not induced by your pinging, and that your Mikrotik ignores it.
 
User avatar
mdd
newbie
Topic Author
Posts: 46
Joined: Mon Oct 02, 2017 4:25 pm
Location: Klaipeda, Lithuania

Re: P2P IPSEC strange behavere

Fri Aug 19, 2022 10:21 am

I am confused. The packet dissection (bottommost picture) shows a packet from the client to your Mikrotik (src-mac-address Cisco, dst-mac-address Mikrotik). That would mean that the ESP traffic is only coming from the client to your Mikrotik as no ESP sent from your address. In such case, it would mean that the ESP traffic from them is not induced by your pinging, and that your Mikrotik ignores it.
What are option to check this ? As for were too strange as i am pinging but no traffic then send out ? Maybe firewall rules stops ?
 
User avatar
mdd
newbie
Topic Author
Posts: 46
Joined: Mon Oct 02, 2017 4:25 pm
Location: Klaipeda, Lithuania

Re: P2P IPSEC strange behavere

Fri Aug 19, 2022 11:23 am

cap_ipsec.PNG
One more strange behave ether1 GW has few public IP on it. 162. 163 . 16X. For multiple IPSEC tunnel ends i am trying to use just one .163.
For some reason one of IPSEC client .177 using .163 MY GW has established connection and tunnel traffic goes now on both sides // not sure why it started working now ???
But for other .75 client , MY GW trying establish IPSEC tunnel using not .163, but .162 IP address? I do not have any routing pref setup for this behave.
.
You do not have the required permissions to view the files attached to this post.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: P2P IPSEC strange behavere

Fri Aug 19, 2022 6:38 pm

So now you know why you should have done the obfuscation the way I suggest in my automatic signature rather than blindly remove the IP addresses completely.

It seems that what I've suggested as an explanation may be true, except that the trigger event is not a WAN failover but simply a long enough pause in traffic, after which the ESP starts being sent again and the action=masquerade chain=srcnat comment=GW out-interface=ether1 src-address=!10.240.240.250 rule changes the source address to the .162 one. Or the IPsec stack is confused, hard to say.

What does /ip firewall connection print detail where dst-address~"ip.of.the.client" show in this state? If it shows src-address=x.x.x.163 and reply-dst-address=x.x.x.162, the ISP stack behaves correctly and we have to prevent the action=masquerade rule from acting on ESP by adding protocol=!ipsec-esp to it; if already the src-address is x.x.x.162, it means that the IPsec stack has chosen a different IP to send the ESP after the pause; in that case, you would have to place a chain=srcnat protocol=ipsec-esp dst-address=ip.of.the.client action=src-nat to-addresses=x.x.x.163 before the first action=masquerade one.
 
User avatar
mdd
newbie
Topic Author
Posts: 46
Joined: Mon Oct 02, 2017 4:25 pm
Location: Klaipeda, Lithuania

Re: P2P IPSEC strange behavere

Mon Aug 29, 2022 9:16 am

Hi Sindy,

Still not luck.. one of IPSEC tunnel works at the moment, but other one still not Working one use the right gw ip .163.
Other non working /ip firewall print still show ip with .162 for some reason. Same thing i can see in captured packets.
cap_not_right_ip.PNG
You do not have the required permissions to view the files attached to this post.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: P2P IPSEC strange behavere

Mon Aug 29, 2022 12:26 pm

This /ip firewall connection print ... shows that the IPsec stack has bound the connection properly to .163 but the firewall has src-nated it to .162.

So do prevent the masquerade rule from acting on ESP by adding a protocol=!ipsec-esp match condition to it as I've suggested in my previous post (it is not a 100% clean solution but good enough for this particular case and I don't want to overload you with details at this stage). Since rules in the nat table only act on the first packet of each connection, you have to remove the existing src-nated ESP connection from the conntrack data after the modification:
/ip firewall connection remove [find where srcnat and protocol=ipsec-esp]
The next packet will re-create the tracked connection, but without the src-nat.
 
User avatar
mdd
newbie
Topic Author
Posts: 46
Joined: Mon Oct 02, 2017 4:25 pm
Location: Klaipeda, Lithuania

Re: P2P IPSEC strange behavere

Mon Aug 29, 2022 1:50 pm

This /ip firewall connection print ... shows that the IPsec stack has bound the connection properly to .163 but the firewall has src-nated it to .162.

So do prevent the masquerade rule from acting on ESP by adding a protocol=!ipsec-esp match condition to it as I've suggested in my previous post (it is not a 100% clean solution but good enough for this particular case and I don't want to overload you with details at this stage).

Where exactly to add this as i am bit now confused. If one of the IPSEC tunnel works, and other dont - how this will affect working one?

Since rules in the nat table only act on the first packet of each connection, you have to remove the existing src-nated ESP connection from the conntrack data after the modification:
/ip firewall connection remove [find where srcnat and protocol=ipsec-esp]

Tried to /ip firewall connection find [where srcnat and protocol=ipsec-esp] and no connections appeared in the search.

The next packet will re-create the tracked connection, but without the src-nat.
 
User avatar
mdd
newbie
Topic Author
Posts: 46
Joined: Mon Oct 02, 2017 4:25 pm
Location: Klaipeda, Lithuania

Re: P2P IPSEC strange behavere

Mon Aug 29, 2022 2:21 pm

Ok. Some how got working both tunnels now. I have added protocol=!ipsec-esp to my main GW .163
/ip firewall nat
add action=masquerade chain=srcnat comment=GW out-interface=ether1 \
src-address=!10.240.240.250 protocol=!ipsec-esp

For the moment seems working. Not 100% sure why ...
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: P2P IPSEC strange behavere

Mon Aug 29, 2022 4:17 pm

Where exactly to add this as i am bit now confused. If one of the IPSEC tunnel works, and other dont - how this will affect working one?
At worst it would cause a loss of several packets of the working one, until the Mikrotik would send a packet towards the client.

Tried to /ip firewall connection find [where srcnat and protocol=ipsec-esp] and no connections appeared in the search.

There are syntactic errors in this command, so if you've entered it exactly this way, no wonder it has shown nothing.
 
User avatar
mdd
newbie
Topic Author
Posts: 46
Joined: Mon Oct 02, 2017 4:25 pm
Location: Klaipeda, Lithuania

Re: P2P IPSEC strange behavere

Mon Aug 29, 2022 4:23 pm

Where exactly to add this as i am bit now confused. If one of the IPSEC tunnel works, and other dont - how this will affect working one?
At worst it would cause a loss of several packets of the working one, until the Mikrotik would send a packet towards the client.
Thats not problem :)

Tried to /ip firewall connection find [where srcnat and protocol=ipsec-esp] and no connections appeared in the search.

There are syntactic errors in this command, so if you've entered it exactly this way, no wonder it has shown nothing.

Tried to adapt to see what i will be removing with this command line:

/ip firewall connection remove [find where srcnat and protocol=ipsec-esp]
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: P2P IPSEC strange behavere

Mon Aug 29, 2022 4:41 pm

Tried to adapt to see what i will be removing with this command line:
So use /ip firewall connection print detail where srcnat and protocol=ipsec-esp .

But as it is working now, chances are high that this command will also show nothing because the NATed connection has expired in the meantime due to lack of any traffic updating it. To double check, use /ip firewall connection print detail where !srcnat and protocol=ipsec-esp , you should see the working connections.
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 2989
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: P2P IPSEC strange behavere

Mon Aug 29, 2022 5:23 pm

take a look on

ip -> ipsec -> installed SAs

to track ipsec status
 
User avatar
mdd
newbie
Topic Author
Posts: 46
Joined: Mon Oct 02, 2017 4:25 pm
Location: Klaipeda, Lithuania

Re: P2P IPSEC strange behavere

Tue Aug 30, 2022 9:42 am

Tried to adapt to see what i will be removing with this command line:
So use /ip firewall connection print detail where srcnat and protocol=ipsec-esp .

But as it is working now, chances are high that this command will also show nothing because the NATed connection has expired in the meantime due to lack of any traffic updating it.
Yes correct i do not see anything.

To double check, use /ip firewall connection print detail where !srcnat and protocol=ipsec-esp , you should see the working connections.

Yes now i see traffic when pinging other side. Complete esp packets :)
Sindy could you explain again for me simply stupid what did fixed this my issue:
That i removed all srn-nated connections ? Or that i have added this rule to my GW ?

/ip firewall nat
add action=masquerade chain=srcnat comment=GW out-interface=ether1 \
src-address=!10.240.240.250 protocol=!ipsec-esp
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: P2P IPSEC strange behavere  [SOLVED]

Tue Aug 30, 2022 10:32 am

what did fixed this my issue:
That i removed all srn-nated connections ? Or that i have added this rule to my GW ?
The modification of the masquerade rule was necessary to prevent newly created ESP connections from getting src-nated.
The removal of already existing src-nated ESP connections may have been necessary, or may have been not.

The thing is that there is the actual ESP connection as handled by the IPsec stack, and there is the understanding of that connection as handled by the firewall. The IPsec stack doesn't know anything about the existence of the firewall, and the firewall has no other information about the actual connection but observation of its packets flowing through it. So if there is a long-term silence (more than 10 minutes by default) in both directions, the firewall forgets about the connection. Once the IPsec stack receives a payload packet from LAN and sends an ESP one carrying it, the firewall cannot find any connection matching that packet, so it creates a new one. Before the modification, this new connection got src-nated, so the remote peer ignored the ESP packet because it was coming from an unexpected address.

And it sometimes worked because in some cases, the first ESP packet after the silence came from the peer, not from the local IPsec stack; in that case, the firewall has created a connection without any NAT.
 
User avatar
mdd
newbie
Topic Author
Posts: 46
Joined: Mon Oct 02, 2017 4:25 pm
Location: Klaipeda, Lithuania

Re: P2P IPSEC strange behavere

Wed Oct 12, 2022 11:22 am

Thank you Sindy help me to get some advance knowleague.

Who is online

Users browsing this forum: GoogleOther [Bot], lurker888, tangent and 59 guests