IPSec failed to pre-process ph2 packet

avdvyver01 · Fri Mar 29, 2019 11:31 am

Hi,

Two days ago I configured 4 x GRE / IPSec tunnels on my CCR running 6.42.12. I use this exact same configuration elsewhere successfully on a CCR running 6.42.6. All 4 tunnels were up and stable and BGP neighbours connected and exchanging routes as expected.

Yesterday morning I noticed that the one tunnel is down. Log indicate ph2 cannot establish and the log is flooded with “ipsec failed to pre-process ph2 packet”. The policy for the tunnel was marked in red (I recall this was usually an indication that the policy was invalid).

Anyways I went through the process of clearing all SAs, enabling and disabling the peer, the policy, the associated addresses etc. multiple times without the ph2 re-establishing. Resetting the tunnel from the far end also had no effect. I deleted the peer and policy and recreated with the exact same result. I double checked all configs making 100% sure there were no overlapping subnets but I could not find any issue. None of the other 3 tunnels showed the same behavior.

At some stage I left the policy disabled for quite a while (guessing > 30 mins). After enabling it, to my surprise, the ph2 established. I was just wondering if this is perhaps know behavior that was introduced somewhere between 6.42.6 and 6.42.12? Or any other thoughts?

It still seems stable ever since.

Thanks

kmansoft · Fri Mar 29, 2019 2:35 pm

I've been seeing this (apparent data corruption in IPSec packets when changing configuration) on my home ac2.

It's fixed in 6.43 or 6.44 - can't remember exactly - but it was mentioned in change logs.

avdvyver01 · Tue Apr 02, 2019 10:19 am

Thanks for the reply. I have read the release notes and though the versions there are a substantial amount of IPSec and IKE changes. I guess I will just test version by version until I get it fixed. Part of the issue here is that it seems that when the tunnel gets interrupted, it does not correctly re-establish ph2. Enabling and disabling the peer or policy does not recreate the issue but if you actually simulate an interruption (dropping traffic with firewalling) ph2 does not re-establish. Anyway, will update my findings here.

avdvyver01 · Mon Apr 08, 2019 2:00 pm

Does anyone here perhaps have any specific information on why the ph2 policies would out of the blue go into an invalid state? This happens randomly and I cannot reproduce on demand so trying to roll forward version by version is going take forever.

Like I mentioned before, disabling the policy for an extended amount of time and then enabling it seems to resolve the problem. The problem is not on the device at the far end as I have other routers terminating tunnels with the same configuration on the same device that never show this behavior.

https://imgur.com/bs9ynhA

Log:

Apr/08/2019 12:03:23 ipsec,error 32.56.77.82 peer sent packet for dead phase2
Apr/08/2019 12:03:23 ipsec,error 32.56.77.82 peer sent packet for dead phase2
Apr/08/2019 12:03:37 ipsec,error 32.56.77.82 peer sent packet for dead phase2
Apr/08/2019 12:03:51 ipsec,info purging ISAKMP-SA 197.45.67.3[500]<=>32.56.77.82[500] spi=411d519e3dff9ebb:8bbc9dbda5d18093.
Apr/08/2019 12:03:51 ipsec,info ISAKMP-SA deleted 197.45.67.3[500]-32.56.77.82[500] spi:411d519e3dff9ebb:8bbc9dbda5d18093 rekey:1
Apr/08/2019 12:03:51 ipsec,info respond new phase 1 (Identity Protection): 197.45.67.3[500]<=>32.56.77.82[500]
Apr/08/2019 12:03:52 ipsec,info ISAKMP-SA established 197.45.67.3[500]-32.56.77.82[500] spi:a4065316fd2443e0:99be3dee19e7e38d
Apr/08/2019 12:03:52 ipsec,error 32.56.77.82 failed to pre-process ph2 packet.
Apr/08/2019 12:04:02 ipsec,error 32.56.77.82 peer sent packet for dead phase2
Apr/08/2019 12:04:11 ipsec,info ISAKMP-SA deleted 197.45.67.3[500]-32.56.77.82[500] spi:a4065316fd2443e0:99be3dee19e7e38d rekey:1
Apr/08/2019 12:05:03 ipsec,info respond new phase 1 (Identity Protection): 197.45.67.3[500]<=>32.56.77.82[500]
Apr/08/2019 12:05:03 ipsec,error no suitable proposal found.
Apr/08/2019 12:05:03 ipsec,error 32.56.77.82 failed to get valid proposal.
Apr/08/2019 12:05:03 ipsec,error 32.56.77.82 failed to pre-process ph1 packet (side: 1, status 1).
Apr/08/2019 12:05:03 ipsec,error 32.56.77.82 phase1 negotiation failed.
Apr/08/2019 12:05:12 ipsec,info respond new phase 1 (Identity Protection): 197.45.67.3[500]<=>32.56.77.82[500]
Apr/08/2019 12:05:12 ipsec,error no suitable proposal found.
Apr/08/2019 12:05:12 ipsec,error 32.56.77.82 failed to get valid proposal.
Apr/08/2019 12:05:12 ipsec,error 32.56.77.82 failed to pre-process ph1 packet (side: 1, status 1).
Apr/08/2019 12:05:12 ipsec,error 32.56.77.82 phase1 negotiation failed.
Apr/08/2019 12:05:32 ipsec,info respond new phase 1 (Identity Protection): 197.45.67.3[500]<=>32.56.77.82[500]
Apr/08/2019 12:05:32 ipsec,error no suitable proposal found.
Apr/08/2019 12:05:32 ipsec,error 32.56.77.82 failed to get valid proposal.
Apr/08/2019 12:05:32 ipsec,error 32.56.77.82 failed to pre-process ph1 packet (side: 1, status 1).
Apr/08/2019 12:05:32 ipsec,error 32.56.77.82 phase1 negotiation failed.
Apr/08/2019 12:06:12 ipsec,info respond new phase 1 (Identity Protection): 197.45.67.3[500]<=>32.56.77.82[500]

sindy · Sat Apr 13, 2019 7:48 pm

As removing and re-creating the peers and policies didn't help while "letting everything cool down for a while" did, I'd suspect some connection tracking issue somewhere, possibly in the network between the two devices, where an existing connection had to time out and disappear from the connection tracking tables in order to allow the peers to establish communication again. However, respond new phase 1 (Identity Protection) followed by no suitable proposal found suggest something more complex, like multiple local side peers being configured, with different Phase 1 proposals (aka peer profiles), and the wrong one to be hit by the incoming packet.

If you want a more detailed response, follow the hint in my automatic signature, configurations from both ends are necessary.

The phase 2 issue existed only for IKEv2 and it only happened once in many renegotiations while both ends showed everything to be just fine except that the data did not get through because the encryption keys differed between the ends. So I think your case is completely unrelated to that issue (and I don't remember in which version it has been fixed a year ago or so although it was me who has reported it).

avdvyver01 · Tue Apr 16, 2019 9:23 am

Hi sindy,

Thank you for your response.

I have another Mikrotik router in another location that terminates GRE/IPSec tunnels on the exact same device on the far end (VyOS running StrongSwan) and I have never seen this behavior. As a test I downgraded the ROS version of the "problematic" router to 6.42.6. 3 days in the issue occurred again - one policy marked as invalid. Without any intervention, 6 hours later it recovered. I think you are correct that it might be due to connection tracking somewhere. The strange thing is that the far end indicates ph1 and ph2 up. Resetting the tunnels from the far side has not effect. The only thing that I can thing of that is different is that the connection over which the policies change to invalid states, is via a PPPoE internet connection. In other locations where this configurations successfully used, the internet connections are direct fibre connections. What are your thoughts on this? One thing I have not tried yet is actually disabling and re-enabling the PPPoE client session to see if that makes a difference (previously I just tried to remove the relevant UDP 500 and ESP connections from connection tracking). Btw, connection tracking on my Mikrotik routers are configured as "auto".

One other thing that you have mentioned are the proposals. Even though all the IPSec tunnels on the router in question are configured to use the same proposal (add enc-algorithms=aes-128-cbc lifetime=1h name=vpn-core) there is another policy (the default one) used for inbound L2TP over IPSec connections from remote users (find default=yes ] enc-algorithms=aes-256-cbc pfs-group=modp2048) . If this policy is somehow used, then I can understand that there will be an issue but I cannot see how that would happen?

Again, you views would be greatly appreciated.

Config below:

Mikrotik side:

/ip address
add address=169.254.200.65 comment="GRE to vr2.ew1b" interface=gre-vr2.ew1b network=169.254.200.65
add address=169.254.100.245/30 interface=gre-vr2.ew1b network=169.254.100.244

/interface gre
add allow-fast-path=no keepalive=5s,3 local-address=169.254.200.65 name=gre-vr2.ew1b remote-address=169.254.200.66

/ip ipsec peer
add address=32.56.77.82/32 dh-group=modp1024 dpd-interval=10s dpd-maximum-failures=3 enc-algorithm=aes-128 lifetime=8h local-address=197.45.67.3 nat-traversal=no secret=secret-here

/ip ipsec policy
add dst-address=169.254.200.66/32 proposal=vpn-core protocol=gre sa-dst-address=32.56.77.82 sa-src-address=197.45.67.3 src-address=169.254.200.65/32 tunnel=yes

/ip ipsec proposal
add enc-algorithms=aes-128-cbc lifetime=1h name=vpn-core

VyOS side:

interfaces {
    dummy dum1 {
        address 169.254.200.66/32
    }
	tunnel tun1 {
        address 169.254.100.246/30
        description "BGP via GRE"
        encapsulation gre
        local-ip 169.254.200.66
        multicast disable
        remote-ip 169.254.200.65
    }
}

vpn {
    ipsec {
        esp-group AWS {
            compression disable
            lifetime 3600
            mode tunnel
            pfs enable
            proposal 1 {
                encryption aes128
                hash sha1
            }
        }
        ike-group AWS {
            dead-peer-detection {
                action restart
                interval 15
                timeout 30
            }
            ikev2-reauth no
            key-exchange ikev1
            lifetime 28800
            proposal 1 {
                dh-group 2
                encryption aes128
                hash sha1
            }
        }
        ipsec-interfaces {
            interface eth0
        }
        site-to-site {
            peer 197.45.67.3 {
                authentication {
                    mode pre-shared-secret
                    pre-shared-secret ****************
                }
                connection-type initiate
                default-esp-group AWS
                ike-group AWS
                ikev2-reauth inherit
                local-address 10.12.2.9
                tunnel 1 {
                    allow-nat-networks disable
                    allow-public-networks disable
                    local {
                        prefix 169.254.200.66/32
                    }
                    protocol gre
                    remote {
                        prefix 169.254.200.65/32
                    }
                }
            }
	}

avdvyver01 · Wed Apr 17, 2019 10:33 am

When I checked this morning, one policy marked as invalid once more.

Enabling and disabling the PPPOE had not effect.

Changing the crypto to the same settings as the ones used in the default policy had not effect.

The device on the far side indicated both ph1 and ph2 up. Bizarrely even though the policy on the Mikrotik is marked as invalid and clearly indicated as not established, the GRE interface is in a running state and the BGP neighbor connected. So this seems to me that even tough the policy is marked as invalid in the UI as well as the terminal, somewhere, somehow a valid policy is still installed. When I enable and disable the peer, the ph1 reestablished immediately but ph2 does not reestablish and obviously the BGP neighbor now looses comms as the GRE is not in a running state. No matter what I do, I cannot force the policy into a valid state again - after x hours, it just magically recovers. Diagnostic logs indicate: ipsec,debug no policy found for id:11473.

I quite urgently need to get this resolved - again, any assistance would be greatly appreciated.

Below the invalid policy with traffic flowing correctly as if the tunnel is working 100% (prior to resetting the peer):

pol invalid 7.JPG

sindy · Wed Apr 17, 2019 11:44 am

What makes my brain slide a bit is that you use the IP address attached to the GRE interface also as a local address to send the GRE transport packets from. Could you, just for testing, create an /interface bridge without any member ports and attach to it the IP address which is used as local-address of /interface gre (and thus the src-address of the /ip ipsec policy) and use a different address for the GRE interface itself (I don't mind which one of the two will remain the current one and for which purpose you'll use a new one)? I have no idea how this would be done at VyOS side but it should be possible as well.

Also, I'm nor sure why you've assigned a /32 address to one GRE tunnel and a /30 address to another one.

As you seem not to mind publishing your public IP addresses, can you post the log which says "no policy found"? There should be more information in the log than just this.

Please show me also the output of /ip ipsec policy print and /ip ipsec installed-sa print when it runs but the policy is marked as inactive. The encryption and authentication keys shown in the installed-sa printouts are temporary ones so no need to edit them out, it is enough to wait up to 30 minutes before posting them.

avdvyver01 · Wed Apr 17, 2019 12:53 pm

Once more thank you for the reply. Below the output of the peer and sa status. I have also attached the log, obfuscated the IPs as per the config provided and marked non-relevant IPs as x.x.x.x.

pol invalid 12.JPG

pol invalid 11.JPG

pol invalid 9.JPG

The reason for the "odd" GRE configuration stems from VyOS documentation where it is recommended to use these /32 loopback IPs to match the IPSec policies on. The a /30 network is used as a link network and also functions as the BGP peer IPs on both sides of the tunnel. I agree that my implementation on the Mikrotik side can probable be simplified or improved which I will try as per your suggestion.

sindy · Wed Apr 17, 2019 1:39 pm

First of all, all occurrences of ipsec policy not found in the log you've sent are preceded by a message ipsec searching for policy for selector: 169.254.200.49 ip-proto:47 <=> 169.254.200.50 ip-proto:47 which doesn't match the parameters of the /ip ipsec policy you've shown in the quotation from your config in post #6 (dst-address=169.254.200.66/32 src-address=169.254.200.65/32). Post #9 shows the policy matching the log, not the configuration. Is this because you've split the IPs as I've asked you before?

Second, showing just parts of the configuration and status outputs breaks the purpose, my intention was to check the existing policies (including dynamically created ones) for shadowing each other, which you've ruined by showing just the single policy and SA. So please post the complete printout as requested - as text, not as screenshot. You can always direct the output of a print to a file by adding file=some-name, download the file and edit it before posting to hide the IPs, but such editing has to preserve consistency - the same IP address must be substituted by the same replacement string in the whole file, and the traffic selector addresses of the policies (src-address and dst-address) must remain completely unchanged to show the eventual shadowing of policies by preceding ones.

avdvyver01 · Wed Apr 17, 2019 4:35 pm

Hi, on your first question - no, as you can see in post #7, (dst-address=169.254.200.66/32 src-address=169.254.200.65/32) is another policy used by another tunnel. This tunnel also shows the same sporadic behaviour. The config for it was supplied in this thread as use the problematic tunnel as an example when the policies go into an invalid state - I cannot predict when this happens and I supplied the information that was available to me at the time.

I have subsequently moved the source address of the GRE tunnel to a bridge interface as per your suggestion.

My apologies that the configs were not supplied to your satisfaction. I have added it below as per your request. I have also substituted all the public addresses consistently.

Policies:

# apr/17/2019 14:55: 1 by RouterOS 6.42.6
# software id = PFRD-4CSN
#
Flags: T - template, X - disabled, D - dynamic, I - invalid, A - active, 
* - default 
 0 T * group=default src-address=::/0 dst-address=::/0 protocol=all 
       proposal=default template=yes 

 1  DA  src-address=x.x.x.x/32 src-port=any dst-address=x.x.x.x/32 
       dst-port=any protocol=udp action=encrypt level=unique 
       ipsec-protocols=esp tunnel=no proposal=default ph2-count=1 

 2  DA  src-address=x.x.x.x/32 src-port=any dst-address=x.x.x.x/32 
       dst-port=any protocol=udp action=encrypt level=unique 
       ipsec-protocols=esp tunnel=no proposal=default ph2-count=1 

 3  DA  src-address=x.x.x.x/32 src-port=any dst-address=x.x.x.x/32 
       dst-port=any protocol=udp action=encrypt level=unique 
       ipsec-protocols=esp tunnel=no proposal=default ph2-count=1 

 4  DA  src-address=x.x.x.x/32 src-port=any 
       dst-address=x.x.x.x/32 dst-port=any protocol=udp 
       action=encrypt level=unique ipsec-protocols=esp tunnel=no 
       proposal=default ph2-count=1 

 5  DA  src-address=x.x.x.x/32 src-port=any dst-address=x.x.x.x/32 
       dst-port=any protocol=udp action=encrypt level=unique 
       ipsec-protocols=esp tunnel=no proposal=default ph2-count=1 

 6  DA  src-address=x.x.x.x/32 src-port=any dst-address=x.x.x.x/32 
       dst-port=any protocol=udp action=encrypt level=unique 
       ipsec-protocols=esp tunnel=no proposal=default ph2-count=1 

 7  DA  src-address=x.x.x.x/32 src-port=any dst-address=169.1.101.47/32 
       dst-port=any protocol=udp action=encrypt level=unique 
       ipsec-protocols=esp tunnel=no proposal=default ph2-count=1 

 8  XI  ;;; vr1.ue1a via ISP1
       src-address=169.254.200.37/32 src-port=any 
       dst-address=169.254.200.38/32 dst-port=any protocol=gre action=encrypt 
       level=require ipsec-protocols=esp tunnel=yes 
       sa-src-address=x.x.x.x sa-dst-address=x.x.x.x 
       proposal=vpn-core ph2-count=0 

 9  XI  ;;; vr1.ew1a via ISP1
       src-address=169.254.200.53/32 src-port=any 
       dst-address=169.254.200.54/32 dst-port=any protocol=gre action=encrypt 
       level=require ipsec-protocols=esp tunnel=yes 
       sa-src-address=x.x.x.x sa-dst-address=x.x.x.x 
       proposal=vpn-core ph2-count=0 

10  A  ;;; vr1.ue1a via ISP2 
       src-address=169.254.200.41/32 src-port=any 
       dst-address=169.254.200.42/32 dst-port=any protocol=gre action=encrypt 
       level=require ipsec-protocols=esp tunnel=yes 
       sa-src-address=11.0.0.11 sa-dst-address=22.0.0.22 
       proposal=vpn-core ph2-count=1 

11  I  ;;; vr2.ue1b via ISP2
       src-address=169.254.200.49/32 src-port=any 
       dst-address=169.254.200.50/32 dst-port=any protocol=gre action=encrypt 
       level=require ipsec-protocols=esp tunnel=yes 
       sa-src-address=11.0.0.11 sa-dst-address=33.0.0.33 
       proposal=vpn-core-2 ph2-count=0 

12  A  ;;; vr1.ew1a via ISP2
       src-address=169.254.200.57/32 src-port=any 
       dst-address=169.254.200.58/32 dst-port=any protocol=gre action=encrypt 
       level=require ipsec-protocols=esp tunnel=yes 
       sa-src-address=11.0.0.11 sa-dst-address=44.0.0.44 
       proposal=vpn-core ph2-count=1 

13  XI  ;;; vr2.ue1b via ISP1
       src-address=169.254.200.45/32 src-port=any 
       dst-address=169.254.200.46/32 dst-port=any protocol=gre action=encrypt 
       level=require ipsec-protocols=esp tunnel=yes 
       sa-src-address=x.x.x.x sa-dst-address=x.x.x.x 
       proposal=vpn-core ph2-count=0 

14  A  ;;; vr2.ew1b via ISP2
       src-address=169.254.200.65/32 src-port=any 
       dst-address=169.254.200.66/32 dst-port=any protocol=gre action=encrypt 
       level=require ipsec-protocols=esp tunnel=yes 
       sa-src-address=11.0.0.11 sa-dst-address=55.0.0.55 
       proposal=vpn-core ph2-count=2 

15  XI  ;;; vr2.ew1b via ISP1
       src-address=169.254.200.61/32 src-port=any 
       dst-address=169.254.200.62/32 dst-port=any protocol=gre action=encrypt 
       level=require ipsec-protocols=esp tunnel=yes 
       sa-src-address=x.x.x.x sa-dst-address=x.x.x.x 
       proposal=vpn-core ph2-count=0 

16  DA  src-address=x.x.x.x/32 src-port=1701 dst-address=x.x.x.x/32 
       dst-port=1701 protocol=udp action=encrypt level=require 
       ipsec-protocols=esp tunnel=no proposal=default ph2-count=1

Installed SAs

# apr/17/2019 14:54:26 by RouterOS 6.42.6
# software id = PFRD-4CSN
#
Flags: H - hw-aead, A - AH, E - ESP 
 0 HE spi=0xBACCB18 src-address=44.0.0.44 dst-address=11.0.0.11 
      state=dying auth-algorithm=sha1 enc-algorithm=aes-cbc enc-key-size=128 
      auth-key="ec907ba4f84f6305f5575e3457e0014ebb3f4fb6" 
      enc-key="c395081c259e0372a9df21b4173f1b4d" addtime=apr/17/2019 13:58:46 
      expires-in=4m20s add-lifetime=48m/1h current-bytes=60516 
      current-packets=961 replay=128 

 1 HE spi=0xC467AE6A src-address=11.0.0.11 dst-address=44.0.0.44 
      state=dying auth-algorithm=sha1 enc-algorithm=aes-cbc enc-key-size=128 
      auth-key="0058fe19e77f9b43f906ef21ab7d79eadedfd907" 
      enc-key="fa7b48b61858c134d9173720f6de5abb" addtime=apr/17/2019 13:58:46 
      expires-in=4m20s add-lifetime=48m/1h current-bytes=69561 
      current-packets=961 replay=128 

 2 HE spi=0xAAD35B4 src-address=22.0.0.22 dst-address=11.0.0.11 
      state=dying auth-algorithm=sha1 enc-algorithm=aes-cbc enc-key-size=128 
      auth-key="9d6c64f88049aeaf77fce7bc18deaf1599dc270b" 
      enc-key="15875ad5d841c13a267ac1b0fe423c8c" addtime=apr/17/2019 13:58:46 
      expires-in=4m20s add-lifetime=48m/1h current-bytes=60772 
      current-packets=963 replay=128 

 3 HE spi=0xCDAF4A45 src-address=11.0.0.11 dst-address=22.0.0.22 
      state=dying auth-algorithm=sha1 enc-algorithm=aes-cbc enc-key-size=128 
      auth-key="ff50657d4bdb387ff810f03d9fa9c395a125220f" 
      enc-key="0e8c5c6f63feca436ffdf27908f5a9bd" addtime=apr/17/2019 13:58:46 
      expires-in=4m20s add-lifetime=48m/1h current-bytes=69845 
      current-packets=963 replay=128 

 4  E spi=0 src-address=11.0.0.11 dst-address=33.0.0.33 state=larval 
      add-lifetime=0s/30s replay=0 

 5 HE spi=0xA30884D src-address=x.x.x.x:4500 
      dst-address=x.x.x.x:4500 state=mature auth-algorithm=sha1 
      enc-algorithm=aes-cbc enc-key-size=256 
      auth-key="f71406fe611cd4964a2cef0153fbb1e3f88981fa" 
      enc-key="33cb955a690e371a3b5d679bd5ee7abdaa1c24750c490b6244fa29970822e1bc
        " 
      addtime=apr/17/2019 14:31:44 expires-in=7m18s add-lifetime=24m/30m 
      current-bytes=2740 current-packets=101 replay=128 

 6 HE spi=0xB230D39 src-address=x.x.x.x:4500 
      dst-address=x.x.x.x:4500 state=mature auth-algorithm=sha1 
      enc-algorithm=aes-cbc enc-key-size=256 
      auth-key="b9fef20e9c875d4fba60de3b2037c11c37e2c4f4" 
      enc-key="fe75130435862a364ccacbfa445750afbadc617204b5d294c87796581636e697
        " 
      addtime=apr/17/2019 14:31:44 expires-in=7m18s add-lifetime=24m/30m 
      current-bytes=2716 current-packets=102 replay=128 

 7 HE spi=0x956A7BA src-address=55.0.0.55 dst-address=11.0.0.11 
      state=mature auth-algorithm=sha1 enc-algorithm=aes-cbc enc-key-size=128 
      auth-key="8969e6ff2a4c9d300f10aab4df0369a622977433" 
      enc-key="af1a2db95e9f43e08544519fc39e7759" addtime=apr/17/2019 14:33:27 
      expires-in=39m1s add-lifetime=48m/1h current-bytes=2980 
      current-packets=34 replay=128 

 8 HE spi=0xC9829AC4 src-address=11.0.0.11 dst-address=55.0.0.55 
      state=mature auth-algorithm=sha1 enc-algorithm=aes-cbc enc-key-size=128 
      auth-key="982f2b2da8fac1ed9c281ef2ee5c8d26a0203648" 
      enc-key="0e0e3c4709e14929299973391a597441" addtime=apr/17/2019 14:33:27 
      expires-in=39m1s add-lifetime=48m/1h current-bytes=2602 
      current-packets=32 replay=128 

 9 HE spi=0xFADCA76 src-address=55.0.0.55 dst-address=11.0.0.11 
      state=mature auth-algorithm=sha1 enc-algorithm=aes-cbc enc-key-size=128 
      auth-key="ef216f14abee2b2364681982f558b68d71bc93cb" 
      enc-key="2f53130e853db0740174029717520555" addtime=apr/17/2019 14:34:33 
      expires-in=40m7s add-lifetime=48m/1h current-bytes=26033 
      current-packets=403 replay=128 

10 HE spi=0xCBC57297 src-address=11.0.0.11 dst-address=55.0.0.55 
      state=mature auth-algorithm=sha1 enc-algorithm=aes-cbc enc-key-size=128 
      auth-key="6a7e133426105e1f93f21e6056b54d9354c65ae7" 
      enc-key="fea9899c606d7d884c48553658017981" addtime=apr/17/2019 14:34:33 
      expires-in=40m7s add-lifetime=48m/1h current-bytes=29668 
      current-packets=403 replay=128 

11 HE spi=0x536F2EA src-address=44.0.0.44 dst-address=11.0.0.11 
      state=mature auth-algorithm=sha1 enc-algorithm=aes-cbc enc-key-size=128 
      auth-key="b83c94ab84fedf75569c25f599e7eb367a0c0091" 
      enc-key="032668b5e46c9d10590ae5522b22534f" addtime=apr/17/2019 14:46:46 
      expires-in=52m20s add-lifetime=48m/1h current-bytes=9622 
      current-packets=150 replay=128 

12 HE spi=0xC2AFB07A src-address=11.0.0.11 dst-address=44.0.0.44 
      state=mature auth-algorithm=sha1 enc-algorithm=aes-cbc enc-key-size=128 
      auth-key="f2c7f8b59e91a7a8ca345098831264d327eb38b6" 
      enc-key="6299eb92dca4a79a3109a5bc59343b5e" addtime=apr/17/2019 14:46:46 
      expires-in=52m20s add-lifetime=48m/1h current-bytes=10979 
      current-packets=150 replay=128 

13 HE spi=0xC6CB1EC src-address=22.0.0.22 dst-address=11.0.0.11 
      state=mature auth-algorithm=sha1 enc-algorithm=aes-cbc enc-key-size=128 
      auth-key="837bd18c42aea4cfec9d7183a6060fb92398d901" 
      enc-key="d7c8a7d085a8bbc360bcb8219e033951" addtime=apr/17/2019 14:46:46 
      expires-in=52m20s add-lifetime=48m/1h current-bytes=9570 
      current-packets=150 replay=128 

14 HE spi=0xC5C9AB07 src-address=11.0.0.11 dst-address=22.0.0.22 
      state=mature auth-algorithm=sha1 enc-algorithm=aes-cbc enc-key-size=128 
      auth-key="aaef7f3bcb6cabdd7a13b6d39383d8ae33757cd4" 
      enc-key="f546152cda0bff0062f7856f859a96f6" addtime=apr/17/2019 14:46:46 
      expires-in=52m20s add-lifetime=48m/1h current-bytes=10932 
      current-packets=150 replay=128 

15 HE spi=0x736C41F src-address=x.x.x.x:4500 
      dst-address=x.x.x.x:4500 state=mature auth-algorithm=sha1 
      enc-algorithm=aes-cbc enc-key-size=256 
      auth-key="893057b46f5c91763ae2e3fe053146da6960756e" 
      enc-key="4e6363a59376f556878676a0532a39fb44668c9d8c67b8ffba08fa74f75056ae
        " 
      addtime=apr/17/2019 14:46:59 expires-in=22m33s add-lifetime=24m/30m 
      current-bytes=700 current-packets=28 replay=128 

16 HE spi=0x3867E70 src-address=x.x.x.x:4500 
      dst-address=x.x.x.x:4500 state=mature auth-algorithm=sha1 
      enc-algorithm=aes-cbc enc-key-size=256 
      auth-key="0189d64459542f214d6d0eb573bfa3ef93cc109a" 
      enc-key="14bf9eeb00aaffbd9698aeb3c38563cbf973ad917a95085e87025f3db7d3ce0f
        " 
      addtime=apr/17/2019 14:46:59 expires-in=22m33s add-lifetime=24m/30m 
      current-bytes=700 current-packets=28 replay=128 

17 HE spi=0x1D80EFD src-address=x.x.x.x:55106 
      dst-address=x.x.x.x:4500 state=mature auth-algorithm=sha1 
      enc-algorithm=aes-cbc enc-key-size=256 
      auth-key="0c65ed80e43f54c832c47e42f78539411c41adbd" 
      enc-key="19be4775cf2d0551934ee627870b3564b09343599269c864e1eccb52da391d21
        " 
      addtime=apr/17/2019 14:47:14 expires-in=22m48s add-lifetime=24m/30m 
      current-bytes=700 current-packets=28 replay=128 

18 HE spi=0xB30D1AF src-address=x.x.x.x:4500 
      dst-address=x.x.x.x:55106 state=mature auth-algorithm=sha1 
      enc-algorithm=aes-cbc enc-key-size=256 
      auth-key="795fe89d95c7c9d9ccef2ed44e82cc4077de18e4" 
      enc-key="937ef7333fc1c878aa089f2700bf4caf50f5e18c654e628608956d49481e9a48
        " 
      addtime=apr/17/2019 14:47:14 expires-in=22m48s add-lifetime=24m/30m 
      current-bytes=700 current-packets=28 replay=128 

19 HE spi=0x6B40F72 src-address=x.x.x.x:55106 
      dst-address=x.x.x.x:4500 state=mature auth-algorithm=sha1 
      enc-algorithm=aes-cbc enc-key-size=256 
      auth-key="53a834975de434b965906aff012bf97b099256aa" 
      enc-key="8ea6f6af04da31faf8f615f2d8c9a7c610f073824eb6002464c250783a137ed0
        " 
      addtime=apr/17/2019 14:47:16 expires-in=22m50s add-lifetime=24m/30m 
      current-bytes=40135 current-packets=498 replay=128 

20 HE spi=0x60F06FE src-address=x.x.x.x:4500 
      dst-address=x.x.x.x:55106 state=mature auth-algorithm=sha1 
      enc-algorithm=aes-cbc enc-key-size=256 
      auth-key="a4df74b9ca46bd9f53bd468f35f40c9d36b66e1e" 
      enc-key="33c9638a313ff4c5429c8e3ff038f16f862887a185f8ca148a28c05d9612e54c
        " 
      addtime=apr/17/2019 14:47:16 expires-in=22m50s add-lifetime=24m/30m 
      current-bytes=51368 current-packets=492 replay=128 

21 HE spi=0x6FC3B38 src-address=x.x.x.x:4500 
      dst-address=x.x.x.x:4500 state=mature auth-algorithm=sha1 
      enc-algorithm=aes-cbc enc-key-size=256 
      auth-key="9617ead9108839caf79be6eced85dc9365d48f1e" 
      enc-key="3b817244ff9d861e31ced7a8c3c929db288815f2c6a16bacec1051c7227e4cbf
        " 
      addtime=apr/17/2019 14:47:21 expires-in=22m55s add-lifetime=24m/30m 
      current-bytes=700 current-packets=28 replay=128 

22 HE spi=0x4079F7F src-address=x.x.x.x:4500 
      dst-address=x.x.x.x:4500 state=mature auth-algorithm=sha1 
      enc-algorithm=aes-cbc enc-key-size=256 
      auth-key="92337d636c6cdb1f4f4dcfd4aa5bf704430e0380" 
      enc-key="6850e81b33ded322828b59aa1b559d2c920b0a0d8ed6556eecd0f3cebe384169
        " 
      addtime=apr/17/2019 14:47:21 expires-in=22m55s add-lifetime=24m/30m 
      current-bytes=700 current-packets=28 replay=128 

23 HE spi=0xB282A54 src-address=x.x.x.x:4500 dst-address=x.x.x.x:4500 
      state=mature auth-algorithm=sha1 enc-algorithm=aes-cbc enc-key-size=256 
      auth-key="200b07167812e8ff8cc9a5cb8d4cba05a239f5a0" 
      enc-key="56120635894d804e8882b8b4fa60a714863519680386e3adbce64d2eedb5401e
        " 
      addtime=apr/17/2019 14:47:25 expires-in=22m59s add-lifetime=24m/30m 
      current-bytes=674 current-packets=27 replay=128 

24 HE spi=0x241DDD4 src-address=x.x.x.x:4500 dst-address=x.x.x.x:4500 
      state=mature auth-algorithm=sha1 enc-algorithm=aes-cbc enc-key-size=256 
      auth-key="14658f8dcdd57f32320f754bf460435f7cd0a216" 
      enc-key="3e04c994c37f1e7afddafa3f5289d7fb6764181ff5db6244fd7124b102c879ff
        " 
      addtime=apr/17/2019 14:47:25 expires-in=22m59s add-lifetime=24m/30m 
      current-bytes=700 current-packets=28 replay=128 

25 HE spi=0x7D77110 src-address=x.x.x.x dst-address=x.x.x.x 
      state=mature auth-algorithm=sha1 enc-algorithm=aes-cbc enc-key-size=256 
      auth-key="d314ee548c480801ffa597bf1e83956544a4bc04" 
      enc-key="da48750250ef0c2d9b467b0aaac0709cc31d6d9091b3513a501bc84e1f197614
        " 
      addtime=apr/17/2019 14:47:47 expires-in=23m21s add-lifetime=24m/30m 
      current-bytes=674 current-packets=27 replay=128 

26 HE spi=0xCC688AE src-address=x.x.x.x dst-address=x.x.x.x 
      state=mature auth-algorithm=sha1 enc-algorithm=aes-cbc enc-key-size=256 
      auth-key="9b594493e116fb64736fc075652228b7dec69247" 
      enc-key="a5d8be8dc82bc59394b7cae87b36537bf8bc3600462b19405ee7ae1758615b14
        " 
      addtime=apr/17/2019 14:47:47 expires-in=23m21s add-lifetime=24m/30m 
      current-bytes=674 current-packets=27 replay=128 

27 HE spi=0x6729136 src-address=x.x.x.x:4500 
      dst-address=x.x.x.x:4500 state=mature auth-algorithm=sha1 
      enc-algorithm=aes-cbc enc-key-size=256 
      auth-key="e24ac5803180b36a365ee204cbc7277ea5598d6a" 
      enc-key="5732f60781ab081127f45e34c56ea410055e8250827ae02168dbac5b7b0c2927
        " 
      addtime=apr/17/2019 14:47:48 expires-in=53m22s add-lifetime=48m/1h 
      current-bytes=1926716 current-packets=12475 replay=128 

28 HE spi=0x568955D8 src-address=x.x.x.x:4500 
      dst-address=x.x.x.x:4500 state=mature auth-algorithm=sha1 
      enc-algorithm=aes-cbc enc-key-size=256 
      auth-key="0c84ceace21d61e3ebb955392ce9377905cf014b" 
      enc-key="1733430faaad32c0c204e4567f263b434a8430e4eef682a417903ee7c113882a
        " 
      addtime=apr/17/2019 14:47:48 expires-in=53m22s add-lifetime=48m/1h 
      current-bytes=18040199 current-packets=16745 replay=128 

29 HE spi=0xF38D2CF src-address=x.x.x.x dst-address=x.x.x.x 
      state=mature auth-algorithm=sha1 enc-algorithm=aes-cbc enc-key-size=256 
      auth-key="7f436936d4e21d9fa62483b857e65ee3b5db6a60" 
      enc-key="34002c993ddeb369f492a59a1d063fe33ab3440a1bc5b4a86df5d8c1f1ceab92
        " 
      addtime=apr/17/2019 14:53:59 expires-in=59m33s add-lifetime=48m/1h 
      current-bytes=314 current-packets=5 replay=128 

30 HE spi=0x114B7824 src-address=x.x.x.x dst-address=x.x.x.x 
      state=mature auth-algorithm=sha1 enc-algorithm=aes-cbc enc-key-size=256 
      auth-key="8997b7b516a5bdfda3d6bb300814c38ebb6de92a" 
      enc-key="1324998194b244bf9a24264b73acefb1cf7dc4d837e7b77d3f372ce8b7ab1a84
        " 
      addtime=apr/17/2019 14:53:59 expires-in=59m33s add-lifetime=48m/1h 
      current-bytes=442 current-packets=11 replay=128

sindy · Wed Apr 17, 2019 6:36 pm

My personal opinion is that has nothing to do with configuration. None of your other policies can shadow the one mysteriously going inactive, even if we'd admit that the order of the policies wouldn't matter. It normally does but when you add a new policy this used to fail in older ROS releases, e.g. on the 6.43.8 where I've just tested it.

So it is either a bug or, if the other Routerboard/CHR /x86 on which you've never seen the issue to happen is the same model like the problematic one, there can be a hardware (RAM) issue on the problematic one.

So you may try the following steps and hope that one of them works around the bug (the last one may "fix" rather than "workaround" it although none of the changelog items listed below is guaranteed to be related):

separate the addresses used to send for the GRE transport (to be matched by the IPsec policy) from the addresses attached to the GRE interface, at least on Mikrotik side, the way I've suggested above; doing so will allow you to remove the protocol=gre from the traffic selector of the /ip ipsec policy
set the sa-src-address of the policy to 0.0.0.0. It may make things better (the policy won't jump inactive) or worse (it won't ever come up), the idea is just to force a different branch of the algorithm, not something precisely targeted to a known issue
set the level to unique instead of the current require - same motivation like above
upgrade to 6.43.12 (the last available one before 6.44 - to keep the structure of IPsec configuration you are used to)

The following items in the changelog may be related:

6.42.7: *) ipsec - improved invalid policy handling when a valid policy is uninstalled;
6.43:
- *) ike2 - fixed initiator first policy selection;
- *) ipsec - improved invalid policy handling when a valid policy is uninstalled;
- *) ipsec - improved reliability on generated policy addition when IKEv1 or IKEv2 used;

The "improved invalid policy handling when a valid policy is uninstalled" is mentioned also in 6.44 changelog, so it seems to be a complex issue.

avdvyver01 · Thu Apr 18, 2019 10:01 am

Thank you very much for your time sindy, I will update this thread with the findings.

avdvyver01 · Tue Jun 11, 2019 12:41 pm

Update:

I have:

changed the policy level to unique. This unfortunately did not resolve the issue.

moved the IP addresses used to match the IPSec policy to a loopback adapter (bridge) and away from the GRE interface. This unfortunately did not resolve the issue.

Upgraded to v6.43.16. The issue however remains

One thing that I have been able to achieve is forcing the policy out of the invalid state. This is achieved by changing the protocol to something else - e.g. all. Changing it back to protocol 47, immediately puts it back into an invalid state. Even if the protocol is set to all, not 47, it still randomly goes into an invalid state (and some hours later, fixes itself, and the process repeats).

I will update Mikrotik support with the findings.

nuffrespect · Wed Jun 26, 2019 3:35 pm

I will update Mikrotik support with the findings.

Hi! Do you have some fresh findings how resolve this bug with GRE (transport) + IPSEC
We have the same situation now...

By the way - for some tunnels i changed from GRE to IPIP tunnel, and they are work without problem, then i returned to GRE => one or more times for the day we have ipsec,error ... dead ph2

Valdis · Thu Sep 12, 2019 7:48 am

Hello, Same here RB3011UiAS upgraded from v 6.44.1 to v 6.45.1 L2TP stopped working and getting error: "src IP address failed to pre-process ph2 packet."
Downgraded back to v6.44.1 still getting the error.
Upgraded to v 6.44.5 (Long term) and still having the issue with L2TP ph2 fail...
PPTP, SSTP working fine...
The configuration is the sames as before first upgrade... Did the "export compact file="config.rsc"" and re checked everything...

Valdis · Thu Sep 12, 2019 11:03 am

Hello, Same here RB3011UiAS upgraded from v 6.44.1 to v 6.45.1 L2TP stopped working and getting error: "src IP address failed to pre-process ph2 packet."
Downgraded back to v6.44.1 still getting the error.
Upgraded to v 6.44.5 (Long term) and still having the issue with L2TP ph2 fail...
PPTP, SSTP working fine...
The configuration is the sames as before first upgrade... Did the "export compact file="config.rsc"" and re checked everything...

Okay, I found that my default IPsec Policies was disabled...
I enabled it...
Now I don't receive error, but still can't connect - MT log at attachment:

Valdis · Thu Sep 12, 2019 2:11 pm

Hello, Same here RB3011UiAS upgraded from v 6.44.1 to v 6.45.1 L2TP stopped working and getting error: "src IP address failed to pre-process ph2 packet."
Downgraded back to v6.44.1 still getting the error.
Upgraded to v 6.44.5 (Long term) and still having the issue with L2TP ph2 fail...
PPTP, SSTP working fine...
The configuration is the sames as before first upgrade... Did the "export compact file="config.rsc"" and re checked everything...
Okay, I found that my default IPsec Policies was disabled...
I enabled it...
Now I don't receive error, but still can't connect - MT log at attachment:

removed and re-added my L2TP client configuration at Win10 and it started working...

avdvyver01 · Fri Sep 27, 2019 9:45 am

After trying many different approaches and new versions, I have upgraded my CCR1036 to v6.45.6 on the 18th of September and so far the IPSec policies have remained stable. I have noticed that dynamic policies used for L2TP over IPSec sometimes go into an invalid state but this seems to be isolated to L2TP and I will dig deeper into that at a later state. For now it looks good. I believe that *) ipsec - fixed policies becoming invalid after changing priority; in the 6.45.1 changelog addressed this. Holding thumbs it stays stable.

@Valdis, the 6.45.6 upgrade also broke my L2TP server. It seems to have removed the IPSec secret which I had to add back and there was also something wrong or mismatched with the policy. I just reconfigured the default policy and cleaned up the auto generated stuff and that solved the issue.

alejo · Tue Oct 15, 2019 2:54 am

Hi, comparing 6.43.11 to 6.45.6, apparently default auth algorithms changed:

/ip ipsec proposal> pr
 0  * name="default" auth-algorithms=sha1 enc-algorithms=aes-256-cbc,aes-192-cbc,aes-128-cbc lifetime=30m pfs-group=modp1024

to

0  * auth-algorithms=sha512,sha256 enc-algorithms=aes-256-cbc,aes-256-ctr,aes-256-gcm,aes-192-ctr,aes-192-gcm,aes-128-cbc,aes-128-ctr,aes-128-gcm lifetime=8h pfs-group=none

So, adding "sha1" to auth-algorithms solved this problem.

Also, I've to change simultaneous sessions:

/interface l2tp-server server set one-session-per-host=no

zerounu · Tue Jun 09, 2020 11:35 am

I was searching a resolve for this problem and i found this post. I tried some things posted by other users here but with no success.
I have many ipsec tunnels and i didn't look carefully at ipsec policy - peer name and i looked only at numbers: it seems that they are changed and all my tunnels used random names from my peers list. I don't know if this happened after i restored a backup or after a router restart. After i choosed the correct peers on the ipsec policy tab everything started working again.

IPSec failed to pre-process ph2 packet

IPSec failed to pre-process ph2 packet

Re: IPSec failed to pre-process ph2 packet

Re: IPSec failed to pre-process ph2 packet

Re: IPSec failed to pre-process ph2 packet

Re: IPSec failed to pre-process ph2 packet

Re: IPSec failed to pre-process ph2 packet

Re: IPSec failed to pre-process ph2 packet

Re: IPSec failed to pre-process ph2 packet

Re: IPSec failed to pre-process ph2 packet

Re: IPSec failed to pre-process ph2 packet

Re: IPSec failed to pre-process ph2 packet

Re: IPSec failed to pre-process ph2 packet

Re: IPSec failed to pre-process ph2 packet

Re: IPSec failed to pre-process ph2 packet

Re: IPSec failed to pre-process ph2 packet

Re: IPSec failed to pre-process ph2 packet

Re: IPSec failed to pre-process ph2 packet

Re: IPSec failed to pre-process ph2 packet

Re: IPSec failed to pre-process ph2 packet

Re: IPSec failed to pre-process ph2 packet

Re: IPSec failed to pre-process ph2 packet

Who is online