It indeed writes "encrypted payload" into the log, however it also shows the decrypted contents in hexadecimal form (if you remove the
!packet part from the
topics list under
/system logging, see
here how to visualize it if really necessary), but even more important, it usually shows the contents in an even more readable form. At least you should see the required traffic selector and the proposed transform values in the request from the Fortigate. See an example below:
01:47:00 ipsec,debug ===== received 604 bytes from 192.168.227.11[4500] to 192.168.227.13[4500]
01:47:00 ipsec,debug,packet 7911f08d 2ebe0b20 04523ba7 30747483 2e202408 0000001e 0000025c 28000240
...
01:47:00 ipsec,debug,packet 27c3261f 0c57f758 9b6df524 9284dade 5239e228 6ba17615 dbb2b40e
01:47:00 ipsec -> ike2 request, exchange: CREATE_CHILD_SA:30 192.168.227.11[4500] 7911f08d2ebe0b20:04523ba730747483
01:47:00 ipsec payload seen: ENC (576 bytes)
01:47:00 ipsec processing payload: ENC
01:47:00 ipsec,debug => iv (size 0x10)
01:47:00 ipsec,debug 88361d79 94aa39b5 41beb48f fd74926b
01:47:00 ipsec,debug decrypted
01:47:00 ipsec,debug,packet => decrypted packet (size 0x150)
01:47:00 ipsec,debug,packet 2200001c 619b82b6 bbcd1ad1 5a6329eb 1ed6afd3 af145619 6a1e5100 21000088
...
01:47:00 ipsec,debug,packet 00000000 00030004 00000000 00190000
01:47:00 ipsec payload seen: NONCE (28 bytes)
01:47:00 ipsec payload seen: KE (136 bytes)
01:47:00 ipsec payload seen: SA (76 bytes)
01:47:00 ipsec payload seen: TS_I (24 bytes)
01:47:00 ipsec payload seen: TS_R (24 bytes)
01:47:00 ipsec payload seen: CONFIG (48 bytes)
01:47:00 ipsec create child: respond
01:47:00 ipsec processing payloads: NOTIFY (none found)
01:47:00 ipsec processing payloads: NOTIFY (none found)
01:47:00 ipsec peer wants tunnel mode
01:47:00 ipsec processing payload: CONFIG
01:47:00 ipsec attribute: internal IPv4 address size: 4
01:47:00 ipsec attribute: internal IPv4 netmask size: 4
01:47:00 ipsec attribute: internal IPv6 subnet size: 8
01:47:00 ipsec attribute: internal IPv4 DNS size: 4
01:47:00 ipsec attribute: internal DNS domain
01:47:00 ipsec processing payload: TS_I
01:47:00 ipsec 192.168.88.1
01:47:00 ipsec processing payload: TS_R
01:47:00 ipsec 192.168.137.146
01:47:00 ipsec canditate selectors: 192.168.137.146 <=> 192.168.88.1
01:47:00 ipsec processing payload: SA
01:47:00 ipsec IKE Protocol: ESP
01:47:00 ipsec proposal #1
01:47:00 ipsec enc: aes256-cbc
01:47:00 ipsec enc: aes192-cbc
01:47:00 ipsec enc: aes128-cbc
01:47:00 ipsec auth: sha1
01:47:00 ipsec dh: modp1024
01:47:00 ipsec searching for policy for selector: 192.168.137.146 <=> 192.168.88.1
01:47:00 ipsec using strict match: 192.168.137.146 <=> 192.168.88.1
01:47:00 ipsec allocated address from pool is not within selector
01:47:00 ipsec,error no proposal chosen
01:47:00 ipsec adding notify: NO_PROPOSAL_CHOSEN