IPSEC: Authentication failed with certificates

Hi everyone,

I am currently struggling with an IPSEC setup involving:

  • Router A: CRS125-24G-1S-2HnD, certificate SAN: a.company.com
  • Router B: hAP ac^2, certificate SAN: b.company.com
  • Router C: <>, certificate SAN: c.company.com


  • All routers have valid certificates signed by the company CA. The company CA is trusted by all Routers.
  • All Routers are running on RouterOS 6.45.6

Summary
IPSEC is configured using auth-method=digital-signature and remote-id=fqdn:x.company.com
Router B is always configured to be the initiator, Router A and C are responders.

What works:
Router B can initiate a connection to Router A, no matter if remote-id is set to the FQDN of router A or Router C. This is very strange and dangerous. Maybe caused by some kind of caching (although it even works after a reboot of B).

What does not work:
B cannot open a tunnel to C, even though remote-id is set to the FQDN of C. The connection always fails with

07:29:03 ipsec,info new ike2 SA (I): <B-IP>[4500]-<C-IP>[4500] spi:a6a5c7a1cddba9c3:b3d7a42035a43040
07:29:03 ipsec,error got fatal error: AUTHENTICATION_FAILED
07:29:03 ipsec,info killing ike2 SA: <B-IP>[4500]-<C-IP>[4500] spi:a6a5c7a1cddba9c3:b3d7a42035a43040

Here are the configurations:
Router A:

  • Identity: peer=B auth-method=digital-signature remote-id=fqdn:b.company.com certificate=a.company.com.cer_0 generate-policy=no
  • Peer: name=“B” address=/32 passive=yes profile=profilename exchange-mode=ike2 send-initial-contact=no
  • Profile: name=“profilename” hash-algorithm=sha256 enc-algorithm=aes-256 dh-group=modp3072 lifetime=1d proposal-check=obey nat-traversal=no dpd-interval=2m dpd-maximum-failures=5
  • Policy: peer=B tunnel=yes src-address=192.168.0.0/24 src-port=any dst-address=192.168.10.0/24 dst-port=any protocol=all action=encrypt level=require ipsec-protocols=esp sa-src-ss=0.0.0.0 sa-dst-address= proposal=proposalname ph2-count=0
  • Proposal: name=“proposalname” auth-algorithms=sha256 enc-algorithms=aes-256-cbc lifetime=30m pfs-group=modp3072

Router B:

  • Identity: peer=A auth-method=digital-signature remote-id=fqdn:a.company.com certificate=b.company.com.cer_0 generate-policy=no
  • Peer: name=“A” address=/32 profile=profilename exchange-mode=ike2 send-initial-contact=yes
  • Profile: name=“profilename” hash-algorithm=sha256 enc-algorithm=aes-256 dh-group=modp3072 lifetime=1d proposal-check=obey nat-traversal=no dpd-interval=2m dpd-maximum-failures=5
  • Policy: peer=A tunnel=yes src-address=192.168.10.0/24 src-port=any dst-address=192.168.0.0/24 dst-port=any protocol=all action=encrypt level=require ipsec-protocols=esp sa-src-address=0.0.0.0 sa-dst-address= proposal=proposalname ph2-count=0
  • Proposal: name=“proposalname” auth-algorithms=sha256 enc-algorithms=aes-256-cbc lifetime=30m pfs-group=modp3072

Router C:

  • Identity: peer=B auth-method=digital-signature remote-id=fqdn:b.company.com certificate=c.company.com.cer_0 generate-policy=no
  • Peer: name=“B” address=/32 passive=yes profile=profilename exchange-mode=ike2 send-initial-contact=no
  • Profile: name=“profilename” hash-algorithm=sha256 enc-algorithm=aes-256 dh-group=modp3072 lifetime=1d proposal-check=obey nat-traversal=no dpd-interval=2m dpd-maximum-failures=5
  • Policy: peer=B tunnel=yes src-address=192.168.0.0/24 src-port=any dst-address=192.168.10.0/24 dst-port=any protocol=all action=encrypt level=require ipsec-protocols=esp sa-src-address=0.0.0.0 sa-dst-address= proposal=proposalname ph2-count=0
  • Proposal: name=“proposalname” auth-algorithms=sha256 enc-algorithms=aes-256-cbc lifetime=30m pfs-group=modp3072

Help is highly appreciated!

You need to enable debug logging and confirm that every peer sends the ID as you are expecting. If a certificate has more than one SAN, IPsec can choose any one of them. What happens if you manually specify the my-id parameter instead of leaving it to auto? Debug logging can be enabled with this command:

/system logging add topics=ipsec,!packet

Thanks for the fast reply!

Debug log of B seems to state that is fine:

11:38:26 ipsec adding payload: AUTH
11:38:26 ipsec,debug => (first 0x100 of 0x108)
[...]
11:38:26 ipsec cert: CN=b.company.com,C= [..]
11:38:26 ipsec adding payload: CERT
11:38:26 ipsec,debug => (first 0x100 of 0x5c4)
[...]
11:38:26 ipsec adding payload: CERTREQ
11:38:26 ipsec,debug => (size 0x5)
11:38:26 ipsec,debug 00000005 04
11:38:26 ipsec ID_R (FQDN): c.company.com
11:38:26 ipsec adding payload: ID_R
11:38:26 ipsec,debug => (size 0x21)
11:38:26 ipsec,debug 00000021 02000000 67776867 622e6c69 6d657373 65637572 6974792e 6c6f6361
11:38:26 ipsec,debug 6c
11:38:26 ipsec adding notify: INITIAL_CONTACT
11:38:26 ipsec,debug => (size 0x8)
11:38:26 ipsec,debug 00000008 00004000
11:38:26 ipsec adding payload: SA
11:38:26 ipsec,debug => (size 0x2c)
11:38:26 ipsec,debug 0000002c 00000028 01030403 0a87d7a1 0300000c 0100000c 800e0100 03000008
11:38:26 ipsec,debug 0300000c 00000008 05000000
11:38:26 ipsec initiator selector: 192.168.10.0/24
11:38:26 ipsec adding payload: TS_I
11:38:26 ipsec,debug => (size 0x18)
11:38:26 ipsec,debug 00000018 01000000 07000010 0000ffff c0a80a00 c0a80aff
11:38:26 ipsec responder selector: 192.168.0.0/24
11:38:26 ipsec adding payload: TS_R
11:38:26 ipsec,debug => (size 0x18)
11:38:26 ipsec,debug 00000018 01000000 07000010 0000ffff c0a80000 c0a800ff
11:38:26 ipsec <- ike2 request, exchange: AUTH:1 <C-IP>[4500]
11:38:26 ipsec,debug ===== sending 2048 bytes from <B-IP>[4500] to <C-IP>[4500]
11:38:26 ipsec,debug 1 times of 2052 bytes message will be sent to <C-IP>[4500]
11:38:26 ipsec,debug ===== received 224 bytes from <C-IP>[4500] to <B-IP>[4500]
11:38:26 ipsec -> ike2 reply, exchange: AUTH:1 <C-IP>[4500]
11:38:26 ipsec payload seen: ENC (196 bytes)
11:38:26 ipsec processing payload: ENC
11:38:26 ipsec,debug => iv (size 0x10)
11:38:26 ipsec,debug 185d62ac 89a24ad5 6f183584 c56169a0
11:38:26 ipsec,debug => plain payload (trimmed) (size 0x8)
11:38:26 ipsec,debug 00000008 00000018
11:38:26 ipsec,debug decrypted
11:38:26 ipsec payload seen: NOTIFY (8 bytes)
11:38:26 ipsec processing payloads: NOTIFY
11:38:26 ipsec   notify: AUTHENTICATION_FAILED
11:38:26 ipsec,error got fatal error: AUTHENTICATION_FAILED

where c.company.com is the expected SAN of C

The responder is closing the connection with AUTHENTICATION_FAILED notification. Debug logs on the other side should reveal more information.

Oh man, thanks for pointing me in the right direction. Once I recognized that I was looking for the problem on the wrong router, I could easily identify the misconfigured clock as the source of the problem!

I will try if I can reproduce the issue, that the connection is established even if the FQDN does not match, but hopefully this was just a chaching issue. But just to be sure that I do not misunderstand the mechanism: if I define auth-method=digital-signature remote-id=fqdn:b.company.com, the peer MUST provide a signed certificate with a matching FQDN, right?

Thanks a lot for the fast support!

Yes, that is correct. Both the initiator and the responder has to provide a certificate with the SAN matching its ID. Only remote-id=ignore will remove this validation.