Can packet sniffing be used to debug IPSec/routing?

I have my primary router configured as an IPSec listener for several IKEv2 remotes. Most of these work the way I want. One of them…I can establish a connection and the routers talk to each other but I cannot reach the LAN of the remote router. Obviously I’ve got something wrong but I can’t see what. So before I start posting export excerpts…can packet sniffing (or anything else) be used to see where the break is?

If so, please be specific - I have managed to setup a tshark listener on a Linux machine to receive a stream from my primary router but I’m clueless when it comes to the filters. And if this is possible - I think spelling this out, maybe in a Wiki page, would help a lot of people.

No, the traffic between IPsec peers is (normally) encrypted and there is very little you can deduce from it other than basic routing problems between the peers.
(e.g. you have traffic in one direction and not the other)

IPsec-level problems like negotiation can not really be debugged from that.
You can enable logging of ipsec debug and have some info from that.
But it is inconvenient when you have “several remotes” as there is no way to enable debug for only a single remote so you will be overwhelmed with output for all the sessions that are running…

Maybe I should rephrase - I understand the IPSec communication itself is encrypted and I’m not really asking about the handshake negotiation. I’m asking about routing & policy decisions. Something that might reveal more than just a ping or traceroute showing “no response” - I don’t know if ICMP, IP, or any other protocol exposes more that might point a “just enough to be dangerous” admin (like myself) to where to look for the break.

Probably what I really want would be a way of seeing the decisions the router is actively making for a given source & destination - and I don’t think that exists.

To my knowledge there is no way to log routing decisions.

What you can use are mangle rules in chain postrouting, with action=log, matching on addresses, protocols, and ports, to see what is the chosen out-interface for a packet (but not why it has been chosen). However, IPsec policies’ traffic selectors check packets after they pass through the complete postrouting chain, and since mangle precedes src-nat in postrouting, the log won’t show you the eventual new source address for the initial packet of a connection; as the NAT handling for the whole connection is determined while nat table handles its initial packet and the rest of that connection’s packets inherit that decision, the new source address is shown in the log for them:

[me@myTik] > log print follow-only where topics~"firewall"
07:39:58 firewall,info postrouting: in:(unknown 0) out:bridge.wan.1, src-mac 37:03:35:78:93:71, proto ICMP (type 8, code 0), 192.168.88.12->1.2.3.4, len 60
07:39:59 firewall,info postrouting: in:(unknown 0) out:bridge.wan.1, src-mac 37:03:35:78:93:71, proto ICMP (type 8, code 0), 192.168.88.12->1.2.3.4, NAT (192.168.88.12->195.21.30.103)->1.2.3.4, len 60
07:40:01 firewall,info postrouting: in:(unknown 0) out:bridge.wan.1, src-mac 37:03:35:78:93:71, proto ICMP (type 8, code 0), 192.168.88.12->1.2.3.4, NAT (192.168.88.12->195.21.30.103)->1.2.3.4, len 60

What the log doesn’t show you is the eventual connection-mark and/or routing-mark assigned to the packet (which is rarely important for IPsec-related investigation). You can visualize the connection-mark using /ip firewall connection print detail where …, but I haven’t found any way to visualize the routing-mark assigned to a packet except using another rule matching on its value.

To find out which IPsec policy matches the packet above, you can use /ip ipsec policy print detail where 195.21.30.103 in src-address and 1.2.3.4 in dst-address (the actual policy matching is done after the NAT treatment has been applied). If more policies are shown, the topmost one will actually handle the packet.

Loosely related, a match condition ipsec-policy=(in|out),(ipsec|none) is available, which compares packet’s hedaer fields to traffic selectors of all existing policies (comparing packet’s source address and port to policy’s source address and port, and destination to destination, for out, and comparing packet’s source to policy’s destination and packet’s destination to policy’s source for in). The primary purpose of this match condition is to allow to exempt packets which should be handled by IPsec from getting fasttracked and/or src-NATed, but you can use them for selective logging too. It is important to realize that this condition uses the current addresses of the packet at the moment of matching, i.e. in prerouting (mangle and nat/dstnat), the destination address is the real one; in all the subsequent chains, it is the new destination address if dst-nat treatment applies for the connection. The source address is always the one before eventual src-nat treatment.

And even more loosely related - you can use action=sniff-tzsp rules in mangle to sniff packets before they get encrypted by IPsec.