OK, so from the very beginning. The basic function is the following:
,initiator.public.ip:500 <=======> responder.public.ip:500.
initiator{ }responder
`initiator.public.ip:4500 <=====> responder-public.ip:4500'
As in generic case both participants of an IPsec “session” are peers, no “random src port” is used on the initiator side. However, the NAT-T mechanism, which is optional for IKE(v1) but intrinsical in IKEv2, can deal with any change of src-port of packets coming from the initiator as they travel to the responder.
What I’ve understood from what you wrote was that the Linux cannot listen on UDP port 500 and 4500 so you have to chose some other ones, such as e.g. 1005 in place of 500 and 1045 in place of 4500 for ease of description.
As your home Mikrotik is placed behind a NATing modem/router, it must act as an initiator, i.e. send the first packet towards the Linux machine which has a fixed address and can listen, acting as an IPsec responder, on some ports (e.g., the 1005 and 1045 ones).
In the IPsec configuration of Mikrotik, you can say “send to responder port 1005 what you would normally send to responder port 500”, but you can not say “send to responder port 1045 what you would normally send to responder 4500”. So if the Linux VPS (whatever it is) contract does not permit your responder to listen at port 4500 and you are forced to use 1045 instead, you have to redirect (dst-pat) the outgoing packets of the initiator somewhere close to it, and redirect (dst-pat) the packets received at port 1045 on the responder Linux machine back to port 4500 where your IPsec stack listens.
On the responder side, the Linux, this is a matter of a single rule, iptables -t nat -A PREROUTING -i ethX -p udp --dport 1045 -j DNAT --to-destination :4500, as dst-nat of received packets is a commonly required handling. However, as dst-nat of sent packets is much less required, and so is src-nat of received packets, normal RouterOS handling doesn’t support either of these.
So as a workaround, you have to let the IPsec stack send the packet to port 4500, but instead of sending it directly out the WAN, you send it to an IP-IP tunnel whose other end is on the same Mikrotik. On that end, the packet becomes an incoming one, so it can be dst-nated to port 1045; using policy routing, you send it out the WAN; the policy routing criteria allowing the action=mark-routing mangle rules to distinguish between the first and second pass of the packet is that the original one has no in-interface as it is locally originated, while the second pass has the IP-IP tunnel interface as in-interface (and has connection-nat-state=dstnat and is handled by chain=prerouting), so you can use a rule like
/ip firewall mangle add chain=output action=mark-routing dst-address=the.responder.ip.addr protocol=udp dst-port=4500 new-routing-mark=via-ipip
to assign the routing mark causing the first pass packets to be routed via the following route:
/ip route add dst-address=0.0.0.0/0 gateway=ip.address.of.outer.ipip.end routing-mark=via-ipip
And you use an
/ip firewall nat add chain=dstnat action=dst-nat to-ports=1045 protocol=udp dst-port=4500 dst-address=the.responder.ip.addr
rule to redirect the packets to port 1045 before they are routed using the default routing table to the destination.
As your home modem/router src-nats the packets, you don’t need to do that on the Mikrotik to force use of UDP-encapsulated ESP.
See the post I’ve referred to above to find out how to set up the IPIP tunnel to the Mikrotik itself.
As you want to use IKEv2, the above is enough because with exchange-mode=ike2, RouterOS acting as initiator starts sending immediately towards port 4500 of the responder, so you don’t need to set up this complex treatment for port 500.
Other than that - yes, you can use StrongSwan at the responder side, with either a pre-shared key configuration or a certificate. But you have to ping through the tunnel periodically, otherwise the security association may close at StrongSwan side if there is no spontaneous traffic for an extended period of time.
