IPSec IKEv2 RoadWarrior - ping works, https not

skrgahr · October 16, 2020, 1:59pm

Hello all,

I have followed tutorial from MUM meeting on how to set up IPSec IKEv2 road warrior configuration.
I want to setup server for our company to get rid of OpenVPN.
We use 192.168.0.0/24 network, and we have VPN with our business partner, and they are in 10.0.0.0/8 and 94.242.41.0/24 (not important range at the moment). We use NAT to connect to their network.

I have created certificates for each employee, and windows/Android clients connect without any issue. I have disabled default route because I don’t want internet traffic to go through VPN.
Problem is that i can ping any host on our/partner network, company DNS works, even SSH works (although with occasional reconnect), but HTTP does not work.
RDC also does not work, it disconnects after few seconds.

Our main WAN IP is 78.134.209.170, IP pool for VPN is 192.168.43.0/24. I have tried with 10.x.x.x and 172.16.x.x, without success.
Where did I do wrong? I’m trying to make it work for 2 days now, trying almost every setting on ipsec and firewall.

# oct/16/2020 15:30:37 by RouterOS 6.46.7
# software id = M0DN-WZ2Z
#
# model = 2011UiAS-2HnD
# serial number = 467304D2829E
/interface bridge
add admin-mac=4C:5E:0C:00:00:01 arp=proxy-arp auto-mac=no comment=3 \
    fast-forward=no name=LAN-WIFI
add name=VPN

/ip address
add address=192.168.43.1/24 interface=VPN network=192.168.43.0
add address=192.168.0.91/24 interface=LAN-WIFI network=192.168.0.0
add address=78.134.209.170/29 interface=ETH1-METRONET network=78.134.209.168
add address=10.124.10.100/16 interface=ETH3-PARTNER network=10.124.0.0

/ip route
add distance=1 gateway=78.134.209.169 routing-mark=METRONET
add distance=1 gateway="T-COM PPPoE" routing-mark=T-COM
add comment="PARTNER mreza" distance=1 dst-address=10.0.0.0/8 gateway=10.124.1.100
add comment="PARTNER mreza" distance=1 dst-address=194.242.41.0/24 gateway=10.124.1.100

/ip ipsec policy group
add name="group vpn.company.hr"
/ip ipsec profile
add dh-group=modp2048,modp1536,modp1024 enc-algorithm=\
    aes-256,aes-192,aes-128,3des hash-algorithm=sha256 name=\
    "profile vpn.company.hr"
/ip ipsec peer
add exchange-mode=ike2 local-address=78.134.209.170 name=peer passive=yes \
    profile="profile vpn.company.hr" send-initial-contact=no
/ip ipsec proposal
set [ find default=yes ] enc-algorithms=3des
add auth-algorithms=sha512,sha256,sha1 enc-algorithms="aes-256-cbc,aes-256-ctr\
    ,aes-256-gcm,aes-192-ctr,aes-192-gcm,aes-128-cbc,aes-128-ctr,aes-128-gcm" \
    lifetime=8h name="proposal vpn.company.hr" pfs-group=none
/ip pool
add name=vpn_ipsec-pool ranges=192.168.43.10-192.168.43.254
/ip ipsec mode-config
add address-pool=vpn_ipsec-pool address-prefix-length=32 name=\
    "modeconf vpn.company.hr" split-include=192.168.0.0/24,10.0.0.0/8 \
    static-dns=192.168.0.10 system-dns=no
/ip firewall filter
add action=accept chain=forward connection-nat-state=dstnat
add action=accept chain=input comment=\
    "accept related and established connections" connection-state=\
    established,related,untracked
add action=drop chain=input comment="drop Invalid connections" \
    connection-state=invalid log-prefix=DROP_INVALID
add action=accept chain=input comment="accept IPSec ports" dst-address=\
    78.134.209.170 dst-port=500,4500 protocol=udp
add action=accept chain=input comment="accept IPSec ESP procotol" \
    dst-address=78.134.209.170 protocol=ipsec-esp
add action=accept chain=input comment="Allow VPN traffic to this router" \
    ipsec-policy=in,ipsec src-address=192.168.43.0/24
add action=fasttrack-connection chain=forward connection-mark=!ipsec \
    connection-state=established,related
add action=accept chain=forward comment=\
    "accept related and established connections" connection-state=\
    established,related,untracked
add action=accept chain=forward comment="DEFAULT: Accept In IPsec policy." \
    ipsec-policy=in,ipsec
add action=accept chain=forward comment="DEFAULT: Accept Out IPsec policy." \
    ipsec-policy=out,ipsec
add action=accept chain=forward comment="Allow VPN traffic to LAN network" \
    dst-address=192.168.0.0/24 ipsec-policy=in,ipsec src-address=\
    192.168.43.0/24
add action=accept chain=forward comment="Allow VPN traffic to PARTNER network" \
    dst-address=10.0.0.0/8 ipsec-policy=in,ipsec src-address=192.168.43.0/24
add action=drop chain=forward comment="Allow VPN traffic to this router" \
    ipsec-policy=in,ipsec src-address=192.168.43.0/24
add action=drop chain=forward comment="drop invalid connections" \

/ip firewall mangle
add action=mark-connection chain=forward comment="Mark IPsec" ipsec-policy=\
    out,ipsec new-connection-mark=ipsec
add action=mark-connection chain=forward comment="Mark IPsec" ipsec-policy=\
    in,ipsec new-connection-mark=ipsec
add action=change-mss chain=forward ipsec-policy=in,ipsec new-mss=1350 \
    passthrough=yes protocol=tcp src-address=192.168.43.0/24 tcp-flags=syn \
    tcp-mss=!0-1350
add action=change-mss chain=forward dst-address=192.168.43.0/24 ipsec-policy=\
    out,ipsec new-mss=1350 passthrough=yes protocol=tcp tcp-flags=syn \
    tcp-mss=!0-1350

/ip firewall nat
add action=masquerade chain=srcnat comment="NAT kroz Metronet" out-interface=\
    ETH1-METRONET
add action=masquerade chain=srcnat comment="NAT kroz T-com" out-interface=\
    "T-COM PPPoE"4
add action=masquerade chain=srcnat comment="NAT PARTNER" out-interface=ETH3-PARTNER

/ip ipsec identity
add auth-method=digital-signature certificate=vpn.company.hr \
    generate-policy=port-strict match-by=certificate mode-config=\
    "modeconf vpn.company.hr" peer=peer policy-template-group=\
    "group vpn.company.hr" remote-certificate=\
    firstname.lastname@company.hr remote-id=\
    user-fqdn:firstname.lastname@company.hr

/ip ipsec policy
set 0 group="group vpn.company.hr" proposal="proposal vpn.company.hr"
add dst-address=192.168.43.0/24 group="group vpn.company.hr" proposal=\
    "proposal vpn.company.hr" src-address=0.0.0.0/0 template=yes

route print on my PC is fine.

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
          0.0.0.0          0.0.0.0     192.168.10.2   192.168.10.100     55
         10.0.0.0        255.0.0.0         On-link    192.168.43.254     46
   10.255.255.255  255.255.255.255         On-link    192.168.43.254    301
   78.134.209.170  255.255.255.255     192.168.10.2   192.168.10.100     56
        127.0.0.0        255.0.0.0         On-link         127.0.0.1    331
        127.0.0.1  255.255.255.255         On-link         127.0.0.1    331
  127.255.255.255  255.255.255.255         On-link         127.0.0.1    331
      192.168.0.0    255.255.255.0         On-link    192.168.43.254     46
    192.168.0.255  255.255.255.255         On-link    192.168.43.254    301
     192.168.10.0    255.255.255.0         On-link    192.168.10.100    311
   192.168.10.100  255.255.255.255         On-link    192.168.10.100    311
   192.168.10.255  255.255.255.255         On-link    192.168.10.100    311
     192.168.15.0    255.255.255.0         On-link      192.168.15.1    291
     192.168.15.1  255.255.255.255         On-link      192.168.15.1    291
   192.168.15.255  255.255.255.255         On-link      192.168.15.1    291
     192.168.43.0    255.255.255.0         On-link    192.168.43.254     46
   192.168.43.254  255.255.255.255         On-link    192.168.43.254    301
   192.168.43.255  255.255.255.255         On-link    192.168.43.254    301
     192.168.56.0    255.255.255.0         On-link      192.168.56.1    291
     192.168.56.1  255.255.255.255         On-link      192.168.56.1    291
   192.168.56.255  255.255.255.255         On-link      192.168.56.1    291
        224.0.0.0        240.0.0.0         On-link         127.0.0.1    331
        224.0.0.0        240.0.0.0         On-link    192.168.10.100    311
        224.0.0.0        240.0.0.0         On-link      192.168.56.1    291
        224.0.0.0        240.0.0.0         On-link      192.168.15.1    291
        224.0.0.0        240.0.0.0         On-link    192.168.43.254    301
  255.255.255.255  255.255.255.255         On-link         127.0.0.1    331
  255.255.255.255  255.255.255.255         On-link    192.168.10.100    311
  255.255.255.255  255.255.255.255         On-link      192.168.56.1    291
  255.255.255.255  255.255.255.255         On-link      192.168.15.1    291
  255.255.255.255  255.255.255.255         On-link    192.168.43.254    301
===========================================================================

sindy · October 16, 2020, 7:16pm

It still smells like an MTU problem - ping are small packets, ssh are small packets most of the time, but http and RDP are large ones. Any VPN eats some bytes of the MTU for its ovehead, so the payload packets must be smaller to fit.

I can see you are forcing the MSS to 1350, but it may still be too large. And it’s not MTU, it’s MSS, so even smaller - try with 1200 for starters and if it helps, you can increase to find the optimum. I hazily remember I’ve ended up with 1280 somewhere.

But it should also be possible to make this work automatically the intended way - since TCP avoids fragmentation of the transport packets by splicing the payload flow into smaller pieces, it uses Path MTU Discovery mechanism, i.e. it marks the packets it sends with a “Don’t Fragment” flag, so the router where they don’t fit to the MTU of the outgoing interface sends back an ICMP message indicating the fact that the packet didn’t fit and what is the acceptable size, and the sender creates a new packet with a smaller portion of the payload and sends it again. This can repeat several times until the section of the path with the smallest MTU is crossed.

So the key is that the ICMP feedback wasn’t blocked anywhere. I can see connection-state=related to be allowed in chain forward of your firewall filter, but something may be blocking the ICMP notifications outside the Mikrotik.

skrgahr · October 16, 2020, 8:00pm

Thank you!
It works on MTU 1200. I will increase until it’s stable. It looks like it’s stable on 1280 as well.
I thought 1350 was very low and there is no need to go lower than that.

How to do part 2 you wrote?

sindy · October 17, 2020, 9:11am

It may be difficult in your case as part of the network is not under your administration.

Your configuration export only shows the VPN settings facing your IKEv2 clients. Static IPsec policies that include remote peer’s internal address in their dst-address range divert ICMP “fragmentation needed” messages sent by the Mikrotik itself to the client, and a policy with action=none preventing this from happening has to be added in such scenarios, but this cannot be your case here.

You wrote you’ve got a VPN to your business partner, so I suppose it is built using some other router. So check the firewall and the VPN settings on that other router; if everything seems fine there, you have to talk to the network administrator at the business partner company and resolve the issue with them. Some people tend to block ICMP without really understanding its essential role in the network functionality, so maybe it’s this case. Traffic sniffing along the path helps a lot to identify the device which blocks the icmp but may not be possible on some routers without touching the physical interconnections.

E.g. if sniffing at ETH3-PARTNER shows that large packets arrive from the partner network when a road warrior client tries to open a HTTP(S) connection and the Mikrotik sends the ICMP “fragmentation needed” back, it means that the MTU bottleneck is the IKEv2 connection to the road warriors, and those ICMP “fragmentation needed” notifications are blocked somewhere on the way to the server in partner’s network. This investigation has to be done with the change-mss rules disabled of course.

skrgahr · October 19, 2020, 7:25am

Thank you!

VPN with our client it’s not under my control. So, this is off. It works with MTU 1320.

Another question…
When I connect with windows, it works fine. However, when I connect with Android client, policy looks like this and connection works only on subnet 192.168.0.0/24.

I tried with native client and StrongSwan.

When I use windows, src address is 0.0.0.0/0.

Why is that and how to fix it?

sindy · October 19, 2020, 7:38am

It’s due to the different handling of split-include. Windows ignore split-include prefixes in the mode-config data (or maybe Mikrotik doesn’t even attempt to send them if it finds out that the peer is a Windows machine, no idea) and negotiate a policy with 0.0.0.0/0 at remote side; then, they use a DHCPINFORM message to request the route list from the peer, and Mikrotik delivers the route list this way (using DHCP Option 249).
Whether Strongswan can be configured to accept multiple prefixes in split-include and create a policy for each is out of my current knowledge.
What Android version do you have that its native client supports IKEv2?

skrgahr · October 19, 2020, 8:00am

I have Android 10 on Samsung Galaxy S10+.
There is type IPSec IKEv2 RSA. My colleague has some lower version, and he doesn’t have RSA, only XAuth and Hybrid.

Strongswan has “Split tunneling custom subnets”, but it does not generate all policies, there is only one, first one configured in mode configs.

sindy · October 19, 2020, 9:44am

It looks to me as a mismatch of concepts. The StrongSwan acting as initiator only requests a single traffic selector, 0.0.0.0/0->0.0.0.0/0, even if you configure the Custom subnets in Split tuneling under Advanced settings to the same list like on Mikrotik’s mode-config row. The Mikrotik responder then restricts this traffic selector to the one for the first subnet in the split-include list and sends an ADDITIONAL_TS_POSSIBLE notification. As use of split-include leads to successful establishment of multiple SAs between two Mikrotiks, I assume that the initiator is allowed to try to add another SA, again proposing a 0.0.0.0/0->0.0.0.0/0 traffic selector, and the Mikrotik responder restricts it to the next prefix on the mode-config list, and this process repeats until all prefixes are covered (so then the ADDITIONAL_TS_POSSIBLE is not sent any more). But as said that’s just a speculation, I haven’t tried that.

What you can do, on top of asking Herr Steffen to agree with Mikrotik on what reaction to ADDITIONAL_TS_POSSIBLE they expect, is not to specify any split-include on the mode-config row(s) used for the StrongSwan clients, let them configure the prefix list at their end manually (so that they would keep bypassing the VPN for any destination addresses outside the listed subnets), and use firewall rules at your end to prevent them from accessing anything but the listed subnets if they fail to configure it properly at their end (don’t forget about firewall filter chain input!)

Unlike in case of /ppp profile, the src-address-list item on the mode-config row only works at initiator, for a specific purpose, so you have to manually populate a src-address-list in your firewall rules to match the individual addresses (or a pool) you assign to the StrongSwan clients.

skrgahr · October 19, 2020, 10:57am

I have configured another mode config without split tunneling and created new identities using different certificates for mobile purposes, so all clients get 0.0.0.0/0.
In Strongswan we defined which subnets we want to use and it works

Thank you!

skrgahr · October 20, 2020, 6:54am

another question..
Mobile clients get 0.0.0.0/0 as default route, and I’m dropping packets where destination is not in VPN networks in input chain, so they can’t go on internet through Mikrotik.
If we don’t use Strongswan and we don’t configure custom subnets, then internet stops working. This happens with native iOS client, there is no possibility to configure subnets.

I now changed action to “return”. so first packet tries to go through mikrotik, and then it goes to another route. Is this a good solution?

sindy · October 20, 2020, 8:05am

It seems you have some gaps in understanding how the firewall works.

chain input handles packets whose destination is the router itself. Chain forward is for packets which transit through the router from one interface to another.
action=return does not return the packet to the sender as you might expect from the name. It is intended to prematurely finish processing of a packet in a custom chain and return the processing to the invocating chain (which normally happens if the last rule in the custom chain is reached and none of the rules has provided final verdict); if used in a built-in chain like input, I assume it has the same effect as action=accept.
if action=drop in chain=input had an effect of blocking internet, it must have been because the router was advertised as a DNS server to the clients, and blocking access to DNS caused that “internet stopped working” from the user perspective.

If there is no way to categorize traffic by any means with the iOS clients, there is no other way than to allow internet access via your site for them, with all the obvious consequences.

skrgahr · October 23, 2020, 8:58am

I have added return action in forward chain for ipsec policy where destination is out of defined subnets. It works fine. iOS users try to go through vpn, packet gets rejected, and it goes through next route.
It may not be best solution, but it works for a few days now.

sindy · October 23, 2020, 10:53am

Show me the actual firewall rules you use, please. It makes little sense this way.

skrgahr · October 25, 2020, 11:48am

/ip firewall address-list
add address=10.0.0.0/8 list=vpn_network
add address=192.168.0.0/24 list=vpn_network
add address=194.242.41.0/24 list=vpn_network
/ip firewall filter
add action=accept chain=forward connection-nat-state=dstnat
add action=accept chain=input comment="accept related and established connections" connection-state=established,related,untracked
add action=drop chain=input comment="drop Invalid connections" connection-state=invalid log-prefix=DROP_INVALID
add action=accept chain=input comment="accept IPSec ports" dst-address=78.134.209.170 dst-port=500,4500 protocol=udp
add action=accept chain=input comment="accept IPSec ESP procotol" dst-address=78.134.209.170 protocol=ipsec-esp
add action=accept chain=input comment="Allow VPN traffic to this router" ipsec-policy=in,ipsec src-address=192.168.43.0/24
add action=return chain=forward comment="Drop VPN traffic outside IS and PARTNER network" dst-address-list=!vpn_network ipsec-policy=in,ipsec log-prefix=DROP-NON-VPN src-address=192.168.43.0/24
add action=fasttrack-connection chain=forward connection-mark=!ipsec connection-state=established,related
add action=accept chain=forward comment="accept related and established connections" connection-state=established,related,untracked
add action=accept chain=forward comment="DEFAULT: Accept In IPsec policy." ipsec-policy=in,ipsec
add action=accept chain=forward comment="DEFAULT: Accept Out IPsec policy." ipsec-policy=out,ipsec
add action=accept chain=forward comment="Allow VPN traffic to IS and PARTNER" dst-address-list=vpn_network ipsec-policy=in,ipsec src-address=192.168.43.0/24
add action=drop chain=forward comment="drop invalid connections" connection-state=invalid log-prefix=DROP_INVALID protocol=tcp
/ip firewall mangle
add action=mark-connection chain=forward comment="Mark IPsec" ipsec-policy=out,ipsec new-connection-mark=ipsec
add action=mark-connection chain=forward comment="Mark IPsec" ipsec-policy=in,ipsec new-connection-mark=ipsec
add action=change-mss chain=forward ipsec-policy=in,ipsec new-mss=1320 passthrough=yes protocol=tcp src-address=192.168.43.0/24 tcp-flags=syn tcp-mss=!0-1320
add action=change-mss chain=forward dst-address=192.168.43.0/24 ipsec-policy=out,ipsec new-mss=1320 passthrough=yes protocol=tcp tcp-flags=syn tcp-mss=!0-1320

sindy · October 25, 2020, 12:57pm

Two points.

as I’ve suggested earlier, action=return in a built-in chain has the same effect like action=accept. You’ve made me test it
From security perspective this is a much better outcome than the other one, which I was afraid of - that the iOS would retry connections via internet if connection setup via VPN failed.
So you have effectively permitted the iOS to connect to internet via your router. There seems to be no nice way around this. Some people suggest use of dst-nat, creating a contigous subnet from all the fragmented ones, but this would require to provide the VPN clients with dedicated DNS responses, as I suppose the VPN clients use fqdns, not IP numbers, to access the hosts in the enterprise networks.
have a look at the principle of a stateful firewall. Except very special cases, there is no need to place any rules before the “accept established,related” in each chain, as this rule never matches on the very first packet of a new connection anyway.
So your currently first two rules in forward chain of filter may as well be placed after the currently fourth one, the overall effect on the packets and connections will be exactly the same, but every single mid-connection packet of all connections which are not fasttracked will only be inspected by two rules, not four.
On the same subject (firewall efficiency): as you use the connection-mark exclusively to prevent the packets matching to IPsec policies’ traffic matchers from triggering fasttracking of the connections they are part of, you can use fewer rules by replacing the “accept if connection-mark=ipsec” one in filter/forward and the two action=mark-connection ones from mangle/forward by placing two ones which are currently placed after the “accept established,related” one, “accept if ipsec-policy=in,ipsec” and “accept if ipsec-policy=out,ipsec”, before the action=fasttrack-connection one. As a result, an average packet will traverse just 1.5 rules per average packet instead of 3. This is how the firewall is configured in the factory default settings.