IPSEC IKEv2 network-to-network problems

nagylzs · December 27, 2020, 3:01pm

I would like to create a site-to-site connection with IPSEC IKEv2. The connection should connecte two internal networks, as shown below:

https://imgur.com/a/dRV3TR1

The real IP addresses have been replaced with 1.2.3.188 and 1.2.3.161.

I could already setup the two routers. I’m going to post (most of) the configurations at the end of this post.

Here are some problems:

I can ping the remote router 192.168.13.254 from the computer 192.168.14.2.
But I cannot ping the remote device 192.168.13.252 from the computer 192.168.14.2
From the routers, I cannot ping anything that is on the other side of the tunnel. E.g. I cannot ping 192.168.14.1 from 192.168.13.254 and vice versa.

In the begining, I was using policy templates. Later I switched to hand written policies because I wanted to understand what is happening and why.
Using the current config, IKE phase1 and phase2 are completed, there are SAs installed and the hand written policies are shown as active:

Flags: T - template, X - disabled, D - dynamic, I - invalid, A - active, * - default
 #     PEER  TUN SRC-ADDRESS                                    DST-ADDRESS
 0 TX*           0.0.0.0/0                                      0.0.0.0/0
 1  A  pee.. yes 192.168.13.0/24                                192.168.14.0/24

There will be various servers and workstations connected to both ends, and I need to be able to reach them from both sides. For example, if somebody asks for help in the office but I’m currently in branch01, then I want to be able to look at her screen with VNC. In other words, I do not want to put these networks behind NAT. I only need this tunnel to create a secure connection between the two networks so that I can route packets between them. (The end goal is to be able to add branch02 and branch03 etc. but I think I’m far away from that.)

Config for the initiator side

Initiator side

/ip address
add address=192.168.14.1/24 comment=defconf interface=ether2 network=192.168.14.0
/interface bridge
add admin-mac=AA:BB:CC:DD:EE:FF arp=proxy-arp auto-mac=no comment=defconf name=bridge
/interface bridge port
add bridge=bridge comment=defconf interface=ether2
add bridge=bridge comment=defconf interface=ether3
add bridge=bridge comment=defconf interface=ether4
add bridge=bridge comment=defconf interface=ether5

/ip ipsec mode-config
add name=modeconf-branch01 responder=no
/ip ipsec policy group
add name=group-branch01
/ip ipsec profile
add dh-group=modp2048 enc-algorithm=aes-256 hash-algorithm=sha256 name=profile-branch01
/ip ipsec peer
add address=1.2.3.188/32 exchange-mode=ike2 name=peer-remote-office profile=profile-branch01
/ip ipsec proposal
set [ find default=yes ] auth-algorithms=sha512,sha256,sha1 enc-algorithms=aes-256-cbc,aes-128-cbc pfs-group=modp2048
add auth-algorithms=sha256 comment=none enc-algorithms=aes-256-cbc name=proposal-branch01 pfs-group=modp2048
/ip ipsec identity
add auth-method=digital-signature certificate=cert_export_laci@vpn.office.my.server.com.p12_0 generate-policy=port-strict mode-config=\
    modeconf-branch01 peer=peer-remote-office policy-template-group=group-branch01
/ip ipsec policy
set 0 dst-address=0.0.0.0/0 src-address=0.0.0.0/0
add dst-address=192.168.13.0/24 peer=peer-remote-office proposal=proposal-branch01 sa-dst-address=1.2.3.188 sa-src-address=\
    0.0.0.0 src-address=192.168.14.0/24 tunnel=yes

/ip firewall filter
add action=accept chain=input port=1701,500,4500 protocol=udp
add action=accept chain=input protocol=ipsec-esp
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input src-address=192.168.13.0/24
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN

add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="accept out ipsec policy" ipsec-policy=out,ipsec
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=accept chain=forward src-address=192.168.14.0/24
add action=accept chain=forward src-address=192.168.13.0/24
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf:  drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN

/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN

Responder side

/ip address
add address=192.168.13.254/24 comment=defconf interface=ether2 network=192.168.13.0
/interface bridge
add admin-mac=11:22:33:44:55:66 auto-mac=no comment=defconf name=bridge
/interface bridge port
add bridge=bridge comment=defconf interface=ether2
add bridge=bridge comment=defconf interface=ether3
add bridge=bridge comment=defconf interface=ether4
add bridge=bridge comment=defconf interface=ether5

/ip ipsec mode-config
add address=10.0.88.1 address-prefix-length=32 name=modeconf-office static-dns=192.168.13.254 system-dns=no
/ip ipsec policy group
add name=group-office
/ip ipsec profile
add dh-group=modp2048 enc-algorithm=aes-256 hash-algorithm=sha256 name=profile-office
/ip ipsec peer
add exchange-mode=ike2 local-address=1.2.3.188 name=peer-local-office passive=yes profile=profile-office
/ip ipsec proposal
set [ find default=yes ] auth-algorithms=sha512,sha256,sha1 enc-algorithms=aes-256-cbc,aes-128-cbc pfs-group=modp2048
add auth-algorithms=sha256 enc-algorithms=aes-256-cbc lifetime=4h name=proposal-office pfs-group=modp2048
/ip ipsec identity
add auth-method=digital-signature certificate=vpn.office.my.server.com generate-policy=port-strict match-by=certificate mode-config=\
    modeconf-office peer=peer-local-office policy-template-group=group-office remote-certificate=laci@vpn.office.my.server.com \
    remote-id=user-fqdn:laci.vpn.office.my.server.com
/ip ipsec policy
set 0 disabled=yes dst-address=0.0.0.0/0 src-address=0.0.0.0/0
add dst-address=192.168.14.0/24 peer=peer-local-office proposal=proposal-office sa-dst-address=1.2.3.161 sa-src-address=\
    1.2.3.188 src-address=192.168.13.0/24 tunnel=yes

/ip firewall filter

add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="Allow UPD 500,4500 for IPSec for vpn.office.my.server.com" dst-address=1.2.3.188 dst-port=500,4500 protocol=udp
add action=accept chain=input comment="Allow IPSec-ESP for vpn.office.my.server.com" dst-address=1.2.3.188 protocol=ipsec-esp
add action=accept chain=input ipsec-policy=in,ipsec src-address=192.168.14.0/24
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN

add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=accept chain=forward src-address=192.168.13.0/24
add action=accept chain=forward src-address=192.168.14.0/24
add action=drop chain=forward comment="defconf:  drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid

/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN

erkexzcx · December 27, 2020, 3:20pm

Not what you are asking, but it might give you some hints: http://forum.mikrotik.com/t/mikrotik-behind-nat-to-mikrotik-ipsec-ike2-with-certs-tunnel-eoip/144952/1

nagylzs · December 27, 2020, 3:27pm

Oh, and here is one thing that I don’t understand. According to the documentation ( https://wiki.mikrotik.com/wiki/Manual:IP/IPsec#Policies ) the sa-src-address and sa-dst-address properties are read only. But if I do “/ip ipsec policy export” then they are exported!

So maybe they are not read only?

I’m totally confused, because I tried to change sa-src-address for my policy on the initiator side, and this is what happened:

[user@laci.my.server.com] /ip ipsec policy> print detail
Flags: T - template, X - disabled, D - dynamic, I - invalid, A - active, * - default
 0 T * group=default src-address=0.0.0.0/0 dst-address=0.0.0.0/0 protocol=all proposal=default template=yes

 1  A  peer=peer-remote-office tunnel=yes src-address=192.168.14.0/24 src-port=any dst-address=192.168.13.0/24
       dst-port=any protocol=all action=encrypt level=require ipsec-protocols=esp sa-src-address=0.0.0.0
       sa-dst-address=1.2.3.188 proposal=proposal-branch01 ph2-count=1
[user@laci.my.server.com] /ip ipsec policy> set 1 sa-src-address=1.2.3.161
[user@laci.my.server.com] /ip ipsec policy> print detail
Flags: T - template, X - disabled, D - dynamic, I - invalid, A - active, * - default
 0 T * group=default src-address=0.0.0.0/0 dst-address=0.0.0.0/0 protocol=all proposal=default template=yes

 1     peer=peer-remote-office tunnel=yes src-address=192.168.14.0/24 src-port=any dst-address=192.168.13.0/24
       dst-port=any protocol=all action=encrypt level=require ipsec-protocols=esp sa-src-address=0.0.0.0
       sa-dst-address=1.2.3.188 proposal=proposal-branch01 ph2-count=0
[user@laci.my.server.com] /ip ipsec policy>

It did not throw an error, but it also did not change the property. It seems to be read only. But in that case, why is it appearing in the export?

nagylzs · December 27, 2020, 3:32pm

I have heard about EoIP but I never tried. Since EoIP emulates an ethernet wire, it might forward all broadcast packets? I won’t ask more questions about this before I try EoIP myself. Thank you for the tip!

nagylzs · December 27, 2020, 3:40pm

Well, the manual page for EoIP is very short. My concern was justified. EoIP tunnels will forward broadcast packets. /interface gre could also be used.

nescafe2002 · December 27, 2020, 8:38pm

Running EoIP to link two sites and then deal with the undesired effects (broadcasts, same subnet) is.. not the best advice imo.

Just plain ipsec tunneling should work fine. To start, get rid of all the custom proposal and profile and use static peer for tunneling.

Default firewall needs no adjustments for ipsec tunneling to work.

Initiator:

/ip ipsec peer
add address=1.2.3.188/32 exchange-mode=ike2 name=peer-remote-office
/ip ipsec identity
add auth-method=digital-signature certificate=cert_export_laci@vpn.office.my.server.com.p12_0 \
    peer=peer-remote-office
/ip ipsec policy
add dst-address=192.168.13.0/24 peer=peer-remote-office src-address=192.168.14.0/24 tunnel=yes

Responder:

/ip ipsec peer
add exchange-mode=ike2 name=peer-local-office passive=yes
/ip ipsec identity
add auth-method=digital-signature certificate=vpn.office.my.server.com match-by=certificate \
    peer=peer-local-office remote-certificate=laci@vpn.office.my.server.com \
    remote-id=user-fqdn:laci.vpn.office.my.server.com
/ip ipsec policy
add dst-address=192.168.14.0/24 peer=peer-local-office src-address=192.168.13.0/24 tunnel=yes

Note that both sides can be both initiator and responder to make config even easier / common between the two.

Communication from the device to the other end will not be picked up because based on routing tables, the other end is routed via WAN and it will pick the WAN address as source.
To overcome this, create a dummy route for the remote subnet to local bridge (which will not be used for routing since the ipsec policy has priority):

Initiator:

/ip route
dst-address=192.168.13.0/24 gateway=bridge

Responder:

/ip route
dst-address=192.168.14.0/24 gateway=bridge

Your scenario is perfectly supported by plain ipsec, so no need for additional tunneling (gre, eoip) with additional overhead. Unless you need dynamic routing, bgp/ospf.

sindy · December 27, 2020, 8:46pm

It’s a historic hangover. Somewhere between 6.44 and 6.46, the handling of sa-src-address and sa-dst-address has changed from manual configuration to inheritance from the peer to which the policy is linked (before, the link was reverse, the peer was deternined by matching the sa-dst-address to the active peers’ addresses and the sa-src-address to the active peers’ local addresses). And for some reason these parameters are still exported.

nagylzs · December 30, 2020, 8:09pm

I’m sorry but you are mistaken here. After some research I realized that the default masquerade NAT rule was changing the src addresses of the packets BEFORE they were processed by the ipsec policy rules. The actual change required is below.

Old (deafult) NAT rules:

/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN

New (working) NAT rule on one side:

/ip firewall nat
add chain=srcnat src-address=192.168.13.254/24  dst-address=192.168.14.254/24  action=accept comment="bypass NAT for VPN tunnel between sites" 
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN

On the other side it is the same, only src-address and dst-address are reversed.

A little bit more explanation:

Switching, MPLS and routing are all processed BEFORE ipsec policies, as shown in https://wiki.mikrotik.com/wiki/Manual:Packet_Flow
Srcnat/masquerade is executed in layer 3 firewall
The layer 3 routing does not know about ipsec policies. So when it sees the packet that was supposed to be forwarded from 192.168.13.X to 192.168.14.Y then it looks for a route for the destination address. The closest route is through the default gateway. As far as the layer 3 routing is concerned, this packet will go out on the WAN interface and it matches the masquerade rule.
Masquerade action is executed, the original source address is replaced from 192.168.13.254/24 to the public/WAN address of the router.
By the time the packet leaves layer 3 routing, it is going from WAN => 192.168.14.Y instead of the original 192.168.13.X => 192.168.14.Y
Then ipsec policy rules are examined on the packet, but none of them will match. (We only have a policy for src-address=192.168.13.254/24, but nothing for src-address=WAN)
So the packet is not encapsulated at all, and it goes out to the WAN interface. Unencrypted!

Now I only need to clamp TCP/MSS values and I’m done.

Yes, it really works with ikev2/ipsec in tunnel mode alone. EoIP/GRE is not needed. I’m a happy camper.

nagylzs · December 30, 2020, 10:04pm

The solution that you suggested also works because it provides a fake route with a non-wan interface for layer 3 routing. This prevents matching the masquerade nat rule.

I do not like using dummy routes because it can be misleading. If you look at that route alone, then you might think that those packets are actually routed out on the wan interface. It has the same effect (bypass nat), but it does that by fooling the router, and possibly it’s administrator.

But I guess it is just my personal preference.

Thank you for your help!

nescafe2002 · December 31, 2020, 8:07am

Correct, they serve the same purpose.

I like to leave the default firewall alone, give the dummy route a higher distance and a comment regarding device initiated ipsec connections.

Also for unenstablished dynamic policies, the dummy route prevents unencrypted packets from leaving through wan. But setting up blackhole routes for RFC 1918 prefixes would work as well.

sindy · December 31, 2020, 10:22am

Routes with type=blackhole drop the packets, so IPsec policies cannot see them.

To prevent packets which should be delivered via an IPsec tunnels from leaking the wrong way, you have to use a route whose gateway is a specially created bridge with no member ports. And this is only necessary if the policy is created dynamically - a static policy matches the packets and diverts them even if the security association (tunnel) linked to it is currently down.

nagylzs · December 31, 2020, 11:22am

Wow Sindy, I always learn something new here! Given this explanation, I changed my mind. Creating a “vpn-blackhole” bridge and adding routes towards it seems to be even more unambiguous than adding NAT bypass rules. Originally I thought that if I add a dummy route then it might confuse me (possibly several months) later when I look at it. But if that route has a gateway named “vpn-blackhole” then its purpose cannot be misunderstood.

Although I have created these policies by hand, I will use dynamically generated policies in the future. A solution that works well in both cases is probably better, because it can be used on devices that has both static and dynamic policies.