wrong ipip tunnel encapsulation when used ipsec

RouterOS General
1

Hello

I try link 2 mikrotik routes (RB750Gr3 and RBD52G-5HacD2HnD-TC, all routers works on ROUTEROS 7.7) with ipsec for security and over ipsec tunnel i build ipip tunnel, and when we all configured almost works, traffic between routes over ipip tunnel was passed RIP routes on each sides was announced so everything works:

our configuration(we use link local addresses for ipsec tunnel and ipip over ipsec tunnel)

169.254.0.2/32 - router01 ipsec tunnel address
169.254.0.1/32 - router02 ipsec tunnel address

169.254.1.10/30 - router01 ipip over ipsec tunnel address
169.254.1.8/30 - router02 ipip over ipsec tunnel address

both address local and remote was provided for ipip tunnel, keepalive is off

For stress test we begin reboot routers, and everything broke, but broke only ipip tunnel, because when we ping ipsec address of each side of tunnel its works:

[admin@MikroTik] > ping src-address=169.254.0.2 169.254.0.1 
  SEQ HOST                                     SIZE TTL TIME       STATUS                                                                                                                                                                                                                                           
    0 169.254.0.1                                56  64 61ms382us 
    1 169.254.0.1                                56  64 63ms641us 
    2 169.254.0.1                                56  64 49ms207us 
    3 169.254.0.1                                56  64 62ms691us 
    sent=4 received=4 packet-loss=0% min-rtt=49ms207us avg-rtt=59ms230us max-rtt=63ms641us

in our investigation we found that ipip traffic encapsulated in wrong ip packet, here one packet when we ping other side of ipiptunell i.e. ping 169.254.1.8 from router01:

Frame 7: 139 bytes on wire (1112 bits), 139 bytes captured (1112 bits) on interface udpdump, id 0
EXPORTED_PDU
TZSP: Ethernet 
Ethernet II, Src: 00:00:00_00:00:00 (00:00:00:00:00:00), Dst: 00:00:00_00:00:00 (00:00:00:00:00:00)
Internet Protocol Version 4, Src: 92.101.161.39, Dst: 169.254.0.1
Internet Protocol Version 4, Src: 169.254.1.10, Dst: 169.254.1.8
Internet Control Message Protocol

as you can see src address of final ip packet Src: 92.101.161.39, but its absolutely wrong because it must be 169.254.0.2(ip sec address of router01 side of ipip tunnel)
first we thinked about is NAT, but why in such case working direct ping between ipsec addresses, also in our firewall was set not track for ipsec adresses in raw table, so NAT can’t be a reason this strange behaviour

Now we think that this is a bug in ROUTEROS 7.7, or we miss something…

2

After some experiments and thoughts we found that if ipip tunnels was disabled, before ipsec established connections, and after that we again enable this tunnels, everything begin works,
so conclusion is that ipip tunnel must be up strongly after ipsec is established, we think that present some kind of cache that will broke all functionally(after reboot we probably got some kind of race condition)

and this is not surprising since on a linux router we do exactly that


So is any way in routeros enable ipip(maybe gre) tunnel strongly after ipsec connection is established and vice versa - disable ipip tunnel before down ipsec connection?

ipip tunnel in routeros have a setting ipsecsecret that seems solve this problem, be it doesn’t work with ike2, not support certificates, and responder peer (короче устарело как г..но мамонта), so doesn’t acceptable for us

3

Final result

The real reason was in fasttrack bypass, to prevent this we add firewall rule

src: 169.254.0.0/16 dst: 169.254.0.0/16 no-track

and thinked that it works,but after some time we noticed, that ipip tunnel connection pass source nat table(in connection tracking table), when we remove this connection it return to source nat table(natted) despite ipsec tunnel up or down(ie some kind of cache),

after that we saw that after reboot out rule that must prevent ipsec traffic from fasttrack moved to the end of the raw table and of course it didn’t work,

so as a workaround we add follow rule to the nat table(the mean of this trick bypass fasttrack before ipsec tunnel is up, and no-track-chain rules of tunnel is applied)

[admin@MikroTik] > /ip/firewall/nat/print  
Flags: X - disabled, I - invalid; D - dynamic 
 0    chain=srcnat action=accept src-address=169.254.0.0/16 dst-address=169.254.0.0/16 log=no log-prefix="" 

 1    ;;; defconf: masquerade
      chain=srcnat action=masquerade out-interface-list=WAN ipsec-policy=out,none

this rule bypass fastrack and doesn’t moved after reboot, so this is a solution in our case.

I doesn’t make sure that moving rule in raw firewall table is bug, but I suppose that this is a bug