Community discussions

MikroTik App
 
nostromog
Member Candidate
Member Candidate
Topic Author
Posts: 226
Joined: Wed Jul 18, 2018 3:39 pm

ikev2 mikrotik to mikrotik strange behaviour

Wed Mar 27, 2019 6:50 pm

I have set up a ikev2 network between mikrotiks, like this. The server has two networks,
one local 192.168.88.0/254 and one for the current vpn: 192.168.89.0/24. I'm setting a new
VPN (192.168.90.0/24) and want it to be used to access all three networks,

In the server, this is the result of /ip ipsec export hide-sensitive, plus the relevant parts of
/ip pool and /ip address:
/ip address
add address=192.168.88.1/24 comment=defconf interface=bridge network=192.168.88.0
add address=192.168.90.1/24 interface=bridge network=192.168.90.0
add address=192.168.89.1/24 interface=bridge network=192.168.89.0
/ip ipsec mode-config
add address-pool=vpn2 name=RW-cfg split-include=192.168.88.0/24,192.168.89.0/24,192.168.90.0/24
/ip ipsec peer
add exchange-mode=ike2 name=server passive=yes
/ip ipsec policy group
add name=RoadWarrior
/ip ipsec identity
add generate-policy=port-strict mode-config=RW-cfg my-id=fqdn:server.dynaddress.org peer=server policy-template-group=RoadWarrior
/ip ipsec policy
add dst-address=192.168.90.0/24 group=RoadWarrior src-address=192.168.88.0/24 template=yes
add dst-address=192.168.90.0/24 group=RoadWarrior src-address=192.168.89.0/24 template=yes
add dst-address=192.168.90.0/24 group=RoadWarrior src-address=192.168.90.0/24 template=yes
/ip pool
add name=vpn2 ranges=192.168.90.2-192.168.90.254

The clients I am using to test are other mikrotiks, travel routers, containing the following config:
/ip ipsec peer
add address=server.dynaddress.org exchange-mode=ike2 name=server send-initial-contact=no
/ip ipsec identity
add generate-policy=port-override mode-config=request-only my-id=fqdn:client.mine peer=server remote-id=fqdn:server.dynaddress.org
It apparently works; associations are quickly established, addresses assigned... but it has a big loss of packets, about 50%

While trying to find out why, I found something really strange: it looses about 50% of packets, unless they are fragmented. Fragmented packets pass through perfectly:
[admin@MikroTik] > /ping count=5 192.168.90.251
  SEQ HOST                                     SIZE TTL TIME  STATUS                                                              
    0 192.168.90.251                             56  64 28ms 
    1 192.168.90.251                                          timeout                                                             
    2 192.168.90.251                                          timeout                                                             
    3 192.168.90.251                             56  64 28ms 
    4 192.168.90.251                             56  64 91ms 
    sent=5 received=3 packet-loss=40% min-rtt=28ms avg-rtt=49ms max-rtt=91ms 

[admin@MikroTik] > /ping count=5 192.168.90.251 size=1500 
  SEQ HOST                                     SIZE TTL TIME  STATUS                                                              
    0 192.168.90.251                           1500  64 54ms 
    1 192.168.90.251                           1500  64 72ms 
    2 192.168.90.251                           1500  64 32ms 
    3 192.168.90.251                           1500  64 117ms
    4 192.168.90.251                           1500  64 99ms 
    sent=5 received=5 packet-loss=0% min-rtt=32ms avg-rtt=74ms max-rtt=117ms 
When packets are fragmented it looses 0%. I can try with 1422 (maximum unfragmented size) and it will
loose about 50%, and with 1423, and it will loose 0%.

Does the big packet loss make sense to anyone? is it a bug or a configuration problem? Also, why fragmented packets are transported without problem?

It is all very puzzling to me.
 
User avatar
emils
Forum Veteran
Forum Veteran
Posts: 906
Joined: Thu Dec 11, 2014 8:53 am

Re: ikev2 mikrotik to mikrotik strange behaviour

Thu Mar 28, 2019 10:31 am

What model routers are involved? Is hardware offloading used? Do you see anything suspicious under IPsec statistics?
 
nostromog
Member Candidate
Member Candidate
Topic Author
Posts: 226
Joined: Wed Jul 18, 2018 3:39 pm

Re: ikev2 mikrotik to mikrotik strange behaviour

Thu Mar 28, 2019 12:38 pm

What model routers are involved? Is hardware offloading used? Do you see anything suspicious under IPsec statistics?
The server is a hAP ac, running 6.44.1. For what I know it does not support hardware offloading. It is in a PPPOE based ISP. Under IPsec statistics there are a few non-zero items, but
I think they are due to previous reboots of some of the clients. They didn't grow during tests:
           in-state-invalid: 4
              out-no-states: 84
      out-state-mode-errors: 15

I'm testing with two different clients. One is a mAP Lite, inside NAT of the other one, a hAP ac^2, which is under CGNAT in a public ISP

The mAP Lite is running 6.44.1 also. Again I don't think it supports hardware offloading. Under IPsec statistics, same caveats as before:
               in-no-states: 5
              out-no-states: 15
The hAP ac^2 is running also 6.44.1, with hardware offloading. It shows the same behaviour as the other one. Disabling one does not change the behaviour of the other, I tried with each peer disabled by turns.
Under IPsec statistics:
               in-no-states: 79
There are several other l2tp/IPsec or 6to4/IPsec connections going on at the same time, both from other clients to the same server and from the mAP to other servers; apparently without interactions or problems, they all perform correctly. The hAP ac^2 does not have more connections.
 
User avatar
emils
Forum Veteran
Forum Veteran
Posts: 906
Joined: Thu Dec 11, 2014 8:53 am

Re: ikev2 mikrotik to mikrotik strange behaviour

Thu Mar 28, 2019 1:19 pm

Sounds very weird. I would try to locate the issue more precisely with packet sniffer. Ping is bidirectional traffic. With packet sniffer you could verify whether the packet is at least received on the other end. Also verify ESP or UDP/4500 packets are properly sent out and received.
 
nostromog
Member Candidate
Member Candidate
Topic Author
Posts: 226
Joined: Wed Jul 18, 2018 3:39 pm

Re: ikev2 mikrotik to mikrotik strange behaviour

Thu Mar 28, 2019 2:00 pm

Sounds very weird. I would try to locate the issue more precisely with packet sniffer. Ping is bidirectional traffic. With packet sniffer you could verify whether the packet is at least received on the other end. Also verify ESP or UDP/4500 packets are properly sent out and received.
Just setting /tool sniffer in the server makes the packet loss disappear, and all traffic is correctly seen while /tool sniffer { set filter-ip-protocol=icmp filter-operator-between-entries=and filter-ip-address=192.168.90.1; start}

Stopping it makes packet loss ~50% happen again
 
User avatar
emils
Forum Veteran
Forum Veteran
Posts: 906
Joined: Thu Dec 11, 2014 8:53 am

Re: ikev2 mikrotik to mikrotik strange behaviour

Thu Mar 28, 2019 2:05 pm

Must be caused by FastTrack. Exclude the traffic subject for IPsec processing from being FastTracked in firewall's forward chain by adding accept rules before the action=fasttrack-connection rule.
 
nostromog
Member Candidate
Member Candidate
Topic Author
Posts: 226
Joined: Wed Jul 18, 2018 3:39 pm

Re: ikev2 mikrotik to mikrotik strange behaviour

Thu Mar 28, 2019 7:54 pm

Must be caused by FastTrack. Exclude the traffic subject for IPsec processing from being FastTracked in firewall's forward chain by adding accept rules before the action=fasttrack-connection rule.
I'm not sure how to exclude this traffic.

I already have firewall rules (this is the beginning of /ip firewall filter export at the server:
/ip firewall filter
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=accept chain=forward comment="Bypass fasttrack for non-zero DSCP" connection-state=established,related dscp=!0
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
It was like this before, and it is not working. Where else do I need to act? Currently the only way I have found to have ikev2 working is to do /ip settings set allow-fast-path=no, which looks quite drastic...
 
User avatar
emils
Forum Veteran
Forum Veteran
Posts: 906
Joined: Thu Dec 11, 2014 8:53 am

Re: ikev2 mikrotik to mikrotik strange behaviour

Fri Mar 29, 2019 10:28 am

Can you post your whole firewall? After double checking, I see you are pinging from one router to the other directly and this traffic should not hit the forward chain at all. Do you have any other fasttrack related rules on your router?
 
nostromog
Member Candidate
Member Candidate
Topic Author
Posts: 226
Joined: Wed Jul 18, 2018 3:39 pm

Re: ikev2 mikrotik to mikrotik strange behaviour

Fri Mar 29, 2019 1:06 pm

It is precisely this rule that causes the problem.

If I disable it, no packet loss, if I enable it, packet loss ~ 40-60%
[admin@MikroTik] > /ip firewall filter print where action=fasttrack-connection 
Flags: X - disabled, I - invalid, D - dynamic 
 0 X  ;;; defconf: fasttrack
      chain=forward action=fasttrack-connection connection-state=established,related 
Note that my ISP connection is a pppoe-client one, running on ether1. Not sure if it is relevant.


Again, not sure if it is related, but I'm using this: viewtopic.php?t=132823
I took care to specify allow-fast-path=no there

I don't understand how it can be, but it is fully reproducible: either /ip settings set allow-fast-path=no or disabling this firewall rule (or start sniffer...) and the connection works ok; allow fast-path and enable the rule, and traffic loss appears...

Who is online

Users browsing this forum: Ahrefs [Bot], Amazon [Bot], Bing [Bot], CGGXANNX, Google [Bot], patrikg and 70 guests