Community discussions

MikroTik App
 
ddejager
Member Candidate
Member Candidate
Topic Author
Posts: 134
Joined: Tue Oct 18, 2011 5:13 am

L2TP tunnel usually fails to set up. Suggestions?

Wed Jan 05, 2022 2:48 am

I have one in service router (CRS125-24G-1S) that will not reliably establish an L2TP with an ipsec tunnel to other mikrotik routers. Only about 5% of the time will the tunnel get established. All other times the ISAKMP-SA is successfully set up, but the tunnel fails to authenticate, it just keeps trying. If I disable IPSEC on the L2TP tunnel it works.

I have other routers (RB951-UI) that I set up that can successfully establish an L2TP session. I've compared the IPSEC configuration for the working an non-working routers and they are identical. I've tried disabling fasttrack and all input firewall rules on the CRS router and the tunnel still fails to authenticate. If I set up an RB951-UI behind the CRS125, it can still establish a connection successfully through the CRS to the same target router, so the ISP is not a fault. When I look at the log on the target router, I see the ISAKMP-SA established, but then nothing, until I disable the outbound connection, then I get a message saying that the first L2TP packet has been received....so it is like the CRS is not sending any traffic until the session is disabled.

I've attached log samples when it works (rarely) and when it does not work. When it starts working, it will work multiple times successfully until I leave it down for over 10 minutes or so, then it will fail for days. That is why I suspected fasttrack, but disabling fasttract does not fix the problem.

Suggestions about how to debug or fix this are appreciated.
bad l2tp.png
OK l2tp.png
You do not have the required permissions to view the files attached to this post.
 
sindy
Forum Guru
Forum Guru
Posts: 10206
Joined: Mon Dec 04, 2017 9:19 pm

Re: L2TP tunnel usually fails to set up. Suggestions?

Wed Jan 05, 2022 10:43 am

Fasttracking only handles forwarded traffic (i.e. potentially, the payload of the IPsec tunnel), but not the own traffic of the router, such as the control session of the IPsec tunnel (IKE/IKEv2) and the IPsec transport packets.

What might be related is connection tracking as such, more likely at the CRS end than at the L2TP server end. Or there may be another L2TP/IPsec client of the same L2TP/IPsec server behind the same public address like the CRS itself, which simply cannot work if you don't take additional measures - see viewtopic.php?t=132823. Such additional measures are very easy for a Mikrotik client, though.
 
ddejager
Member Candidate
Member Candidate
Topic Author
Posts: 134
Joined: Tue Oct 18, 2011 5:13 am

Re: L2TP tunnel usually fails to set up. Suggestions?

Wed Jan 05, 2022 3:33 pm

Sindy,
Thanks for your reply. I can confirm, however, that there is no other L2TP/ipsec client connected to the CRS. The CRS serves my house and two neighbors. Also, as I mentioned if I hook a RB951UI to the CRS and use the RB951 as an L2TP/ipsec client through the CRS to the same destination L2TP/ipsec server the CRS, the connection comes up immediately and works correctly every time.
Last edited by ddejager on Wed Jan 05, 2022 4:04 pm, edited 1 time in total.
 
sindy
Forum Guru
Forum Guru
Posts: 10206
Joined: Mon Dec 04, 2017 9:19 pm

Re: L2TP tunnel usually fails to set up. Suggestions?

Wed Jan 05, 2022 3:44 pm

as I mentioned if I hook a RB951UI to the CRS and use the RB951 as an L2TP/ipsec client through to the same destination L2TP/ipsec server the CRS
It wasn't clear to me that this 951 was there only temporarily, nor that the public IP is on the CRS itself. E.g. if the CRS was connected via an LTE, your other device also connected via an LTE from an apparently unrelated place but using the same ISP could reuse the same public IP from that ISP's pool.

Anyway, to get a useful log, you have to do the following:

/system logging
add topics=ipsec,!packet
add topics=l2tp


This will cause the regular log buffer to overflow quite fast, so the best way is to disable the /interface l2tp-client, run /log print follow-only file=l2tp-ipsec-start topics~"ipsec|l2tp", enable the /interface l2tp-client, wait until it fails, break the /log print ..., download the file and start reading it.
 
ddejager
Member Candidate
Member Candidate
Topic Author
Posts: 134
Joined: Tue Oct 18, 2011 5:13 am

Re: L2TP tunnel usually fails to set up. Suggestions?

Wed Jan 05, 2022 6:22 pm

Sindy,
In looking at the detailed logs it appears that the IPsec tunnel is established (but I'm not positive since I don't know how to interpret all the log data). Then l2tp sends a SCCRQ to the remote host using port 1701. No reply is ever received so it keeps retrying, eventually gives up and try to establish a new IPsec tunnel, tries again, etc.

If I disable IPsec on the l2tp tunnel, I see that after the first SCCRQ message sent to the remote host, immediately I get back an SCCRP message and the tunnel gets established.

So I'm not sure why traffic is not flowing through the IPsec tunnel....
 
sindy
Forum Guru
Forum Guru
Posts: 10206
Joined: Mon Dec 04, 2017 9:19 pm

Re: L2TP tunnel usually fails to set up. Suggestions?  [SOLVED]

Wed Jan 05, 2022 7:02 pm

An IPsec "session" consists of the "control" security association (IKE in this L2TP/IPsec case) and the "transport" association(s) (a single one in this case). The fact that Phase 1 (IKE) has been established successfully doesn't guarantee that the Phase 2 (transport SA) establishes successfully afterwards, and even if they did in your case, there may have been the traffic selector conflict at the server side as described in the topic I've linked before. But normally each peer should receive a notification about any such failure from the remote one by means of the Phase 1 SA.

So there is yet another possibility - could it be that the CRS has a public IP on itself, so does the L2TP/IPsec server, and the firewall on the L2TP/IPsec server doesn't have any "accept ipsec-esp" rule in chain input of its /ip firewall filter? Because if there is no NAT at either end, the Phase 2 ESP transport packets are not encapsulated into UDP and thus they need their own pinhole (tracked connection) on the firewall. If all the other clients are NATed, they don't bump into this. But if this was the explanation, it would be really weird that it sometimes works.

Does the log show port numbers for the Phase 2 SA endpoints? If yes, there is NAT somewhere; if not, it may well be this "missing ESP rule" issue (except for the doubt above).
 
ddejager
Member Candidate
Member Candidate
Topic Author
Posts: 134
Joined: Tue Oct 18, 2011 5:13 am

Re: L2TP tunnel usually fails to set up. Suggestions?

Wed Jan 05, 2022 7:18 pm

So I think the IPsec is all set up properly. See the excerpt from my log before it starts repeatedly to establish l2tp.
09:27:54 ipsec,debug debug: KEYMAT computed. 
09:27:54 ipsec,debug debug: call pk_sendupdate 
09:27:54 ipsec,debug debug: encryption(aes-cbc) 
09:27:54 ipsec,debug debug: hmac(sha1) 
09:27:54 ipsec,debug debug: call pfkey_send_update_nat 
09:27:54 ipsec debug: IPsec-SA established: ESP/Transport my.ip.address[500]->remote.ip.address[500] spi=0x88111e7 
09:27:54 ipsec,debug debug: pfkey update sent. 
09:27:54 ipsec,debug debug: encryption(aes-cbc) 
09:27:54 ipsec,debug debug: hmac(sha1) 
09:27:54 ipsec,debug debug: call pfkey_send_add_nat 
09:27:54 ipsec debug: IPsec-SA established: ESP/Transport remote.ip.address[500]->my.ip.address[500] spi=0x5e8ab7e 
09:27:54 ipsec,debug debug: pfkey add sent. 
09:27:56 l2tp,debug,packet debug: sent control message to my.ip.address:1701 from remote.ip.address:1701 
09:27:56 l2tp,debug,packet debug:     tunnel-id=0, session-id=0, ns=0, nr=0 
09:27:56 l2tp,debug,packet debug:     (M) Message-Type=SCCRQ 
09:27:56 l2tp,debug,packet debug:     (M) Protocol-Version=0x01:00 
09:27:56 l2tp,debug,packet debug:     (M) Framing-Capabilities=0x1 
09:27:56 l2tp,debug,packet debug:     (M) Bearer-Capabilities=0x0 
09:27:56 l2tp,debug,packet debug:     Firmware-Revision=0x1 
09:27:56 l2tp,debug,packet debug:     (M) Host-Name="DeJager_Rtr" 
09:27:56 l2tp,debug,packet debug:     Vendor-Name="MikroTik" 
09:27:56 l2tp,debug,packet debug:     (M) Assigned-Tunnel-ID=151 
09:27:56 l2tp,debug,packet debug:     (M) Receive-Window-Size=4 
09:28:00 l2tp,debug,packet debug: sent control message to my.ip.address:1701 from remote.ip.address:1701 
09:28:00 l2tp,debug,packet debug:     tunnel-id=0, session-id=0, ns=0, nr=0 
09:28:00 l2tp,debug,packet debug:     (M) Message-Type=SCCRQ 
09:28:00 l2tp,debug,packet debug:     (M) Protocol-Version=0x01:00 
09:28:00 l2tp,debug,packet debug:     (M) Framing-Capabilities=0x1 
09:28:00 l2tp,debug,packet debug:     (M) Bearer-Capabilities=0x0 
09:28:00 l2tp,debug,packet debug:     Firmware-Revision=0x1 
09:28:00 l2tp,debug,packet debug:     (M) Host-Name="DeJager_Rtr" 
09:28:00 l2tp,debug,packet debug:     Vendor-Name="MikroTik" 
09:28:00 l2tp,debug,packet debug:     (M) Assigned-Tunnel-ID=151 
09:28:00 l2tp,debug,packet debug:     (M) Receive-Window-Size=4 


There is no input chain rule in the firewall on the server for accept ipsec-esp. But no such rule exists in the default configuration on the rb951 that works. Am I missing something?

Thanks much for your advice!
Last edited by ddejager on Wed Jan 05, 2022 7:42 pm, edited 2 times in total.
 
ddejager
Member Candidate
Member Candidate
Topic Author
Posts: 134
Joined: Tue Oct 18, 2011 5:13 am

Re: L2TP tunnel usually fails to set up. Suggestions?

Wed Jan 05, 2022 7:29 pm

By the way, the server and my CRS DO both have a public IP. There is another rb941 that does connect successfully from Kenya to the server using l2tp. The Kenya router is behind a NAT firewall in the LTE modem provided by Safaricomm. I will experiment with your accept rule to see if that fixes it.
 
sindy
Forum Guru
Forum Guru
Posts: 10206
Joined: Mon Dec 04, 2017 9:19 pm

Re: L2TP tunnel usually fails to set up. Suggestions?

Wed Jan 05, 2022 7:30 pm

The log says it clearly, except that it unfortunately shows ports where it should not (which is a bug, but since I almost never use IKE(v1), it is possible that I have not noticed it yet as it is not present in the log of IKEv2 sessions):

IPsec-SA established: ESP/Transport my.ip.address[500]->remote.ip.address[500] spi=0x88111e7
IPsec-SA established: ESP/Transport remote.ip.address[500]->my.ip.address[500] spi=0x5e8ab7e
There is no input chain rule in the firewall for accept ipsec-esp. But no such rule exists in the default configuration on the rb951 that works. Am I missing something?
Yes. First, for the 951, the rule is not necessary because the 951 is behind a NAT from the perspective of the responder (L2TP server), so the ESP is encapsulated into UDP and shares the USP flow with the IKE. Second, the rule is necessary at the responder (server) side, as the first ESP packet to be sent between the peers is the one carrying the L2TP SCCRQ. The set of firewall rules which comes as a part of the factory default configuration for Mikrotik SOHO devices implements a stateful firewall accepting responses to any requests sent by the Mikrotik itself. So as soon as the CRS sends an ESP packet to some IP address, it is ready to accept ESP packets from that IP address without any additional configuration. But at the server side, the first ESP packet from the CRS is an incoming one, so without the added rule, the firewall drops it.
 
ddejager
Member Candidate
Member Candidate
Topic Author
Posts: 134
Joined: Tue Oct 18, 2011 5:13 am

Re: L2TP tunnel usually fails to set up. Suggestions?

Wed Jan 05, 2022 8:26 pm

Adding an accept rule for ipsec-esp to the server input chain seems to have fixed the problem. Thank you! I never would have figured this out myself.

Two questions remain in my mind:

1) Why did my connection occasionally work? Something in the server router must have decided to let me in occasionally.
2) Why isn't this rule in the default config from MikroTIk? It seems like router to router connections over public IP addresses would be common. Given that there are other rules for VPN in the the default config, why is this one missing?

Thanks again!
 
sindy
Forum Guru
Forum Guru
Posts: 10206
Joined: Mon Dec 04, 2017 9:19 pm

Re: L2TP tunnel usually fails to set up. Suggestions?

Wed Jan 05, 2022 8:47 pm

1) Why did my connection occasionally work? Something in the server router must have decided to let me in occasionally.
That remains a mystery for me too. I'd have to see the existing firewall rules at the server side (/ip firewall export, obfuscate any public IP addresses in there that might identify you) to get an idea.

2) Why isn't this rule in the default config from MikroTIk? It seems like router to router connections over public IP addresses would be common. Given that there are other rules for VPN in the the default config, why is this one missing?
Mikrotik only puts firewall rules into the factory default configuration of SOHO devices, not of the large models where it is assumed that administrators of such devices know enough about networking to set them up according to their actual needs (a stateful firewall for forwarded traffic is good for security on an edge router but a waste of CPU throughput for core routers). And on SOHO devices, even the rules for the UDP ports used for L2TP/IPsec are not part of the factory default firewall.

In fact, it may be an intention - I believe no one should start even thinking of a VPN before fully understanding the role and configuration of a firewall, and maybe Mikrotik attempts to ensure the same?
 
ddejager
Member Candidate
Member Candidate
Topic Author
Posts: 134
Joined: Tue Oct 18, 2011 5:13 am

Re: L2TP tunnel usually fails to set up. Suggestions?

Wed Jan 05, 2022 9:36 pm

Sindy,
Here is the firewall from the server router. Note a few rules are disabled. The only change from when it was working only very rarely is the addition of the accept rule for ipsec-esp. Thanks for your thoughts!
# jan/05/2022 14:28:00 by RouterOS 6.48.6
#
# model = 2011UAS-2HnD
/ip firewall filter
add action=jump chain=input comment="jump to chain ICMP" jump-target=ICMP \
    protocol=icmp
add action=accept chain=input comment=\
    "default configuration - accept established" connection-state=established
add action=accept chain=input comment=\
    "default configuration - accept related" connection-state=related
add action=accept chain=input comment="Required for direct inbound l2tp to wor\
    k. Not required if source is behind NAT" protocol=ipsec-esp
add action=accept chain=input comment="allow L2TP" dst-port=500,1701,4500 \
    protocol=udp
add action=accept chain=input comment="allow PPTP tunnels to router" \
    dst-port=1723 protocol=tcp
add action=drop chain=input comment="Drop invalid" connection-state=invalid
add action=accept chain=input comment="Mikrotik btest" disabled=yes dst-port=\
    2000 protocol=tcp
add action=drop chain=input comment="detect and drop port scan connections" \
    protocol=tcp psd=21,3s,3,1
add action=tarpit chain=input comment="suppress DoS attack" connection-limit=\
    3,32 protocol=tcp src-address-list=black_list
add action=add-src-to-address-list address-list=black_list \
    address-list-timeout=1d chain=input comment="detect DoS attack" \
    connection-limit=60,32 log=yes log-prefix="DOS BLACK LIST" protocol=tcp
add action=log chain=input comment="log dropped input packets" disabled=yes \
    log-prefix=Filter:
add action=drop chain=input comment=\
    "default configuration - drop all else from WAN" in-interface=\
    ether1-gateway
add action=log chain=forward comment="log gprs activity" disabled=yes \
    dst-port=22022 log-prefix=GPRS protocol=udp
add action=fasttrack-connection chain=forward connection-state=\
    established,related
add action=accept chain=forward comment="accept established" \
    connection-state=established
add action=accept chain=forward comment="accept related" connection-state=\
    related
add action=jump chain=forward comment="Jump to ICMP chain - Allow all pings" \
    jump-target=ICMP protocol=icmp
add action=drop chain=forward comment="Drop Invalid" connection-state=invalid
add action=accept chain=forward comment="Rule to allow dst-nat traffic to work\
    \_without opening a port for each device. Normal external traffic will be \
    routed to the INPUT chain, but dst-nat traffic will have destination IP of\
    \_LAN. This also makes UPnP work." in-interface=ether1-gateway
add action=drop chain=forward comment=\
    "drop all else from WAN - there should be nothing to drop!" in-interface=\
    ether1-gateway
add action=accept chain=ICMP comment="0:0 and limit for 5pac/s (echo reply)" \
    icmp-options=0:0-255 limit=5,5 protocol=icmp
add action=accept chain=ICMP comment=\
    "3:3 and limit for 5pac/s (port unreachable)" icmp-options=3:3 limit=5,5 \
    protocol=icmp
add action=accept chain=ICMP comment=\
    "3:4 and limit for 5pac/s (fragmentation needed)" icmp-options=3:4 limit=\
    5,5 protocol=icmp
add action=accept chain=ICMP comment=\
    "8:0 and limit for 5pac/s (echo request)" icmp-options=8:0-255 limit=5,5 \
    protocol=icmp
add action=accept chain=ICMP comment=\
    "11:0 and limit for 5pac/s (timeout exceeded)" icmp-options=11:0-255 \
    limit=5,5 protocol=icmp
add action=drop chain=ICMP comment="Drop everything else" protocol=icmp
 
ddejager
Member Candidate
Member Candidate
Topic Author
Posts: 134
Joined: Tue Oct 18, 2011 5:13 am

Re: L2TP tunnel usually fails to set up. Suggestions?

Wed Jan 05, 2022 9:43 pm

Sorry, forgot to post the NAT rules:
/ip firewall nat
add action=masquerade chain=srcnat comment="default configuration" \
    out-interface=ether1-gateway to-addresses=0.0.0.0
add action=dst-nat chain=dstnat comment="GPS data" dst-port=22022 \
    in-interface=ether1-gateway protocol=tcp to-addresses=192.168.5.109 \
    to-ports=22022
add action=dst-nat chain=dstnat dst-port=22022 in-interface=ether1-gateway \
    protocol=udp to-addresses=192.168.5.109 to-ports=22022
add action=dst-nat chain=dstnat dst-port=30175 in-interface=ether1-gateway \
    protocol=tcp to-addresses=192.168.5.109 to-ports=30175
add action=dst-nat chain=dstnat dst-port=30175 in-interface=ether1-gateway \
    protocol=udp src-port="" to-addresses=192.168.5.109 to-ports=30175
add action=dst-nat chain=dstnat comment="Allow access to map server" \
    disabled=yes dst-port=8081 in-interface=ether1-gateway protocol=tcp \
    to-addresses=192.168.5.109
add action=dst-nat chain=dstnat dst-port=7079 \
    in-interface=ether1-gateway protocol=tcp src-port="" to-addresses=\
    192.168.5.194
add action=dst-nat chain=dstnat comment=\
    "Subsonic Media Server" dst-port=4040 in-interface=\
    ether1-gateway protocol=tcp to-addresses=192.168.5.110
add action=dst-nat chain=dstnat dst-port=443 in-interface=ether1-gateway \
    protocol=tcp to-addresses=192.168.5.201
 
sindy
Forum Guru
Forum Guru
Posts: 10206
Joined: Mon Dec 04, 2017 9:19 pm

Re: L2TP tunnel usually fails to set up. Suggestions?

Wed Jan 05, 2022 10:22 pm

I cannot see anything in those firewall rules that would explain why it sometimes worked.

One explanation could be that the CRS has chosen one of its other addresses than the one associated to the WAN interface to initiate the connection from, causing the connection to be be seen as NATed by the peers. The detailed log will show this if you disable the "accept ipsec-esp" rule and keep trying until the connection succeeds. Since an initial packet of a connection is routed as the first step, and only once the gateway interface is determined, the address of that interface is assigned as the source one for that packet, this can happen if there is more than one default route available and the normal one goes temporarily down for some reason.

Another explanation would be that the server sends an ESP packet to the address of the CRS - it seems less likely to me than the above one as it has no reason to do that.

Other than that, there could be a bug in the firewall code, but it's the least likely explanation to me - I'd expect the firewall to be the most tested part of RouterOS.
 
User avatar
own3r1138
Long time Member
Long time Member
Posts: 681
Joined: Sun Feb 14, 2021 12:33 am
Location: Pleiades
Contact:

Re: L2TP tunnel usually fails to set up. Suggestions?

Wed Jan 05, 2022 10:28 pm

The first time I saw this topic I wanted to say that it may require the protocol 50 IPsec ESP but as I saw these lines I second guess myself.

So why does this get to this point of connection?
2022-01-05_23-53-20.png
You do not have the required permissions to view the files attached to this post.

Who is online

Users browsing this forum: almdandi, Bing [Bot], Seko777 and 90 guests