I have 3 locations that are NAT/CGNAT that I need remote access to - between each other and from outside. I setup a Mikrotik CHR on a Cloud Server (running the VMWare VM instance directly from Mikrotik). The CHR is setup as an L2TP server and the 3 sites are L2TP clients - so they establish the connections to the L2TP server. Site A and B both have Mikrotik 4011s that are connecting to the Mikrotik CHR as clients. Site C is a Windows 10 PC using the Windows 10 VPN client to connect to the cloud CHR. The main site I want to get to (Site A) is being connected to via a UISP LTE device. It seems to be working fine - but has a slow upload rate of 1Mbps.

However - considering the hundreds of videos I've watched where people connect to Mikrotik CHRs (running in free mode with a 1Mbps limit on xfer rates) - WinBox and other connections (like HTTP or SSH) work perfectly fine. I have also been using another service which installs and uses an agent on a VM I am running on a server in the same LAN as the router in Site A - which gives me access to all the devices in that LAN via this agent (over an encrypted connection). Using this service (over the LTE internet connection as well), I can open WinBox from remote locations to Site A's 4011 just fine. I can also connect to Proxmox pves and VM's fine thought this service (via SSH, HTTP, and HTTPS) too. WinBox loads quickly and responds quickly - still going over the 1Mbps UISP LTE connection. Not only that - but the encrypted traffic also goes through the remote service's servers in the cloud before coming back to the client using it - so it really has an additional step of going through the local relay agent - and it works fine on this slow connection.

The reason I stated the last part is people keep trying to say it's the slow or inconsistent (which it is not inconsistent) LTE connection for my issue. I disagree for the reasons states above - especially considering the low volume traffic of SSH and simple HTTP. There appears to be something making it slower than the 1Mbps outgoing connection - not simply an issue of L2TP/IPsec overhead.

The issue is - that when I use a PC behind the Mikrotik 4011 at Site B to try to open WinBox to connect to Site A's Mikrotik 4011 (passing through the Mikrotik CHR L2TP server) - it hangs up at "Downloading Descriptors...". So, I open WinBox, enter the LAN IP of the Mikrotik 4011 at Site A (10.90.1.1), the username, and password - click Connect - it says "Logging in..." for a fraction of a second, then that changes to "Downloading Descriptors..." and then it just sits there making no progress and never connects. If I connect WinBox to the 4011 at Site B (10.0.0.1) - it works flawlessly. If I connect WinBox to the cloud vpn internal IP of the Mikrotik CHR (10.80.1.1) - it connects fine. If I use my PC at Site C (10.14.0.x/24) to connect WinBox to either the cloud CHR or Site B 'tiks - it connects fine - BUT it also has the same issue connecting to Site A's 4011 'tik with WinBox. I had a tech consultant also try from their location using an SSTP connection to the cloud CHR 'tik (acting as client) - and they had the same issue. The L2TP routes and access seem to work everywhere fine - except using any data of consequence from devices at Site A. Initially, I noticed that if I did much of any data use on Site A's 'tik - it hung up. Like, I could SSH into the Site A 'tik over the VPN - but if I did anything that caused even close to a screen worth of info to be show - the connection would freeze and no longer respond. I tried this with multiple SSH clients and the same thing happened. This seemed to improve after adjusting the mtu of the L2TP server to 1400 as the LTE connection seemed to be causing a lot of connection drops when at anything 1410 and higher.

I also tested, from Site B to Site A - removing all Firewall rules on both 'tiks and it made no difference. Opening web pages seems to just not happen. Like if I open dev tools (F12) and look at Network - the pages just simply aren't loading.

So WinBox, SSH, and sites can clearly be opened over the VPN from Site C to Site B. Sites B and C to cloud CHR also work. But anything to Site A seems to hang with any small amount of traffic being used.

I can ping 10.90.1.1 and other devices in that LAN. Generally, 150-300ms responses - but they are there. I have a RasberryPi on Site A that is running iperf3 in server mode. I can connect to it - but it shows 0.00 bits/sec bandwidth on each test line and at the end says 62.1 kbit/sec (receiver) with my PC on Site B being the iperf client and running as receiver. 62.1Mbps is slow since I know the connection is typically around 1Mbps. So maybe this is related to the net speed - BUT it seems to have something to do with Site A and the VPN as the same thing happens with other devices in the LAN - yet when using the 3rd party remote access software - I can connect and speeds are ok to descent.

I have found similar questions posted in here, though not exact, but none of them were answered - so here I am.

Sounds like a classic MTU problem to me, but I don't know enough about these types of fringe cases (or whether it's the problem for certain) to advise a solution, other than to say I've fixed similar problems with wireguard tunnels in the past by running an eoip tunnel over the (already optimised) MTU-constrained tunnel. Obviously that is going to introduce more overhead thereby impacting throughout potential even further, but at least pages will load and you won't have those strange timeouts and traffic stalling as you described due to the packet fragmentation

Or if you only need to fix TCP, you could do MSS clamping instead of the EOIP layer

If there's a better workaround, I'd love to learn about it too

The symptoms indeed suggest an MTU issue - multiple lines of text sent in a single go cause the SSH TCP session to exceed the actual MTU of the path, and if path MTU discovery is broken somewhere, the TCP session never recovers from that.

There is a way to circumvent this by splitting the large packets at PPP encapsulation level rather than at IP level. If you set mrru to 1504, and max-mtu and max-mru to 1300 at both the L2TP client and the L2TP server, the client and server will negotiate use of MLPPP and use two unfragmented transport packets, rather than a single fragmented one, to transport any payload packet that does not fit into a single L2TP transport packet. So from the point of the view of both routers, the MTU of the tunnel is 1500, i.e. even non-TCP traffic doesn't need to bother about MTU. The price to pay is that the packet rate of TCP connections doubles, as TCP slices the byte stream into packets corresponding to the 1500-byte MTU rather than reducing the packet size to accommodate to the actual path MTU.

...
If you set mrru to 1504, and max-mtu and max-mru to 1300 at both the L2TP client and the L2TP server, the client and server will negotiate use of MLPPP and use two unfragmented transport packets, rather than a single fragmented one, to transport any payload packet that does not fit into a single L2TP transport packet.
...

So, are you saying on the PPP...L2TP Server tab - to set the mrru to 1504, max-mtu and max-mru to 1300 and then to do the same on the PPP...Profile tab on the client? Just making sure I understand your suggestion.

If so, just to make sure, this would impact all the client tunnels - not just the slow one. Will the settings as they are on the other clients be ok to leave as they are - or do I need to set all the clients to match the server for these settings as well? Sorry overriding mtu settings is a new thing for me. If you have any good sources on how to understand this process better that would be awesome - I want to learn - not just bum for help. Thanks!

So, are you saying on the PPP...L2TP Server tab - to set the mrru to 1504, max-mtu and max-mru to 1300 and then to do the same on the PPP...Profile tab on the client? Just making sure I understand your suggestion.

On the client side, these items are also not properties of /ppp profile row but directly of /interface l2tp-client (unless something has changed recently).

If so, just to make sure, this would impact all the client tunnels - not just the slow one. Will the settings as they are on the other clients be ok to leave as they are - or do I need to set all the clients to match the server for these settings as well?

It will impact all client tunnels in terms that if MLPPP is not permitted on the client, it will reduce the MTU of the tunnel to that client to 1300. There's unfortunately no way to circumvent this as you cannot configure these values in profile nor can you set up multiple L2TP servers with different settings on the same router. If the other clients are Mikrotik ones too, it's not a big deal; if they are other ones not supporting MLPPP, it is.

Thanks Sindy - I will try this. The client side confusion is surely my mistake - I'm not able to get the these systems at the moment and was working from memory.

Two of the clients are Windows 10 PCs - 2 other clients are Mikrotik routers.

Anyway, I will see about testing this this evening and I'll update. Thanks!

Sorry... I got diverted yesterday evening - so I'll work on trying this out this evening...

Hey Sindy! That has definitely made everything MUCH better. In fact, it may be working totally fine. But, I having one lingering issue that I though was related - but maybe it's not.

I have a backup to email script running on the 'Tik where this LTE connection is at. The other 2 'Tiks (the CHR and the other 4011) are both running the exact same script and all 3 have the Email configured the same way. The problem is that the 2 not using this LTE connection are working fine, but the 1 with the LTE connection is getting an error in the Logs saying that the email is timing out. If I test in terminal with that unit and send a message with no attachment - the email sends. But if I attach anything (everything I've tried anyway) - including a small 2K file - it also times out. At this point I've been uploading files to a server behind the LTE connection from my PC which is using the L2TP tunnel - and the files are much larger and a lot of them. I uploaded 40 or so files that were all 9MB+, a couple in the 60MB range - and they go fine - but this email send ALWAYS fails. Is there any chance this is still related to this MTU/MRU/MRRU issue - or is this another issue I should start another thread about?

In any case - THANKS! I have finally been able to make some progress on work I was behind on due to this connection issue!

There is still a chance that it is an MTU issue, except that this time the issue is in the other direction (packets from the 'Tik rather than ones towards the 'Tik). Is the traffic towards the e-mail server routed through the L2TP tunnel? Some LTE interfaces have a smaller MTU than 1500.

Sorry Sindy - somehow, I missed your last response here. I think it must have been related to the MTU of the LTE connection. My fiber line has since been installed and now the tunnel is moving across that instead of the LTE. Now the emails are working. The emails should have been going over the LTE, but not via the VPN - routes setup should have avoided that. But it is all changed now with the fiber. I did however also have to tweak the MTU on the L2TP client from this location to the Mikrotik CHR in the cloud. I also had to change the settings on the cloud CHR to get it to work over the fiber. I removed the MRRU and set the MTU and MRU back to 1500 there. I think I have everything working from my tower location that now has fiber by setting the MTU to 1400. 1480 and 1450 were still causing some oddities - like Proxmox not being able to open console windows, and WinBox opening - but with no data in any of the screens. The 'Tik at the tower has routes for specific LAN IPs to go through the tunnel, but all other traffic should and seems to be going through the fiber directly - I see my fiber provider listed on speed test sites and such.

Is there some way to really pin this MTU stuff down? The side effects it is having on software is hard to track down - and to figure out that the MTU has any bearing on it. It's quite frustrating. Is the L2TP tunnel adding to this? I mean, are there other tunneling options that are more forgiving or auto-tuning in this respect? I'm having issues with other software too - and not sure if MTU is related. Maybe - but maybe not.

Thanks!