Hello, I have a problem for which I have not been able to find a solution anywhere on the Internet. The problem is that when I upload some data to my main server via FTP, the upload speed is very bad, even though the internet speed on my computer and on the server is over 500Mbps. I rented a dedicated server from a hosting provider and a block of IP addresses that I tunneled to my mikrotik router via a GRE tunnel.
I tried to change MTU, MMS but without any success. I also tried using IPIP tunnel with the same configuration but the same problem happens. The speed of the leased dedicated server is 300+ Mbps.
When I run the speedtest myself on my main server that uses the internet from the dedicated ookla speedtest, the speedtest reports a normal speed for both download and upload (300+down, 250+up). On the dedicated server, I use the ubuntu 20.04 system with iptables rules so that everything is redirected properly (if you also need those commands, I can publish them).
When I try to upload a file through our public IP address that the provider gave us, the speed goes perfectly, 500-600mbps, as it should be, but when it goes through the tunnel, it shows 30-40mbps for a moment and immediately drops to 4-5mbps, which is not okay at all.
For mikrotik router I use hEX S with routerOS version v6.49.7
Could be wrong, but I don’t think you can use fast-track with IPSec-enabled GRE tunnel.
Also, are you setting MTU lower on the GRE interface, or is MTU 1500? If it’s 1500, you can use ping and don’t fragment to determine the MTU (search here/google for using ping to calculate mtu size).
Relatedly, make sure icmp/ping is not blocked over the GRE link (or anywhere on the path).
gre by default has 1476 MTU. you may need to scale if you have vlans in the transports connecting you to the internet and also if you have pppoe encapsulation. a VLAN involves decreasing MTU by 4bytes and pppoe by 8bytes. So if they deliver you internet with pppoe on VLAN you should set 1488 on the pppoe interface and 1464 on the gre. otherwise you have to test with ping using “do not fragment” . For convenience you can also set a change mss on the forward chain in the mangle.
in any case, even if you have improvements you will never have the result you get by pointing to the public IP, because with gre the data is encrypted
I am aware that I will not be able to have the same results on a gre tunnel as I would have directly on a public ip address. I started a company that deals with hosting and we use GRE tunnel for IP addresses and DDoS protection and our clients have problems with uploading their files to the server, 5-6 Mbps is very slow, especially if it is a website with over 10gb of data…
Probably a PMTUD problem. You should not change MTU on the GRE (except to set it lower when the internet MTU is not 1500, e.g. when you use PPPoE).
Instead, make this mange rule to decrease MSS on the TCP connections:
I set the MTU for the gre tunnel to 1500 but without any improvement, the default MTU that mikrotik set for me is 1476. I also tried to reduce the mtu to something smaller but without any improvement either. I also tried to use ping to calculate the mtu size, but I didn’t quite understand how because whatever MTU I set it says Packet needs to be fragmented but DF set
C:\Users\Bendza>ping -f -l 1500 51.x.x.232
Pinging 51.x.x.232 with 1500 bytes of data:
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
The only thing that interests me is that through the speedtest, the speed is as normal as it should be, but when a file is uploaded via ftp, the speed is significantly lower.
[root@cp1 ~]# speedtest-cli
Retrieving speedtest.net configuration...
Testing from OVH SAS (51.77.82.232)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Deutsche Telekom (Berlin) [300.79 km]: 42.132 ms
Testing download speed................................................................................
Download: 189.25 Mbit/s
Testing upload speed......................................................................................................
Upload: 129.57 Mbit/s
It looks like you are not using ipsec, is this correct?
From windows, you need to subtract 28 from your length of ping,
(Windows makes the data in the packet that big, then has 28 bytes of IP header on top).
So for a 1500 byte mtu,
ping -f -l 1472 51.x.x.232
If the MTU the router thinks it has to the other end is incorrect, it can/will cause issues.
I would initially find what the raw outside MTU to the other end of the link is.
Use a PC that doesn’t go via the Gre tunnel, or use the Mikrotik (don’t subtract 28)
(for 1500 from windows)
ping -f -l 1472 217.x.x.x
(for 1492 pppoe from windows)
ping -f -l 1464 217.x.x.x)
If the maximum length of the ping is less, you might need to hunt for it.
eg. Windows: try -f -l 1272 if ok then try -f -l 1372 then…
Often for best efficiency it would be good to set the GRE tunnel MTU to this found
MTU - 24 (the GRE overhead)
eg. If outside MTU=1492, want GRE MTU = 1468
if outside MTU=1500, GRE MTU=1476 (The default)
**Edit: Not this, GRE is layer 3 ** If you have vlan’s running inside your GRE perhaps you want to shrink this MTU by another 4 bytes or more, so the VLan tag information also fits. **
Good explanation @rplant. Does seem like MTU issue… i.e. “slow” often a sign of wrong MTU & “slow” also means it’s getting through the firewall so ain’t fw filters… Even if not, it still best to know/confirm your MTU with the ping test.
In similar MTU vain, make sure icmp is allowed by firewall on input to give PMTUD a shot of working. That’s in the default firewall, but if building your own, perhaps ICMP is getting blocked.
Only other thing I can think of is if FTP is not using standard port 21, you’d want to added it to the firewall helper for FTP in /ip/firewall/service-port