Community discussions

MikroTik App
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Solve ISP VPN L2TP throttle (CCR to CCR)

Fri Aug 20, 2021 3:29 pm

Hardware - CCR1016 at each site
Connection - L2TP
ISP 1 - OTE (Greece)
ISP 2 - Virgin (UK)

OTE in Greece has started limiting single connection bandwidth to 2-3Mbps.

Bandwidth test using RouterOS between two CCRs shows that max Bandwidth usage (30Mbps) is obtained with 15 Connections (Connection Count)

Is there any way to solve this and force the CCRs to establish multiple connections and load balance in some way?

Much obliged.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Fri Aug 20, 2021 5:10 pm

If the payload connection should be a single one, I'm afraid there is no way to beat that - Mikrotik only supports MLPPP as PPPoE client, and no other load distribution method can spread a single connection across multiple links. If you have in mind a total bandwidth shared by multiple connections, it should be doable. But it's probably just a matter of time until OTE starts throttling connections by IP addresses alone, so you might end up with 15 IP addresses at the UK end to get each of your 15 tunnels throttled separately.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Fri Aug 20, 2021 5:25 pm

If the payload connection should be a single one, I'm afraid there is no way to beat that - Mikrotik only supports MLPPP as PPPoE client, and no other load distribution method can spread a single connection across multiple links. If you have in mind a total bandwidth shared by multiple connections, it should be doable. But it's probably just a matter of time until OTE starts throttling connections by IP addresses alone, so you might end up with 15 IP addresses at the UK end to get each of your 15 tunnels throttled separately.
Thank you, this is so infuriating. Netflix and Social Media get no throttling but my VPN does. Super stupid question but there isn't a way to get the two CCRs to connect without it looking like a VPN to the ISP?

EU law used to forbid throttling but now due to the pandemic, important services like Facebook are not throttled but business VPNs are.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Fri Aug 20, 2021 5:33 pm

As you mention EU law, could it be that OTE only throttles links to UK addresses now that Brexit took place?

If the throttling only affects certain types of traffic, you can try an SSTP VPN (as it uses TCP port 443, same as https, at server side) where the client end will be in Greece. But VPNs using TCP as transport have issues of their own, and SSTP doesn't seem to be a speed champion. Try and see.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Fri Aug 20, 2021 5:48 pm

As you mention EU law, could it be that OTE only throttles links to UK addresses now that Brexit took place?
I doubt it, they would ask for money if they could.
If the throttling only affects certain types of traffic, you can try an SSTP VPN (as it uses TCP port 443, same as https, at server side) where the client end will be in Greece. But VPNs using TCP as transport have issues of their own, and SSTP doesn't seem to be a speed champion. Try and see.
I will try and see if it makes a difference. Thanks.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Fri Aug 20, 2021 5:48 pm

... and one more idea, as Google uses QUIC on UDP port 443 rather than TCP, maybe OTE is not brave enough to throttle Google services? I.e. an IKEv2 tunnel with UDP port 443 at the UK side might not be throttled. If the Greek end has a public IP on the CCR itself, you have to force use of NAT for the IPsec, as otherwise it would use bare ESP as transport packets, which would spoil the trick. It must be IKEv2 because IKE(v1) uses two UDP ports at responder side.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Fri Aug 20, 2021 5:51 pm

... actually, even simpler, you can do the same directly with the L2TP - if I remember well, you're using bare L2TP, without IPsec, correct?
 
kevinds
Long time Member
Long time Member
Posts: 575
Joined: Wed Jan 14, 2015 8:41 am

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Sat Aug 21, 2021 7:55 am

Is there any way to solve this and force the CCRs to establish multiple connections and load balance in some way?
An ISP here I worked with had a kind of similar thing, but they slowed the connection down depending on how much data the single connection had transferred.. The first 10 MB was at full speed, 5 mbps, after 10MB the connection slowed to 3 mbps. After 15MB it slowed to 1.5 mbps. Eventually it slowed to 128 kbps for the remainder of time. Worked well for speedtest results, did not work well for VPN.

The solution was to switch to switch to a connectionless tunnel.. GRE, IPoE, something similar. Each packet was a new connection.
Last edited by kevinds on Sat Aug 21, 2021 7:58 am, edited 1 time in total.
 
kevinds
Long time Member
Long time Member
Posts: 575
Joined: Wed Jan 14, 2015 8:41 am

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Sat Aug 21, 2021 7:58 am

maybe OTE is not brave enough to throttle Google services?
If that is the case, perhaps a VM in the Google Compute Engine to put between the two CCRs?
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Sat Aug 21, 2021 7:10 pm

When talking about "not throttling google" I had rather in mind matching on UDP port 443 than matching on a list of Google IP addresses - matching on a single port value is much less resource-consuming than matching on an address list. But if the ISP uses per-connection bandwidth management, maybe it doesn't make a big difference.

Regarding GRE, IPIP etc., it is a lottery - OTE may drop these protocols completely, or it may connection-track them just by IP addresses.

For your (@kevinds) case, where the bandwidth is gradually throttled with the amount of data transported, I could imagine two or three L2TP tunnels with a routing failover and a script that would disable and re-enable them in turns and assign a different source port to each connection, but where the connections are throttled all the time, such an approach wouldn't help.

@AJSG, to my surprise, unlike with SSTP, it is not possible to configure a remote port other than 1701 for /interface l2tp-client in RouterOS, so my idea of using server-side port 443 for L2TP would be hard to implement - a workaround exists but it can easily cause a brain overheat. Hence the test of treatment of connections to UDP port 443 is much easier with IKEv2.

But coming back to the initial idea of multiple tunnels in parallel - it should be actually possible to use mangle rules to distribute the traffic among all of those tunnels "manually", but since this will cause packets to be missequenced at the receiving side (smaller packets sent just after longer ones will often overtake them), the result may be worse than expected. But still worth trying.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Sun Aug 22, 2021 11:27 am

So I've tried the last idea, to "manually" distribute the traffic among three separate L2TP tunnels, each limited to 200 kbps at transport side, imitating the OTE behaviour. It works - for the payload, I've got 580 kbps throughput for a single test UDP connection and about 560 kbps for a single test TCP connection.

What I didn't expect was that if you set up multiple /interface l2tp-client to the same server, they use the same transport connection, so you have to take some extra measures to prevent this.

If you want all the minute details of the setup, let me know.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Mon Aug 23, 2021 1:26 pm

... actually, even simpler, you can do the same directly with the L2TP - if I remember well, you're using bare L2TP, without IPsec, correct?
Just catching up on posts, running IPsec at the moment.

This is the current config out here:

# model = CCR1016-12G
# serial number = 42D402FA6367
/interface bridge
add fast-forward=no mtu=1500 name=bridge-lan protocol-mode=none
/interface ethernet
set [ find default-name=ether1 ] comment="OTE Router - NOT Modem Mode" l2mtu=\
    1590 name=ether1-isp1 speed=100Mbps
set [ find default-name=ether2 ] speed=100Mbps
set [ find default-name=ether3 ] speed=100Mbps
set [ find default-name=ether4 ] speed=100Mbps
set [ find default-name=ether5 ] speed=100Mbps
set [ find default-name=ether6 ] speed=100Mbps
set [ find default-name=ether7 ] speed=100Mbps
set [ find default-name=ether8 ] speed=100Mbps
set [ find default-name=ether9 ] speed=100Mbps
set [ find default-name=ether10 ] speed=100Mbps
set [ find default-name=ether11 ] speed=100Mbps
set [ find default-name=ether12 ] speed=100Mbps
/interface l2tp-client
add add-default-route=yes comment=\
    "Virgin VPN Connection - change only Virgin external IP" connect-to=\
    31.49.235.159 name=l2tp-out1 user=tinos-l2tp
add add-default-route=yes comment=\
    "BT VPN Connection - change only BT external IP" connect-to=31.49.235.159 \
    default-route-distance=2 name=l2tp-out2 user=tinos-l2tp
/interface list
add name=WAN
add name=LAN
add name=VPN
/ip ipsec profile
set [ find default=yes ] dh-group=modp1024
/ip pool
add name=pool-tinos ranges=172.16.110.50-172.16.110.100
/ip dhcp-server
add address-pool=pool-tinos authoritative=after-2sec-delay disabled=no \
    interface=bridge-lan lease-time=1d name=dhcp-tinos
/interface l2tp-client
add comment="VPN Connection to London" connect-to=82.13.218.71 disabled=no \
    max-mru=1380 max-mtu=1380 name=l2tp-out3 profile=default use-ipsec=yes \
    user=tinos-l2tp
/interface bridge port
add bridge=bridge-lan hw=no interface=ether2
add bridge=bridge-lan hw=no interface=ether3
add bridge=bridge-lan hw=no interface=ether4
add bridge=bridge-lan hw=no interface=ether5
add bridge=bridge-lan hw=no interface=ether6
add bridge=bridge-lan hw=no interface=ether7
add bridge=bridge-lan hw=no interface=ether8
add bridge=bridge-lan hw=no interface=ether9
add bridge=bridge-lan hw=no interface=ether10
/ip firewall connection tracking
set udp-stream-timeout=30s
/ip neighbor discovery-settings
set discover-interface-list=!dynamic
/ipv6 settings
set max-neighbor-entries=1024
/interface list member
add interface=ether1-isp1 list=WAN
add interface=bridge-lan list=LAN
add interface=l2tp-out1 list=VPN
add interface=l2tp-out2 list=VPN
/ip address
add address=172.16.110.1/24 interface=bridge-lan network=172.16.110.0
add address=192.168.1.2/24 interface=ether1-isp1 network=192.168.1.0
/ip dhcp-server network
add address=172.16.110.0/24 dns-server=172.16.110.2 gateway=172.16.110.1
/ip dns
set servers=1.1.1.1,1.0.0.1
/ip firewall address-list
add address=172.16.105.0/24 disabled=yes list=primary-subnets
add address=172.16.110.0/24 disabled=yes list=primary-subnets
/ip firewall filter
add action=accept chain=input comment=\
    "defconf: accept established,related,untracked" connection-state=\
    established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=\
    invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="defconf: accept to local loopback" \
    dst-address=127.0.0.1
add action=accept chain=input comment=\
    "allow access to router itself via VPN tunnels" in-interface-list=VPN
add action=accept chain=input src-address=172.16.105.0/24
add action=drop chain=input comment="defconf: drop all not coming from LAN" \
    in-interface-list=!LAN
add action=accept chain=forward comment=\
    "defconf: accept established,related, untracked" connection-state=\
    established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=\
    invalid
add action=drop chain=forward comment="defconf: drop all from WAN not DSTNATed" \
    connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
/ip firewall mangle
add action=mark-routing chain=prerouting disabled=yes new-routing-mark=VOIP \
    passthrough=yes src-address=172.16.110.125
add action=mark-packet chain=forward disabled=yes dst-address=172.16.105.3 \
    new-packet-mark=malford_packets out-interface=l2tp-out3 passthrough=yes
/ip firewall nat
add action=dst-nat chain=dstnat dst-port=8443 protocol=tcp src-address=\
    82.17.140.24 to-addresses=172.16.110.125 to-ports=443
add action=masquerade chain=srcnat out-interface=l2tp-out3
add action=masquerade chain=srcnat out-interface=ether1-isp1
/ip firewall service-port
set ftp disabled=yes
set tftp disabled=yes
set irc disabled=yes
set h323 disabled=yes
set sip disabled=yes
set udplite disabled=yes
set dccp disabled=yes
set sctp disabled=yes
/ip route
add disabled=yes distance=1 gateway=192.168.1.1 routing-mark=VOIP
add comment="Connection from L2TP BT-Malford" distance=1 gateway=10.10.20.1
add comment="Main connection in case of L2TP IP Address Change" distance=3 \
    gateway=192.168.1.1
add comment="Route to London CCR" distance=1 dst-address=82.13.218.71/32 \
    gateway=192.168.1.1
add distance=1 dst-address=82.17.140.24/32 gateway=192.168.1.1
add distance=1 dst-address=109.145.13.198/32 gateway=192.168.1.1
add distance=1 dst-address=172.16.105.0/24 gateway=10.10.20.1
/ip service
set telnet address=172.16.0.0/16
set ftp disabled=yes
set www disabled=yes
set ssh disabled=yes
set api disabled=yes
set winbox address=172.16.0.0/16
set api-ssl disabled=yes
/ip ssh
set allow-none-crypto=yes forwarding-enabled=remote
/ipv6 nd
set [ find default=yes ] advertise-dns=no
/lcd
set default-screen=stats time-interval=hour
/lcd interface
set ether2 disabled=yes
set ether3 disabled=yes
set ether4 disabled=yes
set ether5 disabled=yes
set ether6 disabled=yes
set ether7 disabled=yes
set ether8 disabled=yes
set ether9 disabled=yes
set ether10 disabled=yes
set ether11 disabled=yes
set ether12 disabled=yes
/lcd screen
set 1 disabled=yes
set 2 disabled=yes
set 3 disabled=yes
set 4 disabled=yes
set 5 disabled=yes
/system clock
set time-zone-name=Europe/London
/system health
set fan-mode=manual
/system identity
set name=Tinos1016
/system leds
add disabled=yes leds=user-led type=on
/system logging
add disabled=yes topics=ipsec
add topics=l2tp
/tool bandwidth-server
set authenticate=no
/tool sniffer
set file-name=tinos
[admin@Tinos1016] > 

 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Mon Aug 23, 2021 1:34 pm

So I've tried the last idea, to "manually" distribute the traffic among three separate L2TP tunnels, each limited to 200 kbps at transport side, imitating the OTE behaviour. It works - for the payload, I've got 580 kbps throughput for a single test UDP connection and about 560 kbps for a single test TCP connection.
UDP is not affected at all by the ISP, get full available bandwidth (34.7Mbps) with a Connection Count 1, TCP gives me 1.01Mbps for single. Even with a Connection Count of "50" today I cannot even get to 33Mbps. I suspect that the idea of multiple tunnels might be too complex to implement.
What I didn't expect was that if you set up multiple /interface l2tp-client to the same server, they use the same transport connection, so you have to take some extra measures to prevent this.
If you want all the minute details of the setup, let me know.
I am on the line with OTE at the moment to see if I can at least get a fixed IP out of them. Our current external IP addresses are virtual.

Regards,

Andrew.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Mon Aug 23, 2021 2:14 pm

UDP is not affected at all by the ISP, get full available bandwidth (34.7Mbps) with a Connection Count 1, TCP gives me 1.01Mbps for single.
That changes the whole point of view. Both bare L2TP and IPsec in NAT-T mode use UDP, so if UDP sessions are not throttled, the issue is somewhere else, because the IPsec transport packets look the same no matter what payload packets (UDP, TCP, other ones) they carry, so the only difference the ISP can see between them is size.

What could make a difference would be if OTE was restricting VPNs selectively - UDP to ports 1701 and 4500 to cover bare L2TP and NAT-T IPsec.

So when you say that "UDP is not affected at all whilst TCP is throttled to 1 Mbps", do you run the bandwidth test towards the public IP of the UK machine or inside the L2TP/IPsec tunnel?

TCP throughput is heavily affected by round-trip time, so if the network path between OTE and UK has changed and the delay has grown, this may be the explanation. What's the RTT indicated by ping from Greece to the UK inside and outside the VPN tunnel?

Since the Greek end is on an island, OTE may have had to use some microwave backup due to an outage of the regular wired/optical line.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Mon Aug 23, 2021 4:02 pm

What could make a difference would be if OTE was restricting VPNs selectively - UDP to ports 1701 and 4500 to cover bare L2TP and NAT-T IPsec.

So when you say that "UDP is not affected at all whilst TCP is throttled to 1 Mbps", do you run the bandwidth test towards the public IP of the UK machine or inside the L2TP/IPsec tunnel?
I'm testing within the VPN tunnel using local gateway to remote gateway, ie 172.16.110.1 to 172.16.105.1 (both sides are CCR 1016s). Using the Tool built into RouterOS. However this is backed up by external testing. Both inside an outside the VPN I can download from Google or watch Netflix no problem and I can see Rx flying. When I try and move files or similar within the VPN Rx collapses.

I tried using Teamviewer File transfer within the VPN, and I get 6-7 times the speed of when I transfer the same file via normal VPN.
TCP throughput is heavily affected by round-trip time, so if the network path between OTE and UK has changed and the delay has grown, this may be the explanation. What's the RTT indicated by ping from Greece to the UK inside and outside the VPN tunnel?
Tends to be pretty steady around the 70ms (TTL 64) within the VPN to the internal IPs, 90ms (TTL 64ms) to 8.8.8.8 (all zero packet loss). Outside the VPN ping to 8.8.8.8 is 56ms and TTL is 116.

Cheers.

Andrew.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Mon Aug 23, 2021 5:51 pm

Both inside an outside the VPN I can download from Google or watch Netflix no problem and I can see Rx flying. When I try and move files or similar within the VPN Rx collapses.

I tried using Teamviewer File transfer within the VPN, and I get 6-7 times the speed of when I transfer the same file via normal VPN.
So that I understood it properly - you've got one PC with Teamviewer connected to each of the two CCRs, and any traffic from the LAN in Greece reaches internet via the VPN and thus via the CCR in UK, is that correct?

Because the only way I can imagine how the additional encapsulation could improve the VPN performance would be that the Teamviewer file transfer would use multiple TCP sessions in order to address the RTT issue. It never came to my mind to check that.

70 ms may be quite a lot for the TCP.

Another thing is what is the base for comparison? I.e. was it significantly better in the past, or you just noticed the slowness now but you have no hard data from the past?

What's the ping RTT if you ping the UK public address from Greece outside the tunnel?
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Mon Aug 23, 2021 8:17 pm

So that I understood it properly - you've got one PC with Teamviewer connected to each of the two CCRs, and any traffic from the LAN in Greece reaches internet via the VPN and thus via the CCR in UK, is that correct?


Teamviewer file transfer was just a test so see if (like you write below) it uses multiple connections, which it would assume it does. With the VPN connected I can log in to both CCR via Winbox as both networks x.x.110.x and x.x.105.x are visible.


Another thing is what is the base for comparison? I.e. was it significantly better in the past, or you just noticed the slowness now but you have no hard data from the past?

Yes till about the 10-12th August I would not have any of these issues. Throttling used to be illegal in Europe, but they changed the laws due to lockdown. Unfortunately they also allowed Netflix et at to evade these rules as clearly they are so important to home working (or lack of working!).

What's the ping RTT if you ping the UK public address from Greece outside the tunnel?
My external IP (Virgin in London x.x.218.71) 75ms, pretty stable, 0 loss, TTL 49

Cheers.

Andrew.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Mon Aug 23, 2021 8:27 pm

70 ms may be quite a lot for the TCP.
It was already this high back in July when the connection was allowing full bandwidth usage, obviously certain days were better than others, but I am always talking about "max available" not theoretical. So if London Virgin line upload is 30, then Tinos download would max out at about 28. On a day where Virgin upload is 10 I would be happy wth 8'ish download.

Cheers.

Andrew.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Tue Aug 24, 2021 12:39 pm

I don't think the change of behaviour is caused by some action of the ISP intended to slow down VPNs.

The thing is that no matter what you do inside the L2TP/IPsec tunnel, the ISP can see the whole tunnel traffic as a single UDP connection from port 4500 of the Tinos CCR to port 4500 of the London CCR (as your IP at Tinos is not a public one, London probably sees the connection to come from another port than 4500, but that changes nothing). So if they were using per-connection throttling, the bandwidth inside the VPN would be the same no matter whether you were using UDP, single-session TCP or multi-session TCP.

Many people are actually complaing about throughput of L2TP/IPsec here on the forum, but till now I haven't seen a clear conclusion what are the external conditions to cause that. My assumption is that it is somehow related to packet size and/or order at arrival, where the latter may depend on the former. Both IPsec and TCP are sensitive about packet arrival order to some extent.

If you feel like helping me analyse that, could you please sniff into a file at both devices while downloading a file via the tunnel? If yes, I'll give you a more detailed instruction on that.

As a quick shot, try to set max-mtu and max-mru at both the L2TP server (first) and at the L2TP client to 1300. Each change will probably make the tunnel drop and re-establish. If this change makes the TCP speed good again on a single connection, the issue is definitely related to handling of small packets.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Tue Aug 24, 2021 4:21 pm

Many people are actually complaing about throughput of L2TP/IPsec here on the forum, but till now I haven't seen a clear conclusion what are the external conditions to cause that. My assumption is that it is somehow related to packet size and/or order at arrival, where the latter may depend on the former. Both IPsec and TCP are sensitive about packet arrival order to some extent.
I will do as you suggested over the next few days. I must say that the issue is that I did not have these issues till the beginning of August, was working fine in July. It makes sense that it can only be down to three things, Virgin ISP, OTE ISP or the new RouterOS package.

Considering that many commercial services are unaffected, such as Netflix, Google Docs etc etc, but direct connections are (such as internal file transfer) my assumption is that the ISPs are targeting private VPN traffic.

Regards,

Andrew.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Tue Aug 24, 2021 4:56 pm

I don't think the change of behaviour is caused by some action of the ISP intended to slow down VPNs.
Too add insult to injury;

Speedtest.net reports 3.69 by 2.30 (inside VPN) and Google Speed Test reports 25.6 by 2.42 (inside the VPN)

These two test run one after the other.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Tue Aug 24, 2021 5:16 pm

Large services often serve customers from regional servers (Google does that for sure), whereas the direct connection spans half of Europe. I doubt OTE has a direct peering with Virgin UK, so there are most likely multiple network paths between the two, making packet overtaking, resulting in shuffled packet order at arrival, possible.

I suffer no romantic illusions about ISPs, but I follow my mum's advice - only waste your time looking for evil intention where mere stupidity is not sufficient as an explanation. Nevertheless, it is possible that the ISP applies some specific treatment to the VPN traffic. But if that treatment targets something, I'd rather expect it to be P2P file sharing networks than VPNs. P2P file exhausts network bandwidth whilst itself it doesn't need fast responses, so slowing it down in order to allow interactive services to perform better is something I can understand. And slowing down of TCP traffic inside VPNs may be a neglected side effect if the slowdown is achieved by tampering with small UDP packets.

The sniffing would consist in the following steps:
  • at both routers, run /tool sniffer set file-limit=50000 file-name=l2tp-ipsec-xxx.pcap (replace xxx by either client or server as per the role of the CCR in the L2TP setup)
  • start the bandwidth measurement using a single connection and TCP and let the throughput stabilize
  • use /ip ipsec active-peers print in London to find out from what public IP the connection arrives from Tinos, the contents of the REMOTE-ADDRESS column is important, not of the ID one)
  • at both routers, run /tool sniffer quick ip-address=the.public.ip.of.the.other.ccr
  • after some 10 seconds from starting it at the second one, stop it at both (so we should have 10 seconds of traffic captured at both)
Then download the files and think how to hand them over. Since private messaging is off again, I suggest you send me your e-mail address according to lines 14-16 of this howto (you use your e-mail address instead of the --pass--). Or you may post the files on some file sharing service with a password, put a link to it here and send me just the passwords to the files encrypted as above.

My public key is

public key code

-----BEGIN PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEArauOfFEn3Grb2WhnlKe1
hEG9bwq+7fJiDhFyCK+CcmXwQyGWl4LIop5VCTNUYq/++PSFOAmkoLA1+TMTAN6s
f7ukyVErCxpfUMy+xhutVzsVDHIaAScJFr3QUPmI+544EMplZn70DOWIWgv4Xtac
jeiuqJZ2mExt/8amZTQ2dJ3MHmRaD20gUI+ByJwkQopxUZTOfKAXkhkL/HTcksyO
ORSh+EzFN+ll674084RpbXGKnaDYysAObLSib9UgnHnQ98WLbh+4XZqDc6T82mFW
0BcxLI8pUeITwBaexX8YbDYOFnxskLIlGcRb0F9sRGV8as4H5+zuwlO14K1LDb4Y
suNttQSKYFOZ91G2SOBlGiejO2xf7t3spdv0LurOkvvKJo+wOwAhfz/8zBubUBqd
0jXQyzkQ+WkBMzKBYwOl39YQXUHYHxxGUXMupslSyw1qcyW0QeC91hfM+/GNk+j/
+w+lqLV0So2gc8vD/9mVAcMv9g8eSffvwEKygnXsqlg4QLvjCVNu1TMtYH9NJAgs
2VoBgDaAZrc5id9dk41Zh19L4mYJ+uTiafZ/g/5jx5AEjPdVcsKBsatQVld+RuS5
B+kLTXbPuk8kXRLhKuSOEEBGOvXaCvwqNPdvghNOQSvydVOkVyS6/SL5I0UWOg8s
2B5HsHfn4Ph9gjxDMmiUl1UCAwEAAQ==
-----END PUBLIC KEY-----
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Tue Aug 24, 2021 5:19 pm

Speedtest.net reports 3.69 by 2.30 (inside VPN) and Google Speed Test reports 25.6 by 2.42 (inside the VPN)
This further reinforces my assumption that it is a timing/small packet/fragmented packet issue. For OTE, these two tests are indistinguishable from each other by anything than possibly packet size. The difference between Ookla and Google by IP addresses is only visible at the Virgin end.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Tue Aug 24, 2021 7:44 pm

only waste your time looking for evil intention where mere stupidity is not sufficient as an explanation
Brilliant! I will try what you suggested tomorrow. Thanks again for all the help.

Andrew.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Fri Aug 27, 2021 7:44 pm

Speedtest.net reports 3.69 by 2.30 (inside VPN) and Google Speed Test reports 25.6 by 2.42 (inside the VPN)
This further reinforces my assumption that it is a timing/small packet/fragmented packet issue. For OTE, these two tests are indistinguishable from each other by anything than possibly packet size. The difference between Ookla and Google by IP addresses is only visible at the Virgin end.
I think your stupidity quote was quite apt, the problem was the UK Virgin line not the Greek OTE line. I have a BT backup which is much slower but much higher quality and all the problems have disappeared. I get 90% of capacity at TCP Connection Count 1 (about 8Mbps out of a max 9Mbps) with BT but only about 1Mbps using the Virgin line out of a max 37Mbps.

I will still run the packet inspection as you outlined but ran out of time today as we had to design the system to only use the VPN for internal traffic.

Thanks again.

Andrew.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Fri Sep 10, 2021 3:24 pm

Then download the files and think how to hand them over. Since private messaging is off again,
I am struggling with the SSL side so my suggestion is that once you reply I simply put a temp email address in the reply and edit it off later.

How does that sound.

Thanks,

Andrew.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Fri Sep 10, 2021 3:54 pm

NP, just substitute the @ by something else (like !!!) to make the bots' life harder.
 
AJSG
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 64
Joined: Wed Apr 13, 2016 6:43 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Sat Sep 11, 2021 12:43 am

NP, just substitute the @ by something else (like !!!) to make the bots' life harder.
Thanks,

Andrew.
Last edited by AJSG on Sat Sep 11, 2021 8:52 am, edited 1 time in total.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Solve ISP VPN L2TP throttle (CCR to CCR)

Sat Sep 11, 2021 8:29 am

Message sent, you can edit/remove the post.

Who is online

Users browsing this forum: Google [Bot], GoogleOther [Bot] and 57 guests