hAP ac^2 problems - large latency (>1000ms) for accessing remote VPN server side websites

yhfung · Tue May 01, 2018 7:23 pm

Hello, MikroTik Support Team Members,

I had a problem with hAP ac^2 (RouterOS and firmware upgraded to v6.43rc5) after having crated a VPN connection to the remote RouterOS VPN server. My PC was connected to one of LAN ports, which directed data to the remote VPN server using policy-based routing method. Two websites, namely site 1: http://speedtest.ofca.gov.hk and site 2: www.youtube.com. If I browsed the two websites via the VPN gateway, the latency of http://speedtest.ofca.gov.hk was over 1000ms, which is extremely large and not acceptable. If I browsed www.youtube.com, the pictures in YouTube were extremely slow to be shown.

Please take a look at the following diagram:

Figure 1: Network Configuration for connecting two sites via VPN technology

Setup:

1. Using a PPTP client to connect the remote VPN server (R2) in Country B and form a PP interface named “pptp-out1”

2. R1 in Country A is connected to the Internet via ether1 (WAN)

3. Detach ether5 from “bridge”

4. Ether2, ether3, and ether4 are connected to a logical bridge

5. Create a new bridge named “bridge2”

6. Ether5 is connected to the logical bridge2

7. Assign IP address to bridge2, for example 192.168.90.1/24

8. Using DHCP Setup to create an DHCP server for bridge2 (assigned desired DNS 8.8.8.

9. Using ip->firewall->mangle: chain=prerouting, In. Interface=bridge2, Action=“mark routing”, new routing mark=“VPN-Gateway”

10. Apply ip-firwall-nat, Chain=srcnat, Out. Interface=pptp-out1, Action=masquerade,

11. Add default route 0.0.0.0/0, Gateway=pptp-out1, Routing Mark=VPN-Gateway

When PC2 (without VPN) ran the speed test via the site http://speedtest.ofca.gov.hk, the latency was very small.

When PC1 (with VPN) ran the speed test via the site same site http://speedtest.ofca.gov.hk, the latency was found extremely large, say over 1000ms.

Please note both X86 RouterOS and CCR1009 router do not have these problems, both PC1 and PC2 ran the speed test using the site http://speedtest.ofca.gov.hk. The latency time was very short.

Besides hAP ac^2 has this problem, RB951G also has this problem. I believe other RouterOS on other RouterBoards do have the long latency time.

With the above information, I do believe you will be able to locate the cause of the problem and give the revised code to resolve this issue.

YH

Tue May 01, 2018 8:28 pm

check bandwidth usage over VPN to avoid saturation

check bandwidth availability on upload of router on country 2 (adsl for example only provide 600kbps of sustained reliably upload bandwidth)

try another type of vpn, like SSTP, PPTP has serious performance limitations

how many bandwidh are you getting across that vpn??

yhfung · Wed May 02, 2018 5:11 am

Thank you for your reply.

check bandwidth usage over VPN to avoid saturation

#
When I used CCR1009 or X86 RouterOS, the maximum throughput after connecting the remote VPN server using PPTP was 20Mbps. When I switched to hAP ac^2 or RB951G, the maximum VPN speed is around 15Mbps. The current problem is the latency and not the throughput.

check bandwidth availability on upload of router on country 2 (adsl for example only provide 600kbps of sustained reliably upload bandwidth)

try another type of vpn, like SSTP, PPTP has serious performance limitationshow many bandwidh are you getting across that vpn??

#
If the latency problem came from PPTP, how comes when I used CCR1009 or X86 RouterOS, these two machines do not have the latency problem. Based on these findings, I guess the codes of "CCR1009 or X86 RouterOS" and the codes of "hAP ac^2 and RB951G" are different in somehow.

The same latency problems also happened when two hAP ac^2 (R1, R2) were cascaded together. WAN port of R2 was connected to the Internet and WAN port of R1 was connected to one of LAN ports of R2. R1 was connected to the R2 via VPN. A notebook was connected the Ether5 (configured to connect the R2's VPN server) of R1. Then in the notebook, I checked the latency using the http://speedtest.ofca.gov.hk. I found the latency was over 1000ms.

YH

squeeze · Wed May 02, 2018 10:00 pm

Can you test with something else, e.g. dslreports.com/speedtest and http://www.dslreports.com/tools/pingtest

CZFan · Wed May 02, 2018 10:22 pm

You know what I find unacceptable, users running "RC" versions, which is "expected" to be buggy, and when they experience problems they come and write in big letters here with their red crayons!

Thu May 03, 2018 12:12 am

You know what I find unacceptable, users running "RC" versions, which is "expected" to be buggy, and when they experience problems they come and write in big letters here with their red crayons!

i agree

yhfung · Thu May 03, 2018 4:12 am

Can you test with something else, e.g. dslreports.com/speedtest and http://www.dslreports.com/tools/pingtest

squeeze,

Thank you for your link, which gives me more information about the latency to different places.

In the following context, VPN client is referred to PPTP VPN client. I do believe other types of VPN protocols have the same issue too.

Figures 1-3 show the graphical results without and with VPN. Listings 1-3 show the corresponding numerical results without and with VPN. If we compare the results shown in Figures 2 and 3, both of them were with VPN. Figure 2 (hAP ac^2 with VPN) shows poor performance in "pingtest". However Figure 3 (CCR1009 with VPN) shows very good results in latency test.

As I have described the ping test problems after VPN not only happens on hAP ac^2 but also stable version of RB951G as well. This is not a general configuration problem but internal software coding BUGs for VPN client on some CPUs. I found this problem when I purchased a CCR1009 router but did not realise the cause of problem coming from the different RouterOS on different CPUs.

CCR1009 and X86 RouterOS with VPN client do not have large latency problems.
hAP ac^2 (v6.43rc5) and RB951 (v6.40.6) with VPN client do have large latency problems.

MikroTik Engineering Team should help us to resolve this large latency problem if router has a VPN connection to somewhere (does not need to across different country but within the same as office) since the problems are easily replicated without too much difficulties by anyone and anywhere.

Figure 1: pingtest graphical result, hAP ac^2 without VPN

Figure 2: pingtest graphical result, hAP ac^2 with VPN

Figure 3: pingtest graphical result, CCR1009 ac^2 with VPN

Listing 1: pingtest numerical result, hAP ac^2 without VPN

Location	IP	Min	+PDV
NSW, AUS	54.153.204.70	9999	+0ms	
SC, Brazil	200.237.196.90	426	+8.3ms	
SP, Brazil	18.231.185.113	362	+11.7ms	
Manitoba, Canada	192.219.0.94	256	+12.8ms	
ON, Canada	206.248.155.54	250	+10.5ms	
QC, Canada	173.243.192.222	266	+14.8ms	
AMS, Netherlands	5.153.60.125	274	+11.5ms	
Germany, EU	95.172.92.166	264	+13ms	
Ireland, EU	54.154.207.39	228	+12.1ms	
London, UK	88.80.191.58	298	+197.8ms	
NL, EU	104.155.37.25	330	+241.6ms	
Zurich, EU	83.150.0.50	286	+14.3ms	
Tokyo, Japan	54.65.9.39	92	+9.1ms	
Singapore	52.221.199.170	9999	+0ms	
Taiwan	104.155.201.41	96	+13.2ms	
CA, USA	64.140.161.58	192	+11.5ms	
CO, USA	72.5.102.138	9999	+0ms	
DC, USA	65.79.226.210	260	+10.9ms	
DE, USA	162.151.17.198	260	+11.1ms	
FL, USA	99.24.18.30	288	+100.2ms	
GA, USA	104.153.104.126	244	+9.3ms	
IA, USA	104.197.0.102	9999	+0ms	
IL, USA	162.248.92.123	248	+20.9ms	
IN, USA	99.24.18.74	242	+14.1ms	
KY, USA	199.193.180.23	288	+482.1ms	
LA, USA	162.248.93.162	190	+11.9ms	
MI, USA	68.85.49.234	256	+11.7ms	
MN, USA	64.90.65.50	248	+13.7ms	
MO, USA	99.24.18.50	240	+13.3ms	
NC, USA	99.24.18.90	268	+81.3ms	
NY, USA	162.248.95.144	256	+13.4ms	
OH, USA	99.24.18.58	240	+11.6ms

#

Listing 2: pingtest numerical result, hAP ac^2 with VPN

hap ac2 With VPN to Country B VPN server
Location	IP	Min	+PDV
NSW, AUS	54.153.204.70	9999	+0ms	
SC, Brazil	200.237.196.90	450	+10438.1ms	
SP, Brazil	18.231.185.113	2836	+17738.5ms	
Manitoba, Canada	192.219.0.94	1084	+5804.9ms	
ON, Canada	206.248.155.54	838	+19765.8ms	
QC, Canada	173.243.192.222	456	+5168.7ms	
AMS, Netherlands	5.153.60.125	1476	+6015.9ms	
Germany, EU	95.172.92.166	2236	+9581.8ms	
Ireland, EU	54.154.207.39	324	+21305.4ms	
London, UK	88.80.191.58	604	+8324.6ms	
NL, EU	104.155.37.25	3996	+1945.1ms	
Zurich, EU	83.150.0.50	836	+11882.1ms	
Tokyo, Japan	54.65.9.39	9999	+23985.6ms	
Singapore	52.221.199.170	9999	+0ms	
Taiwan	104.155.201.41	1068	+18538.4ms	
CA, USA	64.140.161.58	272	+11370ms	
CO, USA	72.5.102.138	1576	+10291.3ms	
DC, USA	65.79.226.210	468	+6765.1ms	
DE, USA	162.151.17.198	296	+2214.5ms	
FL, USA	99.24.18.30	388	+2367.4ms	
GA, USA	104.153.104.126	468	+12642.4ms	
IA, USA	104.197.0.102	9999	+0ms	
IL, USA	162.248.92.123	1974	+10927.9ms	
IN, USA	99.24.18.74	326	+1078ms	
KY, USA	199.193.180.23	1830	+481.2ms	
LA, USA	162.248.93.162	860	+5792.2ms	
MI, USA	68.85.49.234	288	+8570.4ms	
MN, USA	64.90.65.50	486	+6174.7ms	
MO, USA	99.24.18.50	300	+2578.6ms	
NC, USA	99.24.18.90	330	+3027.9ms	
NY, USA	162.248.95.144	2270	+20077ms	
OH, USA	99.24.18.58	512	+363.5ms

#

Listing 3: pingtest numerical result, CCR1009 with VPN

Location	IP	Min	+PDV
NSW, AUS	54.153.204.70	9999	+0ms	
SC, Brazil	200.237.196.90	442	+56.2ms	
SP, Brazil	18.231.185.113	392	+45.8ms	
Manitoba, Canada	192.219.0.94	226	+70.8ms	
ON, Canada	206.248.155.54	274	+70.2ms	
QC, Canada	173.243.192.222	278	+73.3ms	
AMS, Netherlands	5.153.60.125	278	+150ms	
Germany, EU	95.172.92.166	264	+89.8ms	
Ireland, EU	54.154.207.39	314	+67.8ms	
London, UK	88.80.191.58	262	+85.1ms	
NL, EU	104.155.37.25	330	+35.6ms	
Zurich, EU	83.150.0.50	334	+70.9ms	
Tokyo, Japan	54.65.9.39	110	+50.6ms	
Singapore	52.221.199.170	9999	+0ms	
Taiwan	104.155.201.41	68	+41.6ms	
CA, USA	64.140.161.58	212	+35.7ms	
CO, USA	72.5.102.138	242	+52.4ms	
DC, USA	65.79.226.210	280	+56.7ms	
DE, USA	162.151.17.198	272	+54ms	
FL, USA	99.24.18.30	286	+54.7ms	
GA, USA	104.153.104.126	282	+55.5ms	
IA, USA	104.197.0.102	9999	+0ms	
IL, USA	162.248.92.123	254	+42.3ms	
IN, USA	99.24.18.74	264	+50.5ms	
KY, USA	199.193.180.23	262	+45.4ms	
LA, USA	162.248.93.162	206	+58ms	
MI, USA	68.85.49.234	266	+38.5ms	
MN, USA	64.90.65.50	244	+61.7ms	
MO, USA	99.24.18.50	266	+51.3ms	
NC, USA	99.24.18.90	296	+40.2ms	
NY, USA	162.248.95.144	298	+63.1ms	
OH, USA	99.24.18.58	262	+36.1ms

yhfung · Thu May 03, 2018 4:20 am

CZFan and chechito,

Please note the same problem also happens in the old model RB951G. I included the name hAP ac^2 to arouse MikroTik's and other people's interest and concerns. The inclusion of firmware version to reflect the current status. If MikroTik does not modify the code, I do believe large latency issue after VPN will also happen in any so-called stable version.

These are general problems in RouterOS on some CPUs.

YH

Thu May 03, 2018 5:02 am

CZFan and chechito,

Please note the same problem also happens in the old model RB951G. I included the name hAP ac^2 to arouse MikroTik's and other people's interest and concerns. The inclusion of firmware version to reflect the current status. If MikroTik does not modify the code, I do believe large latency issue after VPN will also happen in any so-called stable version.

These are general problems in RouterOS on some CPUs.

YH

I have a SSTP VPN deployment to manage my customers and works well, the performance depends on the wan performance connection

variety of mikrotik devices, hap lite, RB750R2, RB750GR2 RB750GR3 Rb951Ui, RB951G, RB1100AHX2, RB1100AHX4, RB3011, CCR1009, CCR1006, CCR1036, SXTAC, LHG5, DISC LITE5, SXT LITE 5, no particular failure with vpn detected on some specific platform

all the problem i had in the past with my vpn´s is related with some issue on ISP on some VPN end point, some times packet loss only makes notorious on vpn because some equipment on some isp´s give priority to ping making hard to detect packet loss, but a ping encapsulated in vpn makes it evident

but i dont use RC routerOS versions

maybe you can try posting this on RC topic

yhfung · Thu May 03, 2018 6:26 am

chechito,

Thank you for your advice. If I do not have problems with hAP ac^2, I generally use stable RouterOS version and do not use any rc version. If PPTP does not work, sometimes I use SSTP or L2TP over IPSec.

Since you have many devices that can be used for verifying this VPN client long ping time issue, if you do not mind, please use the one RB951G that I have here. Upgrade it the current stable version of v6.40.8 (at the time of writing this post). Make a VPN connection to any site that you want and connect the LAN port whose data will be directed to the remote VPN gateway via policy-based routing. Please carry the ping test http://www.dslreports.com/tools/pingtest for with VPN and without VPN.

I have done two tests using VPN and without VPN, as shown Figures 1 and 2, respectively. As the data presented, it clearly shows that there are problems when RB951G was connected to a VPN server in other country (I have made the same test within the same room, both VPN client and VPN server placed in the same location, the results are the same).

By the way, I have already reported this issue under the current v6.43rc thread. I hope MikroTik can realise the importance of this issue which would have a great impact on the company image and product sales of MikroTik. Since this routing issue, I do believe MikroTik have the ability to resolve this quickly after they can replicate this problem.

YH

Figure 1: RB951G with VPN

Figure 2: RB951G without VPN

yhfung · Thu May 03, 2018 4:02 pm

I got a reply from MikroTik Support Team as shown below:

1) mark-routing and fasttrack-connection is not possible on the same traffic, so first i suggest to disable fasttrack-connection rules, reboot, to clear conntrack table, get your policy routing going, and then re-apply fasttrack-connection for default routing table traffic.

2) your policy routing is too basic. first you need to need to mark all coneectiontions with connection-mark, and and only then mark all packets to necessary direction from those connections with new routing mark.

After having disabled (I did not re-activate the FastTrack) with the FastTrack in the ip->firewall->filter, the ping time with VPN became very small. Now I can browse the YouTube smoothly without any interruptions.

Concerning the addition of FastTrack staring from v6.30 (not sure the exact version), there were discussions about the impact on VPN connection, which I did not realised. If we need the FastTrack and VPN client working smoothly, we need to add some tasks as described in the second part of the replied e-mail. Otherwise the WAN->LAN speed will be degraded to around 500Mbps for each TCP download stream.

Both RB951 and hAP ac^2 by default configuration, FastTracks were added to the firewall filter rules, with VPN client, the long ping time problem came. However, there is no default configuration filter rules such as FastTrack rules added to CCR1009 and X86 RouterOS. When these router with VPN client connected to other VPN servers, they did not have any problem with the long ping time issue.

What is the disadvantage of disabling the FastTrack in RB951G and hAP ac^2, the throughput for single TCP stream will be reduced. For instance, the download speed for hAP ac^2 will be reduced by half, down to 500Mbps. If we want to maintain high throughput (9xxMbps), we need to add more rules as described in the second of part of the reply. However due to limited knowledge, I still do not know how to achieve it.

Finally, thank MikroTik Support to resolve this issue so quickly.

YH

squeeze · Thu May 03, 2018 9:02 pm

Very useful to know. Thank you very much.

hAP ac^2 problems - large latency (>1000ms) for accessing remote VPN server side websites [SOLVED]

hAP ac^2 problems - large latency (>1000ms) for accessing remote VPN server side websites

Re: hAP ac^2 problems - large latency (>1000ms) for accessing remote VPN server side websites

Re: hAP ac^2 problems - large latency (>1000ms) for accessing remote VPN server side websites

Re: hAP ac^2 problems - large latency (>1000ms) for accessing remote VPN server side websites

Re: hAP ac^2 problems - large latency (>1000ms) for accessing remote VPN server side websites

Re: hAP ac^2 problems - large latency (>1000ms) for accessing remote VPN server side websites

Re: hAP ac^2 problems - large latency (>1000ms) for accessing remote VPN server side websites

Re: hAP ac^2 problems - large latency (>1000ms) for accessing remote VPN server side websites

Re: hAP ac^2 problems - large latency (>1000ms) for accessing remote VPN server side websites

Re: hAP ac^2 problems - large latency (>1000ms) for accessing remote VPN server side websites

Re: hAP ac^2 problems - large latency (>1000ms) for accessing remote VPN server side websites [SOLVED]

Re: hAP ac^2 problems - large latency (>1000ms) for accessing remote VPN server side websites

Who is online