Community discussions

 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

hEX - IPsec Tunnel slow

Fri Feb 09, 2018 9:01 pm

Hi,

I just bought two hEX Routers (750G r3, 6.41.2) and set up an IPsec tunnel between them. Both are running in Bridge Mode behind another DSL Router. I only want to use them as a VPN Gateway. Problem is that I only get about 16 MBit/s throughput. Internet connection should allow 40 MBit/s. I disabled the firewall by deleting all rules. How can I improve the IPsec tunnel speed?

My configuration:
/ip ipsec peer print
 0     ;;; vpn10
       address=88.77.66.44/32 auth-method=pre-shared-key
       secret="xxx" generate-policy=no
       policy-template-group=default exchange-mode=ike2
       send-initial-contact=yes hash-algorithm=sha256 enc-algorithm=aes-256
       dh-group=modp4096 lifetime=1d dpd-interval=2m

/ip ipsec policy print
 1  A  ;;; vpn10
       src-address=192.168.20.0/24 src-port=any dst-address=192.168.10.0/24
       dst-port=any protocol=all action=encrypt level=require
       ipsec-protocols=esp tunnel=yes sa-src-address=0.0.0.0
       sa-dst-address=88.77.66.44 proposal=secure-proposal ph2-count=1
       
/ip ipsec proposal print
1    name="secure-proposal" auth-algorithms=sha256 enc-algorithms=aes-256-cbc
      lifetime=30m pfs-group=modp4096
Thank you,
Niclas
 
User avatar
CZFan
Forum Guru
Forum Guru
Posts: 1392
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa, Randburg
Contact:

Re: hEX - IPsec Tunnel slow

Sat Feb 10, 2018 7:49 am

Hi,

I just bought two hEX Routers (750G r3, 6.41.2) and set up an IPsec tunnel between them. Both are running in Bridge Mode behind another DSL Router. I only want to use them as a VPN Gateway. Problem is that I only get about 16 MBit/s throughput. Internet connection should allow 40 MBit/s. I disabled the firewall by deleting all rules. How can I improve the IPsec tunnel speed?

My configuration:
/ip ipsec peer print
 0     ;;; vpn10
       address=88.77.66.44/32 auth-method=pre-shared-key
       secret="xxx" generate-policy=no
       policy-template-group=default exchange-mode=ike2
       send-initial-contact=yes hash-algorithm=sha256 enc-algorithm=aes-256
       dh-group=modp4096 lifetime=1d dpd-interval=2m

/ip ipsec policy print
 1  A  ;;; vpn10
       src-address=192.168.20.0/24 src-port=any dst-address=192.168.10.0/24
       dst-port=any protocol=all action=encrypt level=require
       ipsec-protocols=esp tunnel=yes sa-src-address=0.0.0.0
       sa-dst-address=88.77.66.44 proposal=secure-proposal ph2-count=1
       
/ip ipsec proposal print
1    name="secure-proposal" auth-algorithms=sha256 enc-algorithms=aes-256-cbc
      lifetime=30m pfs-group=modp4096
Thank you,
Niclas
If the data sent through VPN is not sensitive data, maybe use PPTP VPN, much lighter and faster
MTCNA, MTCTCE, MTCRE & MTCINE
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Sat Feb 10, 2018 11:58 am

The data is more or less sensitive, but I want this strong encryption. Shouldn't the hEX routers be much more powerful at IPsec? Two cores of the CPU are just at about 8%.
 
User avatar
CZFan
Forum Guru
Forum Guru
Posts: 1392
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa, Randburg
Contact:

Re: hEX - IPsec Tunnel slow

Sat Feb 10, 2018 12:29 pm

Yes, I would think you should get ~50Mb.

Do you get 16Mb both ways? Not maybe limited by one sides upload speed?
MTCNA, MTCTCE, MTCRE & MTCINE
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Sat Feb 10, 2018 1:15 pm

Site 1 has 50/10 MBit/s (Down/up) and site 2 has 100/50 MBit/s. For testing I was at site 1 and downloaded (https) a file from site 2. So there is a bandwidth limit of 50 MBit/s but I only got 16 MBit/s through the IPsec tunnel.

Some ideas: Would it be better to use IPv6 between the sites? Should I play with MTU?
 
andlil
just joined
Posts: 13
Joined: Sat Oct 08, 2016 7:38 pm
Location: Sweden

Re: hEX - IPsec Tunnel slow

Sun Feb 11, 2018 10:30 am

Site 1 has 50/10 MBit/s (Down/up) and site 2 has 100/50 MBit/s.
If those are the correct numbers I am surprised you get 16 MBit/s, since upload speed on site 1 is 10 MBit/s...
 
msatter
Forum Guru
Forum Guru
Posts: 1221
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: hEX - IPsec Tunnel slow

Sun Feb 11, 2018 11:45 am

Try it with a pre-Bridge firmware and that is 6.40.5. I had the strangest things happening with the 750Gr3 on the Bridged firmware version and RC.

https://mikrotik.com/download/archive
Two RB760iGS (hEX S) in series. One does PPPoE and both do IKEv2.
Running:
RouterOS 6.46Beta / Winbox 3.19 / MikroTik APP 1.3.4
Having an Android device, use https://github.com/M66B/NetGuard/releases (no root required)
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Sun Feb 11, 2018 11:50 am

If those are the correct numbers I am surprised you get 16 MBit/s, since upload speed on site 1 is 10 MBit/s...
I thought I would get 50 MBit/s bandwidth limit in one direction and 10 MBit/s in the other direction. Can you tell me why I am wrong? I think you have misunderstood my setup.

Edit:
Try it with a pre-Bridge firmware and that is 6.40.5. I had the strangest things happening with the 750Gr3 on the Bridged firmware version and RC.
I will test it.
 
User avatar
CZFan
Forum Guru
Forum Guru
Posts: 1392
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa, Randburg
Contact:

Re: hEX - IPsec Tunnel slow

Sun Feb 11, 2018 12:38 pm

Site 1 has 50/10 MBit/s (Down/up) and site 2 has 100/50 MBit/s. For testing I was at site 1 and downloaded (https) a file from site 2. So there is a bandwidth limit of 50 MBit/s but I only got 16 MBit/s through the IPsec tunnel.

Some ideas: Would it be better to use IPv6 between the sites? Should I play with MTU?
Not sure how much knowledge / experience you have, but I would do some packet sniffing to see what is happening on the network
MTCNA, MTCTCE, MTCRE & MTCINE
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Sun Feb 11, 2018 12:48 pm

Try it with a pre-Bridge firmware and that is 6.40.5. I had the strangest things happening with the 750Gr3 on the Bridged firmware version and RC.

https://mikrotik.com/download/archive
I downgraded both routers to 6.40.5, but the bridge option still exists and speed hadn't changed. Is this what you expected? I thought pre-Bridge means that this option is not available.
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Sun Feb 11, 2018 1:22 pm

Not sure how much knowledge / experience you have, but I would do some packet sniffing to see what is happening on the network
Which packets shall I sniff and where? How can I see if somethings wrong?
 
User avatar
CZFan
Forum Guru
Forum Guru
Posts: 1392
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa, Randburg
Contact:

Re: hEX - IPsec Tunnel slow

Sun Feb 11, 2018 3:30 pm

Mikrotik has a packet sniffer in Tools menu, sniff the bridge in both directions for IP of your PC while you doing a download.Save the sniffed packet file

You can then view the details with open source product called Wireshark, but you will need in depth knowledge of how protocols, i.e. IP, TCP, UDP, etc work.
MTCNA, MTCTCE, MTCRE & MTCINE
 
msatter
Forum Guru
Forum Guru
Posts: 1221
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: hEX - IPsec Tunnel slow

Sun Feb 11, 2018 3:59 pm

Try it with a pre-Bridge firmware and that is 6.40.5. I had the strangest things happening with the 750Gr3 on the Bridged firmware version and RC.

https://mikrotik.com/download/archive
I downgraded both routers to 6.40.5, but the bridge option still exists and speed hadn't changed. Is this what you expected? I thought pre-Bridge means that this option is not available.
These are bridges between the ports on the Mikrotik itself and before 6.41 Master-Slave mode was used:

viewtopic.php?f=21&t=128915
Two RB760iGS (hEX S) in series. One does PPPoE and both do IKEv2.
Running:
RouterOS 6.46Beta / Winbox 3.19 / MikroTik APP 1.3.4
Having an Android device, use https://github.com/M66B/NetGuard/releases (no root required)
 
User avatar
CZFan
Forum Guru
Forum Guru
Posts: 1392
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa, Randburg
Contact:

Re: hEX - IPsec Tunnel slow

Sun Feb 11, 2018 4:10 pm

Try it with a pre-Bridge firmware and that is 6.40.5. I had the strangest things happening with the 750Gr3 on the Bridged firmware version and RC.

https://mikrotik.com/download/archive
I downgraded both routers to 6.40.5, but the bridge option still exists and speed hadn't changed. Is this what you expected? I thought pre-Bridge means that this option is not available.
These are bridges between the ports on the Mikrotik itself and before 6.41 Master-Slave mode was used:

viewtopic.php?f=21&t=128915
I suspect this will make no difference, reason being is that bride goes via cpu, master / slave config did not. but when routing is involved, it has to go via cpu, so will make no difference here
MTCNA, MTCTCE, MTCRE & MTCINE
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Sun Feb 11, 2018 7:42 pm

I looked at the sniffed traffic while downloading a file over https:
  1. There are many many retransmissions (TCP Fast Retransmissions, TCP Retransmissions and TCP Dup ACK)
  2. ESP packet size is 1506
  3. IP packet size (my download) 1436
I never saw IPsec traffic and I don't know what I have to expect, but these restransmissions seem to be problematic.
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Sat Feb 17, 2018 7:49 pm

Hi,
today I had time to build a small test network like this:
Laptop -- hEX -- FritzBox (Ports forwarded) -- Switch (my "test internet") -- FritzBox -- hEX -- Laptop
I tested some different configurations but end up by only 3 MByte/s through the IPsec tunnel (SMB data transfer).

Then I tried another network config:
Laptop -- hEX -- Switch (my "test internet") -- hEX -- Laptop
Now I got about 11 MByte/s through the IPsec tunnel (SMB data transfer). I think this is the maximum network speed as the switch is only able to handle 100MBit/s.

So the problem seems to be the NAT at the FritzBox. Does someone why these boxes are slowing me down?

EDIT: Just tested the second network config with a gigabit switch and got 28 MByte/s. Really impressive.
 
User avatar
CZFan
Forum Guru
Forum Guru
Posts: 1392
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa, Randburg
Contact:

Re: hEX - IPsec Tunnel slow

Sun Feb 18, 2018 10:52 am

100Mbits/s equates to approx. 12,5 MBytes/s, with overhead, etc 11 MBytes/s is about correct.

Move the PPPoE / DSL authentication, etc off the FritzBox to the Mikrotik and configure the FritzBox as a "modem" only
MTCNA, MTCTCE, MTCRE & MTCINE
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Tue Jun 12, 2018 5:34 pm

Sorry for this late reply. Had no time to test with VDSL modems. Now my setup looks like this:
Local Network 1 -- Mikrotik  -- Modem (100/40 Down/Up) --  Internet -- Modem (50/10 Down/Up) -- Mikrotik -- Local Network 2
The Mikrotik Routers are connected via a IPSec GRE Tunnel. If I ping from one Local Network to the other I have a Latency of about 40ms.
Now the problem: I only get 2,3 MByte/s via SMB, but the full speed of about 4,3 MByte/s via FTP. Is there anything I can do to improve the SMB speed? Some Clients need SMB so there is no option to use FTP only. I already set the SMB Version to 3.1.

Tunnel Configuration:
Actual MTU: 1406
DSCP: inherit
Dont fragment: no
Clamp TCP MSS: yes
Allow fast path: no
 
Van9018
Long time Member
Long time Member
Posts: 505
Joined: Mon Jun 16, 2014 6:26 pm
Location: Canada - Abbotsford

Re: hEX - IPsec Tunnel slow

Wed Jun 13, 2018 4:08 am

Nothing more to do on the router if FTP maxes out your connection.

SMB is a chatty protocol, latency is a killer. You'll have to look more into SMB to see if it can be tuned for better throughput on high latency networks.
 
User avatar
nichky
Long time Member
Long time Member
Posts: 526
Joined: Tue Jun 23, 2015 2:35 pm

Re: hEX - IPsec Tunnel slow

Wed Jun 13, 2018 7:43 am

@nicku How your firewall looks like? use RAW instead of NAT.
Nikola Suminoski
MikroTik Consultan
MTCRE l MTCWE

!) Safe Mode is your friend;
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Fri Jun 15, 2018 12:06 am

@nichky: my Firewall settings are the defaults:
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="Admin page" dst-port=80 protocol=tcp
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf:  drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN
Is there anything I can do better?

@Van9018: I tried playing with the MTU and I set DisableBandwidthThrottling on the windows client, which at least stabilizes the speed to 20 Mbps. With FTP I get 39 Mbps. Do you have any suggestions for me on how tu tune that SMB performance?
 
User avatar
nichky
Long time Member
Long time Member
Posts: 526
Joined: Tue Jun 23, 2015 2:35 pm

Re: hEX - IPsec Tunnel slow

Fri Jun 15, 2018 1:05 am

So you playing with tunnel mode and you didn't set up NAT or RAW rule to accept your LAN and LAN from other side. Can you ping other side?
Nikola Suminoski
MikroTik Consultan
MTCRE l MTCWE

!) Safe Mode is your friend;
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Fri Jun 15, 2018 1:11 am

I changed my initial vpn configuration to a GRE Tunnel (Interface) with IPSec secret and set up a IP Route, like I wrote some posts ago. And yes ping and SMB is working. Just too slow.
 
User avatar
nichky
Long time Member
Long time Member
Posts: 526
Joined: Tue Jun 23, 2015 2:35 pm

Re: hEX - IPsec Tunnel slow

Fri Jun 15, 2018 1:20 am

Can you quickly do only tunnel without IPSec and do test. And how you checking your result by B-test, watchout B-test is CPU extensive.
Nikola Suminoski
MikroTik Consultan
MTCRE l MTCWE

!) Safe Mode is your friend;
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Fri Jun 15, 2018 1:38 am

Without IPSec I get slightly better performance, but it is just a plus of about 1 Mbps at peak. For testing I copy a file from a SMB share of a QNAP NAS to my computer.

Concerning WAN limits: I have a download of 50 MBit/s and on the other site I have a upload of 40 MBit/s. So the FTP speed I measured of 39 MBit/s is the bandwidth limit.
 
Illuru
just joined
Posts: 2
Joined: Fri Nov 02, 2018 2:30 pm

Re: hEX - IPsec Tunnel slow

Sun Nov 25, 2018 12:07 pm

I have axact the same situation.
Any suggestions for the solution? The sniiffer collects a lot of TCP Retransmission.
Thanks in advanced
 
mkx
Forum Guru
Forum Guru
Posts: 2925
Joined: Thu Mar 03, 2016 10:23 pm

Re: hEX - IPsec Tunnel slow

Sun Nov 25, 2018 2:50 pm

I have axact the same situation.
Any suggestions for the solution? The sniiffer collects a lot of TCP Retransmission.
Thanks in advanced
Do you see retransmissions on VPN interface or on underlying interface?
BR,
Metod
 
User avatar
otgooneo
Trainer
Trainer
Posts: 570
Joined: Tue Dec 01, 2009 3:24 am
Location: Mongolia
Contact:

Re: hEX - IPsec Tunnel slow

Mon Mar 25, 2019 4:40 am

I just had a chance to configure two RB750Gr3 with ipsec. No firewall at all on both routers, uplinks allow 100mbps between routers. IPSec uses AES128 for encryption algorithm and SHA1 for authentication algorithm. ~10Mbps ipsec traffic, CPU gets ~38%. One of the CPU reaches to 60% among 4 CPUs. Estimating IPsec max throughput will be not more than 30Mbps in this setup. That is quite slow performance considering 880MHz x 2 CPUs and plus hardware offloading enabled. Even RB450G (single core, no hardware offload on ipsec) can reach to similar performance. On top of slow throughput, RB750Gr3 adds notable delays on IPSec traffic even in case of very tiny traffic usage. In some cases, I see ICMP loss on ipsec packets while there is no packet loss between WAN addresses of two routers.
Quite upset. Seems like this is the capability of RB750gr3.

Forgot to mention that brigde settings enable between ether2-ether5. But this should be offloaded to switch chip. So I don`t think this affects to overall performance that much.
Last edited by otgooneo on Tue Apr 02, 2019 9:16 am, edited 1 time in total.
----------------------------
Want to learn more and more...
 
User avatar
voxmaster
just joined
Posts: 12
Joined: Tue Sep 29, 2015 9:53 am
Location: Kyiv

Re: hEX - IPsec Tunnel slow

Tue Mar 26, 2019 11:35 pm

Same here, after upgrade from version 6.42.7
Last edited by voxmaster on Wed Mar 27, 2019 9:18 am, edited 2 times in total.
 
anav
Forum Guru
Forum Guru
Posts: 2967
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada

Re: hEX - IPsec Tunnel slow

Wed Mar 27, 2019 12:30 am

So the question remains, is this a software bug or a hardware limitation?
I'd rather manage rats than software. Follow my advice at your own risk! (Sob & mkx forced me to write that!)
 
User avatar
voxmaster
just joined
Posts: 12
Joined: Tue Sep 29, 2015 9:53 am
Location: Kyiv

Re: hEX - IPsec Tunnel slow

Wed Mar 27, 2019 9:21 am

Sorry. My mistake. No problems with IPsec over L2TP. It was ISP problem in my case :roll:
 
Zdravac
just joined
Posts: 1
Joined: Wed Sep 25, 2019 3:11 pm

Re: hEX - IPsec Tunnel slow

Wed Sep 25, 2019 3:33 pm

Hello!

First post, so my greetings to All!

I have a noob question about hardware encryption in hEX S;
My VPN Provider (ExpressVPN) supports AES-256-CBC, and I see that hEX S support this kind of hardware encryption.

But the problem might be in RSA certificate identification by the hashing algorithm SHA-512 when hEX S supports only SHA-256.

My question is will it be anyhow hardware encrypted if I select in IPSEC settings SHA512 and if someone has those combination (hEX S and ExpressVPN), which VPN performance could I expect and is it hardware encryption possible with ExpressVPN?

Who is online

Users browsing this forum: No registered users and 38 guests