Community discussions

 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

hEX - IPsec Tunnel slow

Fri Feb 09, 2018 9:01 pm

Hi,

I just bought two hEX Routers (750G r3, 6.41.2) and set up an IPsec tunnel between them. Both are running in Bridge Mode behind another DSL Router. I only want to use them as a VPN Gateway. Problem is that I only get about 16 MBit/s throughput. Internet connection should allow 40 MBit/s. I disabled the firewall by deleting all rules. How can I improve the IPsec tunnel speed?

My configuration:
/ip ipsec peer print
 0     ;;; vpn10
       address=88.77.66.44/32 auth-method=pre-shared-key
       secret="xxx" generate-policy=no
       policy-template-group=default exchange-mode=ike2
       send-initial-contact=yes hash-algorithm=sha256 enc-algorithm=aes-256
       dh-group=modp4096 lifetime=1d dpd-interval=2m

/ip ipsec policy print
 1  A  ;;; vpn10
       src-address=192.168.20.0/24 src-port=any dst-address=192.168.10.0/24
       dst-port=any protocol=all action=encrypt level=require
       ipsec-protocols=esp tunnel=yes sa-src-address=0.0.0.0
       sa-dst-address=88.77.66.44 proposal=secure-proposal ph2-count=1
       
/ip ipsec proposal print
1    name="secure-proposal" auth-algorithms=sha256 enc-algorithms=aes-256-cbc
      lifetime=30m pfs-group=modp4096
Thank you,
Niclas
 
User avatar
CZFan
Forum Veteran
Forum Veteran
Posts: 885
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa
Contact:

Re: hEX - IPsec Tunnel slow

Sat Feb 10, 2018 7:49 am

Hi,

I just bought two hEX Routers (750G r3, 6.41.2) and set up an IPsec tunnel between them. Both are running in Bridge Mode behind another DSL Router. I only want to use them as a VPN Gateway. Problem is that I only get about 16 MBit/s throughput. Internet connection should allow 40 MBit/s. I disabled the firewall by deleting all rules. How can I improve the IPsec tunnel speed?

My configuration:
/ip ipsec peer print
 0     ;;; vpn10
       address=88.77.66.44/32 auth-method=pre-shared-key
       secret="xxx" generate-policy=no
       policy-template-group=default exchange-mode=ike2
       send-initial-contact=yes hash-algorithm=sha256 enc-algorithm=aes-256
       dh-group=modp4096 lifetime=1d dpd-interval=2m

/ip ipsec policy print
 1  A  ;;; vpn10
       src-address=192.168.20.0/24 src-port=any dst-address=192.168.10.0/24
       dst-port=any protocol=all action=encrypt level=require
       ipsec-protocols=esp tunnel=yes sa-src-address=0.0.0.0
       sa-dst-address=88.77.66.44 proposal=secure-proposal ph2-count=1
       
/ip ipsec proposal print
1    name="secure-proposal" auth-algorithms=sha256 enc-algorithms=aes-256-cbc
      lifetime=30m pfs-group=modp4096
Thank you,
Niclas
If the data sent through VPN is not sensitive data, maybe use PPTP VPN, much lighter and faster
MTCNA, MTCTCE, MTCRE & MTCINE
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Sat Feb 10, 2018 11:58 am

The data is more or less sensitive, but I want this strong encryption. Shouldn't the hEX routers be much more powerful at IPsec? Two cores of the CPU are just at about 8%.
 
User avatar
CZFan
Forum Veteran
Forum Veteran
Posts: 885
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa
Contact:

Re: hEX - IPsec Tunnel slow

Sat Feb 10, 2018 12:29 pm

Yes, I would think you should get ~50Mb.

Do you get 16Mb both ways? Not maybe limited by one sides upload speed?
MTCNA, MTCTCE, MTCRE & MTCINE
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Sat Feb 10, 2018 1:15 pm

Site 1 has 50/10 MBit/s (Down/up) and site 2 has 100/50 MBit/s. For testing I was at site 1 and downloaded (https) a file from site 2. So there is a bandwidth limit of 50 MBit/s but I only got 16 MBit/s through the IPsec tunnel.

Some ideas: Would it be better to use IPv6 between the sites? Should I play with MTU?
 
andlil
just joined
Posts: 13
Joined: Sat Oct 08, 2016 7:38 pm
Location: Sweden

Re: hEX - IPsec Tunnel slow

Sun Feb 11, 2018 10:30 am

Site 1 has 50/10 MBit/s (Down/up) and site 2 has 100/50 MBit/s.
If those are the correct numbers I am surprised you get 16 MBit/s, since upload speed on site 1 is 10 MBit/s...
 
msatter
Forum Veteran
Forum Veteran
Posts: 853
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: hEX - IPsec Tunnel slow

Sun Feb 11, 2018 11:45 am

Try it with a pre-Bridge firmware and that is 6.40.5. I had the strangest things happening with the 750Gr3 on the Bridged firmware version and RC.

https://mikrotik.com/download/archive
RB760iGS (hEX S) with the SFP being cooled.
Running:
RouterOS 6.43RC45 / Winbox 3.16 / MikroTik APP 0.69
Cooling a SFP module: viewtopic.php?f=3&t=132258&p=671105#p671105
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Sun Feb 11, 2018 11:50 am

If those are the correct numbers I am surprised you get 16 MBit/s, since upload speed on site 1 is 10 MBit/s...
I thought I would get 50 MBit/s bandwidth limit in one direction and 10 MBit/s in the other direction. Can you tell me why I am wrong? I think you have misunderstood my setup.

Edit:
Try it with a pre-Bridge firmware and that is 6.40.5. I had the strangest things happening with the 750Gr3 on the Bridged firmware version and RC.
I will test it.
 
User avatar
CZFan
Forum Veteran
Forum Veteran
Posts: 885
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa
Contact:

Re: hEX - IPsec Tunnel slow

Sun Feb 11, 2018 12:38 pm

Site 1 has 50/10 MBit/s (Down/up) and site 2 has 100/50 MBit/s. For testing I was at site 1 and downloaded (https) a file from site 2. So there is a bandwidth limit of 50 MBit/s but I only got 16 MBit/s through the IPsec tunnel.

Some ideas: Would it be better to use IPv6 between the sites? Should I play with MTU?
Not sure how much knowledge / experience you have, but I would do some packet sniffing to see what is happening on the network
MTCNA, MTCTCE, MTCRE & MTCINE
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Sun Feb 11, 2018 12:48 pm

Try it with a pre-Bridge firmware and that is 6.40.5. I had the strangest things happening with the 750Gr3 on the Bridged firmware version and RC.

https://mikrotik.com/download/archive
I downgraded both routers to 6.40.5, but the bridge option still exists and speed hadn't changed. Is this what you expected? I thought pre-Bridge means that this option is not available.
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Sun Feb 11, 2018 1:22 pm

Not sure how much knowledge / experience you have, but I would do some packet sniffing to see what is happening on the network
Which packets shall I sniff and where? How can I see if somethings wrong?
 
User avatar
CZFan
Forum Veteran
Forum Veteran
Posts: 885
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa
Contact:

Re: hEX - IPsec Tunnel slow

Sun Feb 11, 2018 3:30 pm

Mikrotik has a packet sniffer in Tools menu, sniff the bridge in both directions for IP of your PC while you doing a download.Save the sniffed packet file

You can then view the details with open source product called Wireshark, but you will need in depth knowledge of how protocols, i.e. IP, TCP, UDP, etc work.
MTCNA, MTCTCE, MTCRE & MTCINE
 
msatter
Forum Veteran
Forum Veteran
Posts: 853
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: hEX - IPsec Tunnel slow

Sun Feb 11, 2018 3:59 pm

Try it with a pre-Bridge firmware and that is 6.40.5. I had the strangest things happening with the 750Gr3 on the Bridged firmware version and RC.

https://mikrotik.com/download/archive
I downgraded both routers to 6.40.5, but the bridge option still exists and speed hadn't changed. Is this what you expected? I thought pre-Bridge means that this option is not available.
These are bridges between the ports on the Mikrotik itself and before 6.41 Master-Slave mode was used:

viewtopic.php?f=21&t=128915
RB760iGS (hEX S) with the SFP being cooled.
Running:
RouterOS 6.43RC45 / Winbox 3.16 / MikroTik APP 0.69
Cooling a SFP module: viewtopic.php?f=3&t=132258&p=671105#p671105
 
User avatar
CZFan
Forum Veteran
Forum Veteran
Posts: 885
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa
Contact:

Re: hEX - IPsec Tunnel slow

Sun Feb 11, 2018 4:10 pm

Try it with a pre-Bridge firmware and that is 6.40.5. I had the strangest things happening with the 750Gr3 on the Bridged firmware version and RC.

https://mikrotik.com/download/archive
I downgraded both routers to 6.40.5, but the bridge option still exists and speed hadn't changed. Is this what you expected? I thought pre-Bridge means that this option is not available.
These are bridges between the ports on the Mikrotik itself and before 6.41 Master-Slave mode was used:

viewtopic.php?f=21&t=128915
I suspect this will make no difference, reason being is that bride goes via cpu, master / slave config did not. but when routing is involved, it has to go via cpu, so will make no difference here
MTCNA, MTCTCE, MTCRE & MTCINE
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Sun Feb 11, 2018 7:42 pm

I looked at the sniffed traffic while downloading a file over https:
  1. There are many many retransmissions (TCP Fast Retransmissions, TCP Retransmissions and TCP Dup ACK)
  2. ESP packet size is 1506
  3. IP packet size (my download) 1436
I never saw IPsec traffic and I don't know what I have to expect, but these restransmissions seem to be problematic.
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Sat Feb 17, 2018 7:49 pm

Hi,
today I had time to build a small test network like this:
Laptop -- hEX -- FritzBox (Ports forwarded) -- Switch (my "test internet") -- FritzBox -- hEX -- Laptop
I tested some different configurations but end up by only 3 MByte/s through the IPsec tunnel (SMB data transfer).

Then I tried another network config:
Laptop -- hEX -- Switch (my "test internet") -- hEX -- Laptop
Now I got about 11 MByte/s through the IPsec tunnel (SMB data transfer). I think this is the maximum network speed as the switch is only able to handle 100MBit/s.

So the problem seems to be the NAT at the FritzBox. Does someone why these boxes are slowing me down?

EDIT: Just tested the second network config with a gigabit switch and got 28 MByte/s. Really impressive.
 
User avatar
CZFan
Forum Veteran
Forum Veteran
Posts: 885
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa
Contact:

Re: hEX - IPsec Tunnel slow

Sun Feb 18, 2018 10:52 am

100Mbits/s equates to approx. 12,5 MBytes/s, with overhead, etc 11 MBytes/s is about correct.

Move the PPPoE / DSL authentication, etc off the FritzBox to the Mikrotik and configure the FritzBox as a "modem" only
MTCNA, MTCTCE, MTCRE & MTCINE
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Tue Jun 12, 2018 5:34 pm

Sorry for this late reply. Had no time to test with VDSL modems. Now my setup looks like this:
Local Network 1 -- Mikrotik  -- Modem (100/40 Down/Up) --  Internet -- Modem (50/10 Down/Up) -- Mikrotik -- Local Network 2
The Mikrotik Routers are connected via a IPSec GRE Tunnel. If I ping from one Local Network to the other I have a Latency of about 40ms.
Now the problem: I only get 2,3 MByte/s via SMB, but the full speed of about 4,3 MByte/s via FTP. Is there anything I can do to improve the SMB speed? Some Clients need SMB so there is no option to use FTP only. I already set the SMB Version to 3.1.

Tunnel Configuration:
Actual MTU: 1406
DSCP: inherit
Dont fragment: no
Clamp TCP MSS: yes
Allow fast path: no
 
Van9018
Member
Member
Posts: 456
Joined: Mon Jun 16, 2014 6:26 pm
Location: Canada - Abbotsford

Re: hEX - IPsec Tunnel slow

Wed Jun 13, 2018 4:08 am

Nothing more to do on the router if FTP maxes out your connection.

SMB is a chatty protocol, latency is a killer. You'll have to look more into SMB to see if it can be tuned for better throughput on high latency networks.
 
User avatar
nichky
Member
Member
Posts: 367
Joined: Tue Jun 23, 2015 2:35 pm

Re: hEX - IPsec Tunnel slow

Wed Jun 13, 2018 7:43 am

@nicku How your firewall looks like? use RAW instead of NAT.
FxUxRx
Struga/Macedonia
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Fri Jun 15, 2018 12:06 am

@nichky: my Firewall settings are the defaults:
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="Admin page" dst-port=80 protocol=tcp
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf:  drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN
Is there anything I can do better?

@Van9018: I tried playing with the MTU and I set DisableBandwidthThrottling on the windows client, which at least stabilizes the speed to 20 Mbps. With FTP I get 39 Mbps. Do you have any suggestions for me on how tu tune that SMB performance?
 
User avatar
nichky
Member
Member
Posts: 367
Joined: Tue Jun 23, 2015 2:35 pm

Re: hEX - IPsec Tunnel slow

Fri Jun 15, 2018 1:05 am

So you playing with tunnel mode and you didn't set up NAT or RAW rule to accept your LAN and LAN from other side. Can you ping other side?
FxUxRx
Struga/Macedonia
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Fri Jun 15, 2018 1:11 am

I changed my initial vpn configuration to a GRE Tunnel (Interface) with IPSec secret and set up a IP Route, like I wrote some posts ago. And yes ping and SMB is working. Just too slow.
 
User avatar
nichky
Member
Member
Posts: 367
Joined: Tue Jun 23, 2015 2:35 pm

Re: hEX - IPsec Tunnel slow

Fri Jun 15, 2018 1:20 am

Can you quickly do only tunnel without IPSec and do test. And how you checking your result by B-test, watchout B-test is CPU extensive.
FxUxRx
Struga/Macedonia
 
nicku
just joined
Topic Author
Posts: 12
Joined: Fri Feb 09, 2018 8:46 pm

Re: hEX - IPsec Tunnel slow

Fri Jun 15, 2018 1:38 am

Without IPSec I get slightly better performance, but it is just a plus of about 1 Mbps at peak. For testing I copy a file from a SMB share of a QNAP NAS to my computer.

Concerning WAN limits: I have a download of 50 MBit/s and on the other site I have a upload of 40 MBit/s. So the FTP speed I measured of 39 MBit/s is the bandwidth limit.

Who is online

Users browsing this forum: NikNikols and 16 guests