IPSec Site-to-Site VPN slow

RouterOS General
1

Hi,

I used to run a IPSec Site-to-Site VPN between two Fritz Box 6490s, but the Fritz Boxes are not powerful enough. Therefore I wanted to switch to Hex RB750Gr3 with hardware encryption and use this for the VPN connection. Due to many legacy things running on the Fritz Box I cannot simply move out those boxes immediately, hence I wanted to just start with the VPN offloading. I started with moving the VPN from one site to a Hex running in bridge mode, but keeping the Fritz Box (and the VPN configuration there) on the other site untouched.

Network overview:
Hex (192.168.1.44) → Fritz Box 1 (192.168.1.0/24 - 400/20MBit) → Internet → Fritz Box 2 (192.168.178.0/24 - 200/8MBit)

I have successfully setup an IPSec Site-to-Site VPN, however the performance is bad. I used to get around 6-7MBit from my NAS (connected to Fritz Box 1) at site 1 to a client on site 2 in SMB or FTP, but with the new setup I end up with less than 1MBit.

I read some forum posts about the MTU/MSS issue and added a NAT mangle rule, but that did not solve the problem.Therefore I hope that you can help to find the root cause for the issue.

My configuration:

/interface bridge
add admin-mac=<MAC> auto-mac=no comment=defconf name=bridge
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile
set [ find default=yes ] html-directory=flash/hotspot
/ip ipsec policy group
add name=ios
/ip ipsec profile
add dh-group=modp1024 dpd-interval=20s enc-algorithm=aes-256 name=fritz-otn
add dpd-interval=disable-dpd enc-algorithm=aes-256,aes-128 name=ios
/ip ipsec peer
add address=<DynDNS Site 2> exchange-mode=aggressive name=fritz-otn profile=fritz-otn
add exchange-mode=aggressive name=goddi-vpn passive=yes profile=ios
/ip ipsec proposal
set [ find default=yes ] enc-algorithms=aes-256-cbc,aes-128-cbc lifetime=8h
add enc-algorithms=aes-256-cbc,aes-256-gcm,aes-128-gcm,3des name=fritz-otn
/ip pool
add name=dhcp ranges=0.0.0.1-0.0.0.254
add name=vpn ranges=192.168.2.201-192.168.2.250
/ip dhcp-server
add address-pool=dhcp disabled=no interface=bridge name=defconf
/ip ipsec mode-config
add address-pool=vpn name=vpn split-include=192.168.1.0/24,192.168.178.0/24
/interface bridge port
add bridge=bridge comment=defconf interface=ether2
add bridge=bridge comment=defconf interface=ether3
add bridge=bridge comment=defconf interface=ether4
add bridge=bridge comment=defconf interface=ether5
add bridge=bridge interface=ether1
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface list member
add comment=defconf interface=ether1 list=WAN
add interface=ether2 list=LAN
add interface=ether3 list=LAN
add interface=ether4 list=LAN
add interface=ether5 list=LAN
/ip address
add address=192.168.88.1/24 comment=defconf disabled=yes interface=ether2 network=192.168.88.0
/ip dhcp-client
add comment=defconf disabled=no interface=bridge
/ip dhcp-server network
add address=0.0.0.0/24 comment=defconf gateway=0.0.0.0 netmask=24
/ip dns
set allow-remote-requests=yes
/ip dns static
add address=192.168.88.1 name=router.lan
/ip firewall address-list
add address=192.168.2.0/24 list=VPN
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=accept chain=input comment="Winbox via WAN" dst-port=8291 log=yes protocol=tcp
add action=accept chain=input comment="allow IPSec" dst-port=500,4500 log=yes log-prefix=allow-ipsec protocol=udp
add action=accept chain=forward comment="allow IPSec" log=yes log-prefix=allow-ipsec protocol=ipsec-esp
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec log-prefix=ip-sec-in
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec log-prefix=ip-sec-out
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf:  drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN log=yes
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN
/ip ipsec identity
add my-id=fqdn:<Site 1 DynDNS> notrack-chain=prerouting peer=fritz-otn remote-id=fqdn:<Site 2 DynDNS>
add auth-method=pre-shared-key-xauth generate-policy=port-strict mode-config=vpn notrack-chain=prerouting peer=goddi-vpn policy-template-group=ios username=<User>
/ip ipsec policy
add dst-address=192.168.178.0/24 peer=fritz-otn proposal=fritz-otn sa-dst-address=77.64.199.4 sa-src-address=192.168.1.44 src-address=192.168.1.0/24 tunnel=yes
add dst-address=192.168.2.0/24 group=ios src-address=192.168.1.0/24 template=yes
/system clock
set time-zone-name=Europe/Berlin
/system logging
add topics=ipsec,!debug
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN

Note: The configuration also contains a working road warrior VPN setup that also works with iOS and Shrew Soft VPN.

Fritzbox -> Mikrotik VPN
2

You have fixed only one end of the connection, right?
It would be required to have an RB750Gr3 at the other side too, so you can have full encrypted speed…
Also make sure you set port-forwarding of ports 500 and 4500 in the Fritzbox to your MikroTik router, the Fritzbox firewall is buggy and will cause IPsec issues when you don’t do that.
(not performance related, but it will drop connections sometimes)
Would you not have NAT in place, you would also need to forward the ESP protocol (50).

3

Hi,

yes, I just switch to RB750Gr3 on one end (the one with the 20MBit up) without touching the other end (same IPsec configuration there as well). I expected to have at least the same speed as the previous setup, however it turned out that I got much less…

The port forwarding of 500, 4500 & ESP are all done, including static route to the Mikrotik for the remote net. NAT is done on both sides.

I did some further investigations and I only get the slow speed when connected to the Gbit LAN on site 2, with Wifi there I can max out the 20MBit up of site 1. Strange…Expert question: Is there any valid reason why this could be the case due to LAN/Wifi differences??(apart from the Fritz Box being bad at this :wink:).

My second RB750Gr3 arrived today and will upgrade my site 2 as well. I will report back my results when I completed the new setup (won’t have time this weekend, so will be next week at the earliest).

4

I have no idea about that, I run a GRE/IPsec VPN between one site with a 250/250 fiber connection with a CCR1009, and another site which is a 70/10 VDSL with Fritzbox and RB750Gr3 behind it, and I have no problem maxing out the VPN (limited by the VDSL speed).

5

For just IPSec I don’t think you need a route. Check out the packet flow diagram. The logic says you need a src-nat rule with action=Accept when the packet’s dest ip is the remote network. This prevents the packet from being masqueraded. Without a route, that packet will go out the wan interface, but the IPSec policy will match and at that time, the IPSec will grab the packet, wrap it in IPSec headers and drop it back into the routing logic with a dest-ip of the remote public IP.
Maybe the route is causing issue?

Next you can check Tools >Profile, that’ll show CPU levels. Does the CPU have a high usage when maxing out the IPSec connection? If the process that’s high is encryption/decription, then that could mean that encryption/decription is happening in the CPU and not hardware. Check that you’re using AES.

6

Hi,

the static route is on the fritz box 1 side in order to route the remote net 192.168.178.0/24 to the mikrotik 192.168.1.44. Also I could analyze that it’s not on the Mikrotik side, as when I’m on the site 2 with Wifi I can max out the 20Mbit upstream. So there seems to be some kind of issue on the fritz box when routing IPSec traffic into the LAN interfaces. I can rule out any faulty LAN installation in my home as I connected directly to the fritz box. What is really strange: When I had setup the IPsec between the two fritz boxes I got around 6-7MBit, however I do not remember what hash algo’s, encryption, etc. was negotiated between them. Maybe there is a different set used compared to the previous setup and this causes a different load on the fritz box impacting the routing to the LAN interfaces. At least I cannot see high load via the web UI but the insight into the fritz box is very limited…

I just got my second mikrotik for site 2 and will work on the setup this week. I will report back here with my results.

7

Update with second Mikrotik :

I did setup the site 2 with a second Hex RB750Gr3 in bridge mode as well, so both sites are running with the same setup. Furthermore I upgraded the upstream of site 1 from 20MBit to 40Mbit and I can max. out those 40Mbit upstream when located at the site 2 LAN. So very happy with the VPN performance of RB750Gr. So the issue was on the Fritz Box side.

The only issue I’m currently facing is that the speed is sometimes dropping to only 2-4 Mbit and I need to further investigate here.