Community discussions

 
Chrisszzyy
just joined
Topic Author
Posts: 5
Joined: Thu Sep 07, 2017 3:57 pm

CHR <-> RB750Gr3 via GRE over IPSEC Performance issue

Thu Sep 07, 2017 5:55 pm

Hi All,

So I have a virtualized CHR running in a datacentre which has a GRE Tunnel running over IPSEC to my home router, which is a hEX v3 (RB750Gr3).

For some reason when using an IPSEC tunnel, I only seem to be able to achieve around 20Mbps with a bandwidth test from the CHR to RB750Gr3. However if I perform the bandwidth test directly from the CHR to the public IP of the RB750Gr3 (Directly over the internet instead of IPSEC tunnel), I am able to achieve maximum throughput.

The resources allocated to the CHR are:

CPU: 4 Cores of a D-1531 Xeon processor,
Memory: 1GB,
Network: VirtIO Adapters

When running a bandwidth test over the IPSEC tunnel, the CPU of the CHR sits at 25% with one core maxed out at 100%. The CPU of the hEX v3 sits happily around 10%.

I'm using the following IPSEC settings:

Auth Algorithm: sha1
Encr. Algorithm: aes-256-cbc
PFS group: modp1024


Does anyone have any idea how I could improve performance over IPSEC? I realize the hEX has hardware acceleration, but shouldn't I be achieving more than a measly 20Mbps over an IPSEC tunnel between these routers?
 
idlemind
Forum Guru
Forum Guru
Posts: 1101
Joined: Fri Mar 24, 2017 11:15 pm
Location: USA

Re: CHR <-> RB750Gr3 via GRE over IPSEC Performance issue

Fri Sep 08, 2017 12:08 am

Do you have a server with a single CPU core capable of more than 2.2GHz? Your post clearly stated the performance is capping out 1 core of the CHR. That is your limitation.

viewtopic.php?t=122963

Might be an interesting read, first things first. What version is your CHR? Make sure it is at least up to 6.39 and ensure your hardware and hypervisor is allowing the AES extensions through to the CHR VM.
 
Chrisszzyy
just joined
Topic Author
Posts: 5
Joined: Thu Sep 07, 2017 3:57 pm

Re: CHR <-> RB750Gr3 via GRE over IPSEC Performance issue

Fri Sep 08, 2017 11:46 am

Do you have a server with a single CPU core capable of more than 2.2GHz? Your post clearly stated the performance is capping out 1 core of the CHR. That is your limitation.

viewtopic.php?t=122963

Might be an interesting read, first things first. What version is your CHR? Make sure it is at least up to 6.39 and ensure your hardware and hypervisor is allowing the AES extensions through to the CHR VM.
I'm using 6.40.1 and I've ensured my hardware and hypervisor (Proxmox) allowed AES extensions.

The processor I'm using is the D-1531 with a base clock of 2.20GHz and 2.70GHz turbo.

I'm still stumped; I've tried emulating a KVM64 CPU type as well as the 'host' CPU type. Neither makes any difference to the throughput.
 
pe1chl
Forum Guru
Forum Guru
Posts: 5816
Joined: Mon Jun 08, 2015 12:09 pm

Re: CHR <-> RB750Gr3 via GRE over IPSEC Performance issue

Fri Sep 08, 2017 12:09 pm

Try aes-128-cbc or even no encryption at all (AH protocol).
It is not useful to assign 4 processors to CHR - it will not use them for parallel processing for tasks like this.
 
Chrisszzyy
just joined
Topic Author
Posts: 5
Joined: Thu Sep 07, 2017 3:57 pm

Re: CHR <-> RB750Gr3 via GRE over IPSEC Performance issue

Fri Sep 08, 2017 12:50 pm

Try aes-128-cbc or even no encryption at all (AH protocol).
It is not useful to assign 4 processors to CHR - it will not use them for parallel processing for tasks like this.
Thanks for the suggestion.

I've just tried using AH Protocol and it hasn't made any difference I'm afraid. I'm stumpted.

Any more ideas?
 
idlemind
Forum Guru
Forum Guru
Posts: 1101
Joined: Fri Mar 24, 2017 11:15 pm
Location: USA

Re: CHR <-> RB750Gr3 via GRE over IPSEC Performance issue

Fri Sep 08, 2017 4:02 pm

Try setting the CPU to host or host-passthrough to be 100% certain AES extensions are getting through and enabled.

Disregard: Just read you tried host mode already.
Last edited by idlemind on Fri Sep 08, 2017 6:05 pm, edited 1 time in total.
 
pe1chl
Forum Guru
Forum Guru
Posts: 5816
Joined: Mon Jun 08, 2015 12:09 pm

Re: CHR <-> RB750Gr3 via GRE over IPSEC Performance issue

Fri Sep 08, 2017 4:08 pm

Try aes-128-cbc or even no encryption at all (AH protocol).
It is not useful to assign 4 processors to CHR - it will not use them for parallel processing for tasks like this.
Thanks for the suggestion.

I've just tried using AH Protocol and it hasn't made any difference I'm afraid. I'm stumpted.

Any more ideas?
It must be another problem then. I am using AH protocol between a 2011 and CCR and I can saturate the link.
(which I cannot do using ESP because of the slow CPU in the 2011)
With your 750Gr3 you have accellerated AES and you could do ESP without problem, but apparently for you the CHR side is the bottleneck.
Put a 750Gr3 there too :-)
 
bbs2web
Member Candidate
Member Candidate
Posts: 198
Joined: Sun Apr 22, 2012 6:25 pm
Location: Johannesburg, South Africa
Contact:

Re: CHR <-> RB750Gr3 via GRE over IPSEC Performance issue

Sat Sep 30, 2017 11:43 am

I'm running CHR on Intel Haswell, without TSX, to support high availability failover to Intel Xeon CPU E5-2640v3. I've confirmed AES pass through by booting the CHR guest using CentOS 7 recovery environment.

Confirming 'aes' instruction availability:
grep -m1 -o aes /proc/cpuinfo

We obtain the following benchmarks in the VM:
openssl speed -evp aes-128-cbc
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes                     
aes-128-cbc     511101.85k   547731.20k   555776.60k   560752.67k   558724.44k
openssl speed -evp aes-256-cbc
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes                     
aes-256-cbc     371521.68k   394245.03k   401446.36k   399955.91k   402183.22k
Directly on slowest hardware:
openssl speed -evp aes-128-cbc
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes                     
aes-128-cbc     541865.83k   585278.50k   595671.30k   602248.53k   603339.43k
openssl speed -evp aes-256-cbc
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes                     
aes-256-cbc     398849.01k   423887.98k   430012.84k   431852.89k   432622.25k
That equates to 5.2 Gbps, when using AES 128 bit CBC encoding within the virtual guest. I don't see L2TP IPSec in CHR reporting 'Hardware AEAD' when reviewing the installed SAs either...
 
andriys
Forum Guru
Forum Guru
Posts: 1179
Joined: Thu Nov 24, 2011 1:59 pm
Location: Kharkiv, Ukraine

Re: CHR <-> RB750Gr3 via GRE over IPSEC Performance issue

Sat Sep 30, 2017 8:23 pm

When running a bandwidth test over the IPSEC tunnel, the CPU of the CHR sits at 25% with one core maxed out at 100%.
What is in /tool profile ?
Do you happen to run btest from the CHR itself?
 
Chrisszzyy
just joined
Topic Author
Posts: 5
Joined: Thu Sep 07, 2017 3:57 pm

Re: CHR <-> RB750Gr3 via GRE over IPSEC Performance issue

Fri Jan 19, 2018 2:24 pm

When running a bandwidth test over the IPSEC tunnel, the CPU of the CHR sits at 25% with one core maxed out at 100%.
What is in /tool profile ?
Do you happen to run btest from the CHR itself?
Sorry, I haven't really had time to look at this lately.

Today I've completely rebuilt the router with the 6.41 build but the same issue occurs.

Profile:
Image

BTest to itself (CHR):

Image

Any ideas?
 
AceBlade258
just joined
Posts: 8
Joined: Tue Sep 12, 2017 4:19 am

Re: CHR <-> RB750Gr3 via GRE over IPSEC Performance issue

Sat May 05, 2018 4:06 am

A thought: it does not appear you accounting for various MTU overheads anywhere, so it may be an issue with path maximum transmission unit (MTU) discovery (PMTUD), or TCP maximum segmentation size (TCP MSS). Things to remember:
  • Traditional Ethernet MTU is 1500, so most things use this by default
  • TCP MSS = MTU -40
  • IPSec has an 8 bit overhead: MTU -8
  • If you or your ISP is using PPPoE: MTU -8
  • PMTUD is generally unreliable.

Who is online

Users browsing this forum: No registered users and 4 guests