OSPF goes down under load on GRE+IPSec link

Hi all,
I’m running a test with two CCR1009-8G-1S-1S+ routers. Routers are connected via SFP+ interfaces. There’s created also GRE tunnel with IPSec Secret between them.

My IPSec proposal is:

> /ip ipsec proposal print 
 0  * name="default" auth-algorithms=sha256 enc-algorithms=aes-256-cbc lifetime=2h pfs-group=modp1536

OSPF runs through the GRE+IPSec tunnel. When I did performance tests I discovered that OSPF frequently lost it’s route. My max throughput was around 500Mbps over GRE+IPSec, but more than 200Mbps throughput cause OSPF to fail. All OSPF routes are lost for a while. OSPF comes up after few seconds, but it repeats again.

There’s a iperf output through GRE+IPSec tunnel.

[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  59.7 MBytes   500 Mbits/sec  422   29.5 KBytes       
[  4]   1.00-2.00   sec  48.3 MBytes   405 Mbits/sec  365   34.9 KBytes       
[  4]   2.00-3.00   sec  58.8 MBytes   493 Mbits/sec  402   26.8 KBytes       
[  4]   3.00-4.00   sec  2.83 MBytes  23.7 Mbits/sec   24   1.34 KBytes       
[  4]   4.00-5.00   sec  0.00 Bytes  0.00 bits/sec    1   1.34 KBytes       
[  4]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec    1   1.34 KBytes       
[  4]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec    2   1.34 KBytes       
[  4]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec    0   1.34 KBytes       
[  4]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec    1   1.34 KBytes       
[  4]   9.00-10.00  sec  0.00 Bytes  0.00 bits/sec    0   1.34 KBytes       
[  4]  10.00-11.00  sec  59.3 MBytes   498 Mbits/sec  385   34.9 KBytes       
[  4]  11.00-12.00  sec  60.8 MBytes   510 Mbits/sec  350   28.2 KBytes       
[  4]  12.00-13.00  sec  61.6 MBytes   517 Mbits/sec  383   36.2 KBytes       
[  4]  13.00-14.00  sec  59.3 MBytes   497 Mbits/sec  348   26.8 KBytes       
[  4]  14.00-15.00  sec  57.8 MBytes   485 Mbits/sec  329   26.8 KBytes       
[  4]  15.00-16.00  sec  57.9 MBytes   486 Mbits/sec  344   24.2 KBytes       
[  4]  16.00-17.00  sec  57.6 MBytes   483 Mbits/sec  338   26.8 KBytes       
[  4]  17.00-18.00  sec  57.9 MBytes   486 Mbits/sec  326   20.1 KBytes       
[  4]  18.00-19.00  sec  58.0 MBytes   486 Mbits/sec  334   26.8 KBytes       
[  4]  19.00-20.00  sec  58.1 MBytes   488 Mbits/sec  330   26.8 KBytes       
[  4]  20.00-21.00  sec  57.6 MBytes   484 Mbits/sec  362   26.8 KBytes       
[  4]  21.00-22.00  sec  48.7 MBytes   409 Mbits/sec  262   38.9 KBytes       
[  4]  22.00-23.00  sec  60.0 MBytes   503 Mbits/sec  438   29.5 KBytes       
[  4]  23.00-24.00  sec  2.46 MBytes  20.6 Mbits/sec   20   1.34 KBytes       
[  4]  24.00-25.00  sec  0.00 Bytes  0.00 bits/sec    1   1.34 KBytes       
[  4]  25.00-26.00  sec  0.00 Bytes  0.00 bits/sec    1   1.34 KBytes       
[  4]  26.00-27.00  sec  0.00 Bytes  0.00 bits/sec    2   1.34 KBytes       
[  4]  27.00-28.00  sec  0.00 Bytes  0.00 bits/sec    0   1.34 KBytes       
[  4]  28.00-29.00  sec  0.00 Bytes  0.00 bits/sec    1   1.34 KBytes       
[  4]  29.00-30.00  sec  0.00 Bytes  0.00 bits/sec    0   1.34 KBytes       
[  4]  30.00-31.00  sec  54.7 MBytes   459 Mbits/sec  395   30.9 KBytes       
[  4]  31.00-32.00  sec  49.0 MBytes   411 Mbits/sec  355   36.2 KBytes       
[  4]  32.00-33.00  sec  63.3 MBytes   531 Mbits/sec  404   36.2 KBytes

When I tried to run OSPF outside of GRE+IPSec tunnel everything worked fine and throughput was around 940Mbps.

Please anyone help me how to make OSPF more reliable?

I’ve found out that problem is not in OSPF process. I did a test with static routing instead of OSPF. Under load (200Mbps) there were still time windows when no traffic pass through, but windows were much shorter than in the case of OSPF. (OSPF takes some time to set up)

It looks like GRE+IPSec interface goes down for a while when traffic is higher 200Mbps+.