Page 1 of 1

Unstable IPSEC over PPPOE interface

Posted: Mon Oct 07, 2019 10:59 am
by hancik
Hello board,

I have strange problem with stability of IPSEC tunnel - it's site-to-site one with following schema:
[siteA-client ip 192.168.0.10] --- [RB2011 with PPPOE interface] --- IPSEC/ESP tunnel --- [RB3011 with ethernet interface] ---- [siteB-client ip 192.168.10.10]

Both MK's have firmware v6.45.6 (latest). PPPOE/IPSEC line is ok, well connected and fast as expected. But when I try to download file from siteA to siteB and vice versa over http/https, I'm geetting randomly error "reset connection by peer" or just "Failed download" in Firefox or simple stuck when I try wget... In case of CIFS there are errors like "file is locked" or some generic i/o errors.. Test file for download is approx 500 MB and error is on random byte - sometimes 5%, sometimes 50% - it's really random. SiteA/B clients are Windows 10 1903. The PPPOE client and IPSEC line is in case of fail up and working - ping is ok on both sides without any problem.

I tried some magic with MSS mangle rules, but absolutely without success. Default MTU for PPPOE is 1480, tried to change to 1500, 1400, 1300, 1200... also on siteB WAN interface - no change. I have "Change TCP MSS" pppoe option set to Yes or Default - no luck..

Has anybody similar problem?

Thanks for help!
When I move IPSEC tunnel to ethernet connection (siteA - different provider) - everything works like a charm! So it looks there's any problem with pppoe..

Re: Unstable IPSEC over PPPOE interface

Posted: Mon Oct 07, 2019 11:07 am
by dmitris
What is CPU load on both sides when you copying files?

Re: Unstable IPSEC over PPPOE interface

Posted: Mon Oct 07, 2019 11:13 am
by hancik
of course on rb3011 approx 5% and rb2011 100% - line is 100Mbps and I know the rb2011 is not sufficient. But my customer needs to be sure the upgrade 2011 to 3011 or 4011 will work smoothly :) I also tried to make limitation of speed to approx 10 Mbps and then rb2011 cpu load was about 60% - but fails was the same...

Do you think the root cause is cpu load on rb2011? At this time is not easy to replace rb2011 ...

Re: Unstable IPSEC over PPPOE interface

Posted: Mon Oct 07, 2019 11:34 am
by dmitris
indeed, rb2011 don't support ipsec hw acceleration at all. When CPU usage is permanently 100% than router behaves unpredictable (internal facilities like nat, dhcp, ipsec,etc not working)
BTW what encryption are used under ipsec peer and policies.
Do you use encryption on PPPoE also? Look at PPPoE profile.

https://wiki.mikrotik.com/wiki/Manual:IP/IPsec

Re: Unstable IPSEC over PPPOE interface

Posted: Mon Oct 07, 2019 11:58 am
by hancik
ok, it can be true - but it's strange the router is working well without PPPOE - and CPU load is also 100% :)

Re: Unstable IPSEC over PPPOE interface

Posted: Mon Oct 07, 2019 2:19 pm
by dmitris
check under /tool profile, what exactly using RB CPU so heavily

Re: Unstable IPSEC over PPPOE interface

Posted: Mon Oct 07, 2019 2:39 pm
by hancik
hmmm, cpu0=100%, encrypting approx 60%, networking 20%, firewall 15%... :)

Re: Unstable IPSEC over PPPOE interface

Posted: Mon Oct 07, 2019 2:59 pm
by dmitris
What encryption and ciphers on ipsec configured ?

viewtopic.php?t=94931

Re: Unstable IPSEC over PPPOE interface

Posted: Tue Oct 08, 2019 6:49 pm
by hancik
ipsec is ESP with aes-128/sha256, with sha1 was performance a bit better, but bad stability was the same :) I also tried different AES algoritms, but results was the same (some was a bit slower)

Re: Unstable IPSEC over PPPOE interface

Posted: Mon Oct 21, 2019 1:03 pm
by hancik
Hello, I've just upgraded RB2011 to RB4011 and ... problem is absolutely the same - stuck while downloading file, CPU load is 5% on both sides...

Re: Unstable IPSEC over PPPOE interface

Posted: Mon Oct 21, 2019 6:09 pm
by akschu
I suspect you have a split routing or MTU issue as there are a lot of people using ipsec over pppoe without issue. I have one box plugged into 1000/100 pppoe internet connection and it's doing a lot of VPN work for an office of engineers.

schu