Community discussions

 
ntmanxp
just joined
Topic Author
Posts: 21
Joined: Fri Feb 15, 2008 9:04 pm

Interface PPP Server side flaps/disconnects all

Mon Sep 18, 2017 6:06 pm

Hello there.

I am facing this problem since january, and even tech support cant give me a solution.
I have 6 CCRs 12G 4S acting as PPP servers.
Each PPP server listening on vlans, working in pairs ( PPP1 and PPP2 with vlans 102 and 103, PPP3 and PPP4 with vlans 101,201 and 251, and PPP5 and 6 with vlans 221, 231, 22 and 252 )
These way, there are almost 1200/1300 users per CCR at a time, except when one goes down, where the other takes control until the reboot or recover.
Some firewalls rules to protect itself and customers ( like TR069 protocol block from outside ).
The problem is that -apparently- with traffic higher than 500 mbps, at some point the interface PPP side goes down, and up again. Not the WAN side.
This of course disconnects all users.
The ppp bandwidth control is with simple queues, most users dynamic IP, some others fixed public IP.
With versions like 6.29.1, the behaviour was suddenly unkown queues, IP address hooked by the ip pool, so must to reboot to release all and goes back to normal operation.
With 6.39.x to 6.40.2, latest, as told before, -same rules, same all- the interface flaps.
Tried disabling IPv6, no luck.
Disabled ppp compression, disabled ip neighbours, but nothings seems to help.
Seems crazy, no autosupout, nor log info, nothing.
Is like if someone unplugg/plug the PPP cable!!!
If someone can explain what to do, I'll be grateful.

Regards
Andres.-
 
User avatar
pukkita
Trainer
Trainer
Posts: 2984
Joined: Wed Dec 04, 2013 11:09 am
Location: Spain

Re: Interface PPP Server side flaps/disconnects all

Tue Sep 19, 2017 11:16 am

Which interface flaps? the VLANs? What's the CPU load at those times?
Simplicity is the Ultimate Sophistication - Da Vinci
Getting the most out of this forum
 
ntmanxp
just joined
Topic Author
Posts: 21
Joined: Fri Feb 15, 2008 9:04 pm

Re: Interface PPP Server side flaps/disconnects all

Wed Sep 20, 2017 2:14 am

Hi,
Yes, interfaces on the PPP side.
Some PPP interfaces carry only one vlan. Some others, 2 or 3 vlans.
In those cases, the status tab of the PPP interface shows the link down time, and the link up time. About 15/20 seconds difference between them.
3 CCRs are in one switch, using MikroTik cables for sfp. This ones, has 2 interfaces for the PPP side. And just one for the wan side. Seems that the most loaded PPP interface, flaps. The other does not.
The other 3 CCRs are plugged to another switch, using in this case miniGbics and fiber cable. In this case, just one interface for PPP, one for wan. But same behaviour, the links goes down and up.
Strange, ah? And besides, the switches says that the link goes down/up in just a second. Not 20 secs as the MKT says.
The CPU is between 10/15%. But once the interface flaps, it goes to 90/100% ( must take care of radius disconnects and so on )
But I do have not less than 2 flaps per day. So customers are not happy.
Really do not know how to fix it.
Regards

Andres
 
User avatar
pukkita
Trainer
Trainer
Posts: 2984
Joined: Wed Dec 04, 2013 11:09 am
Location: Spain

Re: Interface PPP Server side flaps/disconnects all

Wed Sep 20, 2017 12:12 pm

I'd say you have two problems; a L2, root one that causes the pppoe sessions to disconnect, you need to find the cause. Interface flapping on PPPoE Uplinks are not admisible on production scenarios, check wiring/fiber, SFP modules, Switches, etc.

Are you using OSPF? The second one (high CPU load) seems to point to either misconfigured OSPF, that tries to form adjacencies with pppoe interfaces, or SRCNAT being applied to pppoe interfaces; that would explain the high CPU load at flapping time.

Have a look at Janis Megis' Most Underused and Overused RouterOS Features Presentation.

Are the CCRs up to date with regards firmware (see System > Routerboard)?
Simplicity is the Ultimate Sophistication - Da Vinci
Getting the most out of this forum
 
ntmanxp
just joined
Topic Author
Posts: 21
Joined: Fri Feb 15, 2008 9:04 pm

Re: Interface PPP Server side flaps/disconnects all

Wed Sep 20, 2017 4:07 pm

Hello
Yeah, an L2 problem is there, almost for sure.
Cables were 1st option. But so far, all tests says cables are OK.
Besides, it replies to mikrotik cables, to miniGBICs modules, using different SFP port in the CCR, an different port in the switch.
The OSPF is not used.
If you disconnect massive users in pppoe, then CCR send notification to Radius.
All fw/sw are up to date.
 
User avatar
Melody5781
newbie
Posts: 29
Joined: Thu Sep 14, 2017 12:42 pm

Re: Interface PPP Server side flaps/disconnects all

Thu Sep 21, 2017 11:39 am

I have a very similar problem like this and it haunted me for months. Turns out that there is something wrong with the cables.
 
ntmanxp
just joined
Topic Author
Posts: 21
Joined: Fri Feb 15, 2008 9:04 pm

Re: Interface PPP Server side flaps/disconnects all

Sat Dec 30, 2017 3:06 pm

Well, after some testing, we unplugged ALL SFP optical links from the CCRs in the PPP side.
And replace the PPP server side to Electrical Interfaces.
Only that, no other changes, And got 2 months in a raw without any issue.
The 'suspected' ports in the switch, cables, sfp, etc. are now serving the WAN side of 3 new CCRs.
Without a flap.

So, I can say now that the PPP interface in a Optical intreface, do flap.
NO idea so far why. But it's happening.
Recipe: Use SFPs in WAN side. ANd electrical in PPP side.

Besides of the high CPU use once the PPP sessions disconnects ( viewtopic.php?t=117958 ) no other issue seem so far.

Regards

Andres

Who is online

Users browsing this forum: No registered users and 108 guests