Community discussions

 
User avatar
ViREnG
Member Candidate
Member Candidate
Topic Author
Posts: 217
Joined: Tue Aug 11, 2009 4:01 pm
Location: Yekaterinburg

❗️❓ UNSTABLE VPLS on Wireless networks

Mon Jun 10, 2019 5:51 pm

Hi guys
We've deployed our WISP with a central PPPOE via VPLS tunnels .
VPLS works well and we have full 1500 bytes trough it , each tunnel have one PPPOE server .

But the problem is VPLS tunnels disconnect for 60 seconds (minimum) for any packet lost on remote sites, so all of that site's pppoe users will disconnect.
The long distance remotes sites connected via Point to point wireless got packet lost 10 times in 24 hour and each time VPLS tunnels will down for long time .

If I change the VPLS to EOIP , pppoe users connected trough it just got some packet lost (very stable than VPLS) , but VPLS have a better performance and lower CPU usage.

Is there anyway to reduce the down time or prevent from disconnecting ? (other than fix the long distance wireless links :)

Example diagram :
Diagram.png
You do not have the required permissions to view the files attached to this post.
 
flynno
Member Candidate
Member Candidate
Posts: 240
Joined: Wed Aug 27, 2014 8:11 pm

Re: ❗️❓ UNSTABLE VPLS on Wireless networks

Mon Jun 10, 2019 6:11 pm

Why don't you put routers at the towers and terminate the pppoe connections on the tower that the clients are connected to?
Are the VPLS tunnels back to the core with one PPPoE server and all your clients are terminating at the core?
 
User avatar
ViREnG
Member Candidate
Member Candidate
Topic Author
Posts: 217
Joined: Tue Aug 11, 2009 4:01 pm
Location: Yekaterinburg

Re: ❗️❓ UNSTABLE VPLS on Wireless networks

Mon Jun 10, 2019 7:02 pm

Why don't you put routers at the towers and terminate the pppoe connections on the tower that the clients are connected to?
Saving Public IP's , limited public IP resources and NAT is prohibited.
 
Are the VPLS tunnels back to the core with one PPPoE server and all your clients are terminating at the core?
Each Site connect with one PPPoE server that come from Central CCR via a VPLS tunnel, YES! all users of that site will terminate when VPLS disconnect.
 
mducharme
Trainer
Trainer
Posts: 795
Joined: Tue Jul 19, 2016 6:45 pm

Re: ❗️❓ UNSTABLE VPLS on Wireless networks

Tue Jun 11, 2019 5:02 pm

We run long distance wireless links and do not experience this.

Even if VPLS tunnels go down for some reason, they typically only take 10 seconds to come back, not 60. The 60 seconds makes me suspect that it is actually something else that is going down, most likely something like OSPF, or possibly LDP neighbor. You should be prioritizing OSPF and LDP over the wireless link to avoid other traffic from interfering with your routing protocols.
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 5921
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: ❗️❓ UNSTABLE VPLS on Wireless networks

Tue Jun 11, 2019 5:05 pm

Agree, check OSPF stability over particular wireless link.
 
User avatar
CZFan
Forum Guru
Forum Guru
Posts: 1363
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa, Randburg
Contact:

Re: ❗️❓ UNSTABLE VPLS on Wireless networks

Tue Jun 11, 2019 6:35 pm

One bit of info missing, does the wireless link go down for 60 seconds before connects again?

Reason I am asking is we have a wireless link, 2 x LHG 5ac's, short distance (about 700 meters) but sometimes takes up to 3 minutes to connect again if the wireless link dropped and reason seems to be taking long to scan through frequencies to connect again.
MTCNA, MTCTCE, MTCRE & MTCINE
 
User avatar
ViREnG
Member Candidate
Member Candidate
Topic Author
Posts: 217
Joined: Tue Aug 11, 2009 4:01 pm
Location: Yekaterinburg

Re: ❗️❓ UNSTABLE VPLS on Wireless networks

Tue Jun 11, 2019 7:38 pm

Even if VPLS tunnels go down for some reason, they typically only take 10 seconds to come back, not 60. The 60 seconds makes me suspect that it is actually something else that is going down, most likely something like OSPF, or possibly LDP neighbor. You should be prioritizing OSPF and LDP over the wireless link to avoid other traffic from interfering with your routing protocols.
Thanks ,Yes, you are right. It seems from OSPF because of VPLS connect exactly when OSPF reconnect , and if I disable/enable a VPLS it will connect about 5 sec.
I don't find any good document about it, if is possible get some help that how can I configure it.
I don't have any QOS or priority configuration on routers / links.

NOTE : if I disable/enable a wireless links , the CPE will connect in 10 seconds and packet transmit trough it but the OSPF will reconnect/establish in same 60 seconds. it seems broadcast it in a fixed 60 seconds.
 
mducharme
Trainer
Trainer
Posts: 795
Joined: Tue Jul 19, 2016 6:45 pm

Re: ❗️❓ UNSTABLE VPLS on Wireless networks

Tue Jun 11, 2019 9:33 pm

I don't find any good document about it, if is possible get some help that how can I configure it.
I don't have any QOS or priority configuration on routers / links.
What radios are you using?
 
scampbell
Trainer
Trainer
Posts: 446
Joined: Thu Jun 22, 2006 5:20 am
Location: Wellington, NZ
Contact:

Re: ❗️❓ UNSTABLE VPLS on Wireless networks

Wed Jun 12, 2019 3:41 am

Even if VPLS tunnels go down for some reason, they typically only take 10 seconds to come back, not 60. The 60 seconds makes me suspect that it is actually something else that is going down, most likely something like OSPF, or possibly LDP neighbor. You should be prioritizing OSPF and LDP over the wireless link to avoid other traffic from interfering with your routing protocols.
Thanks ,Yes, you are right. It seems from OSPF because of VPLS connect exactly when OSPF reconnect , and if I disable/enable a VPLS it will connect about 5 sec.
I don't find any good document about it, if is possible get some help that how can I configure it.
I don't have any QOS or priority configuration on routers / links.

NOTE : if I disable/enable a wireless links , the CPE will connect in 10 seconds and packet transmit trough it but the OSPF will reconnect/establish in same 60 seconds. it seems broadcast it in a fixed 60 seconds.
I have several networks like this I have set up and work reliably so it should work :-)

Have you managed to find out why you are getting wireless disconnects - this is the first thing to fix.

next question - are you routing all the way through your network or are the radios bridged ? If OSPF breaks it can take a while to renegotiate and select DR, BDR etc.

If using routed links with /30 addresses then be sure to select OSPF network-type=point-to-point on the relevant interfaces so no DR selection is required.

Are you using Mikrotik radios and routers or a mix of technologies ? If Mikroitk set multicast-helper=full on the wlan interfaces - turns multicast OSPF into unicast and is much more stable.

If you are bridging your wireless links then be sure to enable BFD under OSPF so you get fast discovery of a failure and restore.
e.g Router A (BFD enabled)----Bridge Radio1------Bridge Radio2--- (BFD Enabled|) Router B

In the above example Router A & B interfaces connected to the Radios would also be set to network-type=point-to-point if the radios do not support ospf. In that scenario just set a gateway on the radios that points back towards the router nearest your management network.
MTCNA, MTCWE, MTCRE, MTCTCE, MTCSE, MTCINE, Trainer
___________________
Mikrotik Distributor - New Zealand
http://www.campbell.co.nz
 
mducharme
Trainer
Trainer
Posts: 795
Joined: Tue Jul 19, 2016 6:45 pm

Re: ❗️❓ UNSTABLE VPLS on Wireless networks

Wed Jun 12, 2019 7:02 am

Decreasing downtime is good so certainly follow that feedback but also prioritize OSPF to prevent the downtime from happening in the first place. We have very reliable OSPF over wireless, in some places even set to broadcast (although as scampbell said point-to-point is better), that are completely stable and highly available because we prioritize the OSPF traffic.
 
User avatar
ViREnG
Member Candidate
Member Candidate
Topic Author
Posts: 217
Joined: Tue Aug 11, 2009 4:01 pm
Location: Yekaterinburg

Re: ❗️❓ UNSTABLE VPLS on Wireless networks

Wed Jun 12, 2019 8:42 am

@scampbell : Thanks , I'll try to to do this tips and let you know.
@mducharme : Thanks , I learned to do with below configs, if it needs more please explain 🙏🙏 .

Have you managed to find out why you are getting wireless disconnects - this is the first thing to fix.
We have packet lost or disconnect always in humidity days in coolest time of night (in summer always).
 
next question - are you routing all the way through your network or are the radios bridged ? If OSPF breaks it can take a while to renegotiate and select DR, BDR etc.
All of network is routed on each site and just wireless links are in P2P mode / Bridge.

If using routed links with /30 addresses then be sure to select OSPF network-type=point-to-point on the relevant interfaces so no DR selection is required.
each site have a /29 addresses for self or for next site , for example :
Router [IP x.x.x.1/29] ===> AP device (bridged) [IP x.x.x.2/29] ===> CPE device (bridged) [IP x.x.x.3/29] ===> Router [IP x.x.x.4/29] .
For this type of connection I still need to set " point to point " as network type in OSPF ?

Are you using Mikrotik radios and routers or a mix of technologies ? If Mikroitk set multicast-helper=full on the wlan interfaces - turns multicast OSPF into unicast and is much more stable.
All of Routers and Wireless links are Mikrotik.


I comment on simple for any future readers like me (Correct me if I'm wrong).
1. OSPF interface [network type] from broadcast change to point to point , do this in Routers .
2. OSPF interface , enable Use BFD
3. Wlan interface , [Multicast Helper] from default to full.
 
mducharme
Trainer
Trainer
Posts: 795
Joined: Tue Jul 19, 2016 6:45 pm

Re: ❗️❓ UNSTABLE VPLS on Wireless networks

Wed Jun 12, 2019 8:45 am

If you have MikroTik radios, use "set priority" action to 7 for OSPF traffic. If the radio is routing you can do that with a mangle rule, or if the radio is bridging you will need to do it with a bridge filter. That will prioritize it as long as you are using either NV2 or WMM. If you are using NV2 change to the 8 queue option instead of the default 2 queues.

Also, some people have had problems with BFD making OSPF less stable so be careful, see viewtopic.php?t=93775 for details.
 
scampbell
Trainer
Trainer
Posts: 446
Joined: Thu Jun 22, 2006 5:20 am
Location: Wellington, NZ
Contact:

Re: ❗️❓ UNSTABLE VPLS on Wireless networks

Thu Jun 13, 2019 2:31 am

I comment on simple for any future readers like me (Correct me if I'm wrong).
1. OSPF interface [network type] from broadcast change to point to point , do this in Routers .
2. OSPF interface , enable Use BFD
3. Wlan interface , [Multicast Helper] from default to full.
Hi ViREnG,

Network type=point-to-point is only used where there are just two OSPF devices involved in a subnet. Easiest example is where two Mikrotik's connect wlan1-wlan1 and have a /30. If more than two devices running ospf the NBMA might be a better choice for your wireless. see https://wiki.mikrotik.com/wiki/Manual:O ... A_networks

If you have a /29 and a Mikrotik at each endpoint and a pair of bridged non-ospf wireless devices in the middle then I would also use P2P rather than broadcast on that segment. This is also an example of where BFD should be used. If the wireless link breaks then the interfaces on the Mikrotik routers will not be immediately aware and wait unitl the timers expire before realising the link is down.

If you are Mikrotik Wireless to Mikrotik Wireless and each wlan1 is using ospf then you dont need BFD as OSPF would be aware instantly wlan1 failed.

I always use the multicast-helper=full on wireless interfaces running OSPF. No negative impact, only stability in my experience.

You mention wireless disconnects due to humidity etc - perhaps you could share a bit more info on the setup there - frequency, distance, devices etc and what is the TX/RX Signal and CCQ when working (check your registration table) ? This is most likely the cause of the issue while all the other thoughts will only help once the link re-establishes....... Be sure to check CCQ while there is live traffic passing :-)

I also agree with mducharme regarding QoS - here is a great presentation on the subject - https://mum.mikrotik.com/presentations/US13/lutz.pdf
MTCNA, MTCWE, MTCRE, MTCTCE, MTCSE, MTCINE, Trainer
___________________
Mikrotik Distributor - New Zealand
http://www.campbell.co.nz
 
millenium7
Member Candidate
Member Candidate
Posts: 195
Joined: Wed Mar 16, 2016 6:12 am

Re: ❗️❓ UNSTABLE VPLS on Wireless networks

Tue Jul 02, 2019 2:38 am

Just regarding NBMA OSPF type. I would advise against it. I've had issues with NBMA being unreliable for no apparent reason
I've found the best option on MikroTik if you have a point-to-multipoint setup is to instead create separate VLAN interfaces (1 for each neighboring router) and then use the point-to-point network type over those VLAN interfaces instead. Much more reliable

Who is online

Users browsing this forum: No registered users and 5 guests