Community discussions

 
dr4g0nsr
just joined
Topic Author
Posts: 3
Joined: Tue Jan 24, 2012 4:03 pm

PPPOE Server with many connections lags

Thu Sep 06, 2012 4:52 pm

Our network is, in short, like this:
(AP & EOIP) -> Link to NOC -> (EOIP & PPPOE Server)
So we have our AP bridged over EOIP to one router with many EOIP tunnels on one bridge. On that bridge is one pppoe server.
Server was i5 and later on i7, with 2 Broadcom PCIe Gigabit cards.
Server is tested with almost million of packets and without any problem before we put MT L6 on it.

The problem we have is that pings starting to get higher even with localhost when there are over 1000 connections on i5 and 1500 connections with i7 (same motherboard, different CPU). Server CPU usage in case of i5 was 40% on 1000 connections and 35% with 1500 connections on i7.
The profiling show nothing usual, most used was firewall and queue. We turned off snmp to prevent high cpu usages.
We also tried same setup with RB1000AH and the limit is a bit over 300 pppoe connections but the cpu was near 100%.
We are using RADIUS auth and dynamic simple queue using MT radius attribute.

My question would be:
1. Which amount of PPPOE connections can be expected to run on one machine and what is the spec of that machine?
2. Is there some bug that prevents us to reach higher number of pppoe connections without lag while cpu is idling at 60%?
3. Is it possible that CPU usages shows incorrect result?

And, yes, i contacted support, but did not get useful answer.
 
zizobaddy
Member Candidate
Member Candidate
Posts: 107
Joined: Mon Sep 13, 2010 10:13 am

Re: PPPOE Server with many connections lags

Thu Sep 06, 2012 5:11 pm

Hi

What are the services running on the PC @i5 and i7

Lets start with that
 
sup5
Member
Member
Posts: 322
Joined: Sat Jul 10, 2010 12:37 am

Re: PPPOE Server with many connections lags

Thu Sep 06, 2012 6:13 pm

1) Avoid EoIP whenever possible!
It affects CPU and Bandwidth badly.
Even if the CPU-usage is low, there are weird issues with throughput.

2) Replace EoIP by VLANs and/or MPLS-VPLS

3) If you are forced to use EoIP,
try to concentrate EoIP one one machine, while terminating PPPoE one the other machine.

4) 300 PPPoE connections with subscriber rates between 3 and 30 Mbps are doable with a RB1100AH, if PPPoE is the only job it has to do. (Filtering an Tunnel concentration will do severe harm to your perceived performance)
 
dr4g0nsr
just joined
Topic Author
Posts: 3
Joined: Tue Jan 24, 2012 4:03 pm

Re: PPPOE Server with many connections lags

Thu Sep 06, 2012 6:56 pm

Replacing EOIP with something else is a problem becouse it's a big network and there are often changes on it by people not so technically skilled.
Besides, EOIP contrary to most beliefs, uses very little CPU, around 0.5% on 200mbit, or the Profiler is lying.
I tought to get all EOIP tunnels on one machine bridged with network interface, put that interface on separate switch and have many pppoe concentrators there. This way we could do load-balancing + failover on pppoe links. Still, i really doubt that will fix our problem (and i hope that other one won't appear...).
There is 300+ PPPOE connections with 1.5mbits in average (rough estimate).
On i7 (replaced i5, it's same machine) there is a bridge on which pppoe server works, and eoip tunnel ends. In firewall there is one rule to change mss (optimized, instead of having 3000 rules, each for one pppoe connection). There is some few basic rules on input to prevent unath access and that's all. No NAT, no MANGLE (except change mss). Connection tracking is on.
 
dr4g0nsr
just joined
Topic Author
Posts: 3
Joined: Tue Jan 24, 2012 4:03 pm

Re: PPPOE Server with many connections lags

Fri Sep 07, 2012 1:19 pm

Is there any MT based PPPoE concentrator that runs 1500+ on one machine?
Alot of people say that 2.9.51 runs fine as PPPoE concentrator and i'm wondering why? Becouse of older kernel?
That old version would not work on most new hardware (it needs IDE instead of SATA, doesn't support PCIe and Multi-CPU and other stuff...)
 
vlatko
newbie
Posts: 48
Joined: Sat Feb 12, 2005 11:28 am

Re: PPPOE Server with many connections lags

Sun Sep 09, 2012 11:15 pm

actualy this is very similar problem like in http://forum.mikrotik.com/viewtopic.php?f=2&t=29321 , but thats post is from 2009 and nothing change from that time.
we try large number of combination and optimisation found in this forum and nothing give as satisfactory result
we dont wana throw problem on mikrotik, we just wont some sugestion about possible amount of pppoe session per several x86 server combination.
its very hard, experiment on live subscribers, with hardware cost several thousands dollar.
and please we tested concentrator on up to 700 pppoe connection for several months witout any issue, so dont tell me you have 300 hundred or something connections and everything is ok, question is on whitch modern x86 server based expencive hardware is possible several thousand connections, 1500,2000+ ????
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1369
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: PPPOE Server with many connections lags

Sun Oct 07, 2012 8:21 pm

Test any machine with:

- only one GigabitEtherent interface on PCI-e bus connected to a Switch with VLANs
- turned off multi-threading in the BIOS and turned off multi-cpu in RouterOS

do not use Simple Queues and dynamic stuff

Report back.
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
User avatar
Dragonmen
Frequent Visitor
Frequent Visitor
Posts: 72
Joined: Thu Jun 16, 2005 6:20 pm
Location: Sabac, Serbia
Contact:

Re: PPPOE Server with many connections lags

Tue Nov 13, 2012 11:23 pm

Before the router went into production it was troughly tested and there was no problem with bandwidth or latency.
Bandwidth was full 1Gbit and latency was below 20, so this problem must be related either to many queues or many pppoe connections.

Anyway, it's solved by replacing mikrotik with plain linux+accel-ppp.
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1369
Joined: Mon Jan 05, 2009 6:23 pm
Location: Worldwide
Contact:

Re: PPPOE Server with many connections lags

Wed Nov 14, 2012 8:48 am

What PPP authentication is used ?

Can you show the configuration of the MT that was "replaced" ?

- Lag problem from first post I think is due to Queues. -> How do you limit the users on the new Linux solution?
wiki.mikrotik.com/wiki/NetworkPro_on_Quality_of_Service
 
marceloru
newbie
Posts: 47
Joined: Tue Jul 08, 2008 12:00 am
Location: Argentina

Re: PPPOE Server with many connections lags

Sun Feb 17, 2013 12:06 am

Dragonmen

I'm reading about RADIUS, see it's psoible create queues for each connection to the attributes that are passed, but there is some way to create a firewall rule for attribute too? It is important to give debtors notices to customers. -

Also you consult, you tried roureros version 6, is that it is pre-release, but to see if it corrects these problems. -
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8319
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: PPPOE Server with many connections lags

Mon Feb 18, 2013 4:32 pm

there is some way to create a firewall rule for attribute too? It is important to give debtors notices to customers. -
you may use 'Address-List' attribute to add customer's address to that address list. then create firewall rules to work with those customers
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.

Who is online

Users browsing this forum: No registered users and 67 guests