MikroTik

Our network is, in short, like this:
(AP & EOIP) -> Link to NOC -> (EOIP & PPPOE Server)
So we have our AP bridged over EOIP to one router with many EOIP tunnels on one bridge. On that bridge is one pppoe server.
Server was i5 and later on i7, with 2 Broadcom PCIe Gigabit cards.
Server is tested with almost million of packets and without any problem before we put MT L6 on it.

The problem we have is that pings starting to get higher even with localhost when there are over 1000 connections on i5 and 1500 connections with i7 (same motherboard, different CPU). Server CPU usage in case of i5 was 40% on 1000 connections and 35% with 1500 connections on i7.
The profiling show nothing usual, most used was firewall and queue. We turned off snmp to prevent high cpu usages.
We also tried same setup with RB1000AH and the limit is a bit over 300 pppoe connections but the cpu was near 100%.
We are using RADIUS auth and dynamic simple queue using MT radius attribute.

My question would be:
1. Which amount of PPPOE connections can be expected to run on one machine and what is the spec of that machine?
2. Is there some bug that prevents us to reach higher number of pppoe connections without lag while cpu is idling at 60%?
3. Is it possible that CPU usages shows incorrect result?

And, yes, i contacted support, but did not get useful answer.

Hi

What are the services running on the PC @i5 and i7

Lets start with that

1) Avoid EoIP whenever possible!
It affects CPU and Bandwidth badly.
Even if the CPU-usage is low, there are weird issues with throughput.

2) Replace EoIP by VLANs and/or MPLS-VPLS

3) If you are forced to use EoIP,
try to concentrate EoIP one one machine, while terminating PPPoE one the other machine.

4) 300 PPPoE connections with subscriber rates between 3 and 30 Mbps are doable with a RB1100AH, if PPPoE is the only job it has to do. (Filtering an Tunnel concentration will do severe harm to your perceived performance)

Replacing EOIP with something else is a problem becouse it's a big network and there are often changes on it by people not so technically skilled.
Besides, EOIP contrary to most beliefs, uses very little CPU, around 0.5% on 200mbit, or the Profiler is lying.
I tought to get all EOIP tunnels on one machine bridged with network interface, put that interface on separate switch and have many pppoe concentrators there. This way we could do load-balancing + failover on pppoe links. Still, i really doubt that will fix our problem (and i hope that other one won't appear...).
There is 300+ PPPOE connections with 1.5mbits in average (rough estimate).
On i7 (replaced i5, it's same machine) there is a bridge on which pppoe server works, and eoip tunnel ends. In firewall there is one rule to change mss (optimized, instead of having 3000 rules, each for one pppoe connection). There is some few basic rules on input to prevent unath access and that's all. No NAT, no MANGLE (except change mss). Connection tracking is on.

Is there any MT based PPPoE concentrator that runs 1500+ on one machine?
Alot of people say that 2.9.51 runs fine as PPPoE concentrator and i'm wondering why? Becouse of older kernel?
That old version would not work on most new hardware (it needs IDE instead of SATA, doesn't support PCIe and Multi-CPU and other stuff...)

actualy this is very similar problem like in http://forum.mikrotik.com/viewtopic.php?f=2&t=29321 , but thats post is from 2009 and nothing change from that time.
we try large number of combination and optimisation found in this forum and nothing give as satisfactory result
we dont wana throw problem on mikrotik, we just wont some sugestion about possible amount of pppoe session per several x86 server combination.
its very hard, experiment on live subscribers, with hardware cost several thousands dollar.
and please we tested concentrator on up to 700 pppoe connection for several months witout any issue, so dont tell me you have 300 hundred or something connections and everything is ok, question is on whitch modern x86 server based expencive hardware is possible several thousand connections, 1500,2000+ ????

Test any machine with:

- only one GigabitEtherent interface on PCI-e bus connected to a Switch with VLANs
- turned off multi-threading in the BIOS and turned off multi-cpu in RouterOS

do not use Simple Queues and dynamic stuff

Report back.

Before the router went into production it was troughly tested and there was no problem with bandwidth or latency.
Bandwidth was full 1Gbit and latency was below 20, so this problem must be related either to many queues or many pppoe connections.

Anyway, it's solved by replacing mikrotik with plain linux+accel-ppp.

What PPP authentication is used ?

Can you show the configuration of the MT that was "replaced" ?

- Lag problem from first post I think is due to Queues. -> How do you limit the users on the new Linux solution?

Dragonmen

I'm reading about RADIUS, see it's psoible create queues for each connection to the attributes that are passed, but there is some way to create a firewall rule for attribute too? It is important to give debtors notices to customers. -

Also you consult, you tried roureros version 6, is that it is pre-release, but to see if it corrects these problems. -

there is some way to create a firewall rule for attribute too? It is important to give debtors notices to customers. -

you may use 'Address-List' attribute to add customer's address to that address list. then create firewall rules to work with those customers