Hello,
new on the forum, I post my first message.
my configuration
RB4011 router release 6.49.7
I work for an ISP
we have 1Gb sysmetric fiber.
The delivery of our links are done in PPPoE
When I do throughput tests on my link, I don’t go beyond 800Mb, I noticed that the cpu0 was at 100% (see attached), however the other 3 are not working.
I passed the delivery of the link in IP on the interface and I don’t have the same problem, we can see the CPUs that share their load (see attached).
I have the impression that multi-threading is not managed when we switch to PPPoE.
I think there is a problem at the system level, because I have the same problem on the 6.47.x and 6.48.x releases
How to proceed to solve this problem which is very impacting?
To carry 1Gbps of useful data, you need more than 1Gbps …
If you have 1Gbps maximum, you can only receive at lower speeds, because something is used to transport the useful data, on useful way.
1Gbps is only the link speed…
Assuming (without be excessively precise, and just for example) exactly 1500 bytes for each packet, 40Bytes go away for transport, and remaining only 1460.
On this way you need two packet for transport 1500Bytes (1460+40), ad is needed on total 80Bytes more (40+40).
At this point, for transport 1Gbps (125MBps) you need 125000000 / 1460 packet = ~85617 packets and 85617 packets need 40Bytes for each packet for transport,
and you have 3424680Bytes more to be transmitted, another 28Mbits.
So, for this approximative example, to transmit 1Gbits you actually need 1,028Gbits, but if your internet speed is 1Gbps and the max data speed can be only lower.
Introducing pppoe, regardless of whether it is multicore or not, reduce the maximum size of each packet by another 20Bytes, so the link can only go slower.
Thanks for your feedback.
I understand what you mean, but I have 1Gb to transport and I have difficulty exceeding 750Mb in PPPoE, if we listen to what you say I will have to approach 950 Mb.
Moreover, we can see that the CPU of one of the 4 cores is at 100%, while the mikrotik should distribute the load.
My example is approximative for not be one network lesson, on 1Gbps “1526 standard - I.P. - pppoe - pppoe encryption” link
I do not expect more than 825Mbps on any brand or ISP.
Your observation is very correct, PPPoE activities are done on 1 CPU-core. Always have, and probably always will. (not only on RouterOS but eg. OpenWRT, BSD too etc)
So you would benefit from a very high-clock rate to further increase this.
On these more 1Gbps (and up) links … move away from PPPoE and turn to DHCP-82 with an ISP that supports it.
Apart from that, when aiming for these speeds make sure you try to use “fast-path” as much as possible. There are several “dependencies” that might break fastpath and have much more CPU involved in the traffic-processing. This will severly limit throughput.
Thanks for understanding my problem better.
so, mikrotik will never fix this problem?
because it’s very blocking for us, all our ISP propose PPP and not DHCP.
I’m pretty sure several other products with the hardwar-design (= generic CPU/same clockspeeds, no ASIC’s) face the same with PPPoE (client) setups. This is not limited to Mikrotik only.
Your test of using PPPoE on a 10Gbits/sec link is really not a good “realworld example”. Nobody in the world will use PPPoE on such links…
Not sure if you could try to create another PPPoE connection-profile to the same ISP and I wonder if that would be bound on another cpu-core.
Once you have 2 PPPoE sessions (with 2 public IP’s) you might try to stuff to use both “links”.
Thanks for your complete feedback.
I don’t know if other brands of routers have the same problems.
For you it’s a normal problem that can’t be fixed, or on v7 it can be?
Yes I could mount two PPPoE sessions on the same link, but in the end it will be hard to distribute the load between the 2. Moreover I could not manage the QOS correctly.
It looks like some of the modern SoC (CPUs) used in routers do have hardware supported PPPoE, but MikroTik is not making use of it. Some other software might use it.
If that were used the handling of PPPoE (probably only the data packets) would not load the CPU and there is no “is it using one or more cores” issue either.
In general, it is not as easy as you may think to distribute load over multiple cores. For example, software does not expect the order of packets to be changed (although it really should, as IP does not guarantee the order of packets!). So when you distribute the incoming PPPoE traffic over multiple cores, it may happen that packets get routed out of sequence and some applications get upset by that.
You will generally find that setting up a single connection and trying to max out the traffic (e.g. a single TCP connection from a single client on the local network) will max-out a single CPU core. That is often found during benchmark testing. But in real use, especially with a larger number of clients (like in a business setting) you may well find that load is better distributed over the cores. But not for the PPPoE itself, just for other functions like firewall, queues, etc.
With your 4011 you might be hitting a limit yes in a PPPoE-client setup if you want to exceed 1Gbps…
It’s not exact science and various variables influence the result.
My ISP here for example, for home (fiber) “GPON” subscriptions, PPPoE is not used anymore … using DHCP and ethernet-frames are coming in tagged.
I will have to validate another product for our 1Gb links, so as not to lose too much on the margin.
I think the RB1100AH should be able to do the job.
Yes I think the future will be based on DHCP.
but all our french ISPs offer 95% PPPoE
