Community discussions

MUM Europe 2020
 
rastod
Member Candidate
Member Candidate
Topic Author
Posts: 122
Joined: Sat Jun 04, 2005 11:35 pm
Location: Slovakia

PPPoE server performance and packet lose problems

Wed Mar 19, 2008 5:23 pm

Hi all,

we have big problems with performance of PPPoE server running 3.x versions. Today we run 3.4 version and still having problem. The problem is that we have packetlose on localhost (127.0.0.1) and all interfaces. It seems like it is a performance problem or problem of interface queues.

We run PPPoE server with quad core cpu xeon 2400, with board chipset intel 3000 server chipset with two gbit onboard ethernet cards. Traffic is up to 40/10mbit download/upload.

Enclosed is screen copy of our pppoe server. You can see packet lose to address 127.0.0.1 and high ping time response. Can someone observe the same problem?

We have this problem on all PPPoE server under heavy load.
You do not have the required permissions to view the files attached to this post.
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Wed Mar 19, 2008 5:59 pm

I have this same problem here, I changed the machine and it remains to loose about 4% on all interfaces. Even with low CPU usage (20%) I have packet loss on all interfaces (even local).

I already tried to disable conntrack and TCPMSS to reduce processing but it didn´t solve the problem.

In fact I also saw this happening in 2.9.50 version, but not on early versions.

This is a huge problem that I cannot find a solution.
 
rastod
Member Candidate
Member Candidate
Topic Author
Posts: 122
Joined: Sat Jun 04, 2005 11:35 pm
Location: Slovakia

Re: PPPoE server performance and packet lose problems

Wed Mar 19, 2008 8:03 pm

At least, I am not alone.

And what are you doing now? Downgrade back to 2.9.xx?
 
rastod
Member Candidate
Member Candidate
Topic Author
Posts: 122
Joined: Sat Jun 04, 2005 11:35 pm
Location: Slovakia

Re: PPPoE server performance and packet lose problems

Wed Mar 19, 2008 8:24 pm

I have new observation.

When I switched off "Change TCP MSS" on all used profiles, so there were no dynamic iptable's Mangle rules, the problem diminish. It is still here, but not so huge.
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Wed Mar 19, 2008 10:39 pm

I only have one router using version 3.4 (the other ones still use old 2.9 versions).

For me disabling Change TCPMSS doesn´t solve the problem, I made this test today and even with this option disabled I continue to have packetloss (about 2%). Probably with TCPMSS off the packetloss is lower but it still not acceptable.

And I´m not talking about too much connections. With more than 100 pppoe connections it´s already possible to see packetloss on a Pentium D 3.4Ghz (with multicore enabled or not).

We have machines with a lot worst hardware than that using old 2.9 version and more than 500 users with 0% of packeloss..
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Thu Mar 20, 2008 2:52 pm

I think the best way is report it to Mikrotik support, and pray for a solution as soon as possible.

If everybody who has this same problem report it to Mikrotik support they will pay more attention to it.
 
rastod
Member Candidate
Member Candidate
Topic Author
Posts: 122
Joined: Sat Jun 04, 2005 11:35 pm
Location: Slovakia

Re: PPPoE server performance and packet lose problems

Thu Mar 20, 2008 3:57 pm

I did, but still no answer. I wanted to check if there are others that meet the problem.
 
rkorolev
newbie
Posts: 43
Joined: Tue Oct 23, 2007 1:49 pm

Re: PPPoE server performance and packet lose problems

Thu Mar 20, 2008 6:20 pm

Do you use queues?
We have the same problem with packet loss and one of the problems is that ROS drops a packet every time a queue is created or deleted.
When many users connecting/disconnecting you get a packet loss.
Maybe that's not the only reason, but thats a reproducable and Mikrotik confirmed this issue.
 
rastod
Member Candidate
Member Candidate
Topic Author
Posts: 122
Joined: Sat Jun 04, 2005 11:35 pm
Location: Slovakia

Re: PPPoE server performance and packet lose problems

Thu Mar 20, 2008 6:36 pm

Yes, I suppose that it can be problem. But in older versions it was correct. Maybe MT should consider this.
But not only packet lose, also high time response to locahost ping is problem. Maybe it is common problem.
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Thu Mar 20, 2008 9:29 pm

I use queues and I agree that it could be related to problem. But i can´t stop to use queues!

If this problem was not solved soon this will make the pppoe server resource unusable for me and everybody I know that use it.
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 5950
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: PPPoE server performance and packet lose problems

Thu Mar 20, 2008 10:16 pm

I believe MT is working on this issue.
But there is always workaround - use static queues or queue tree. There are also several benefits to use static queues:
*) few queues, instead of hundreds for each user
*) easy to manage
*) reduced CPU and memory usage.

Probably you can find more.
 
bokili
Member Candidate
Member Candidate
Posts: 135
Joined: Wed Aug 16, 2006 8:52 pm

Re: PPPoE server performance and packet lose problems

Fri Mar 21, 2008 4:40 am

When I write about this nobody responded :)

Even some of you write that we (my company) do not know to use mikrotik and other bullshits...

So:)? What now:) Who has the right about packet loss in v3.X ??? :)

The problem is really in MT. From my experience I can tell you that there is not packet loss when you use mikrotik v3.X versions for other purpose, but especially when you use it together with PPPoE, packet loss appear immediatelly.
As number of pppoe tunnels grow, packet loss also grows. As bandwith in this router grows, packet loss also grows.

Then CPU Load go high as 90-100% and than router is not accessible.
 
rkorolev
newbie
Posts: 43
Joined: Tue Oct 23, 2007 1:49 pm

Re: PPPoE server performance and packet lose problems

Fri Mar 21, 2008 7:20 am

I believe MT is working on this issue.
But there is always workaround - use static queues or queue tree. There are also several benefits to use static queues:
Probably you can find more.
Static queues are not suitable for at least one situation: when you have a number of bandwidth plans and you set the speed with "Mikrotik-Rate-Limit" radius attribute.

Maybe you'll recommend, how to set up a speed limit with predefined static queue based on radius reply?
All users have dynamic IPs, so I can't just shape "that group of ips to 128k, other ips to 256k, etc".
 
rastod
Member Candidate
Member Candidate
Topic Author
Posts: 122
Joined: Sat Jun 04, 2005 11:35 pm
Location: Slovakia

Re: PPPoE server performance and packet lose problems

Fri Mar 21, 2008 10:45 am

I believe MT is working on this issue.
But there is always workaround - use static queues or queue tree. There are also several benefits to use static queues:
*) few queues, instead of hundreds for each user
*) easy to manage
*) reduced CPU and memory usage.

Probably you can find more.
We terminate up to 700-900 users on one PPPoE server, I suppose it is not possible to deal with static queues. It would be possible only if services will be the same for all users or there will be some other limitations.
 
rastod
Member Candidate
Member Candidate
Topic Author
Posts: 122
Joined: Sat Jun 04, 2005 11:35 pm
Location: Slovakia

Re: PPPoE server performance and packet lose problems

Fri Mar 21, 2008 11:00 am

It seems to be true - I watched log and ping and it seems that every time new PPPoE session is created or closed, packet lose occures.

So, it seems there are two problems:
- packet lose when creating/closing pppoe session
- high ping response to locahost when higher utilization even CPU is about 50%, so it can be problem of some queues like interface queues
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Mon Mar 24, 2008 2:57 pm

Use static queues to solve this problem is impossible. Queues are created dynamically by radius attribute.
 
bokili
Member Candidate
Member Candidate
Posts: 135
Joined: Wed Aug 16, 2006 8:52 pm

Re: PPPoE server performance and packet lose problems

Mon Mar 24, 2008 10:07 pm

Use static queues to solve this problem is impossible. Queues are created dynamically by radius attribute.
It will not help to resolve problem. We already tried.
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Tue Mar 25, 2008 10:44 pm

If using static queues doesnt solve the problem so it´s not related with the creation process of the queue.
 
rastod
Member Candidate
Member Candidate
Topic Author
Posts: 122
Joined: Sat Jun 04, 2005 11:35 pm
Location: Slovakia

Re: PPPoE server performance and packet lose problems

Tue Mar 25, 2008 10:54 pm

I have reply from MT support, they confirmed problem and they are working on solution.
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Wed Mar 26, 2008 2:19 pm

I have reply from MT support, they confirmed problem and they are working on solution.
Let´s cross our fingers and hope they find a solution as soon as possible.
 
rastod
Member Candidate
Member Candidate
Topic Author
Posts: 122
Joined: Sat Jun 04, 2005 11:35 pm
Location: Slovakia

Re: PPPoE server performance and packet lose problems

Wed Mar 26, 2008 2:28 pm

it seems they found. I am testing 3.7 and it works much more better...
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Wed Mar 26, 2008 3:41 pm

it seems they found. I am testing 3.7 and it works much more better...
Good news!

But where did you find this version?
 
rastod
Member Candidate
Member Candidate
Topic Author
Posts: 122
Joined: Sat Jun 04, 2005 11:35 pm
Location: Slovakia

Re: PPPoE server performance and packet lose problems

Wed Mar 26, 2008 3:45 pm

MT sent me to check corrections.
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Wed Mar 26, 2008 3:52 pm

MT sent me to check corrections.
Ok, I think they will release it soon..

How are your tests? It´s not loosing packets?
 
rkorolev
newbie
Posts: 43
Joined: Tue Oct 23, 2007 1:49 pm

Re: PPPoE server performance and packet lose problems

Wed Mar 26, 2008 4:09 pm

3.7 beta works fine, as we can see for this short period of time:
Packets: Sent = 4546, Received = 4546, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 6ms, Maximum = 478ms, Average = 8ms

Before this I had 2%-5% loss on 200 packets.
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Wed Mar 26, 2008 4:22 pm

3.7 beta works fine, as we can see for this short period of time:
Packets: Sent = 4546, Received = 4546, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 6ms, Maximum = 478ms, Average = 8ms

Before this I had 2%-5% loss on 200 packets.

Excellent result!!! My packet loss is the same you had with previous versions.

I´m waiting for this version release to solve my problem.
 
bokili
Member Candidate
Member Candidate
Posts: 135
Joined: Wed Aug 16, 2006 8:52 pm

Re: PPPoE server performance and packet lose problems

Thu Mar 27, 2008 2:03 am

3.7 beta works fine, as we can see for this short period of time:
Packets: Sent = 4546, Received = 4546, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 6ms, Maximum = 478ms, Average = 8ms

Before this I had 2%-5% loss on 200 packets.
This is great news!
Why do you have so big latency?

What queue type do you use? Is this over wireless or ?
 
rkorolev
newbie
Posts: 43
Joined: Tue Oct 23, 2007 1:49 pm

Re: PPPoE server performance and packet lose problems

Thu Mar 27, 2008 7:58 am

It's a DSL shaped to 128kbit and latency is pretty good (except "maximum", but maybe it was a lag on a line).
 
rkorolev
newbie
Posts: 43
Joined: Tue Oct 23, 2007 1:49 pm

Re: PPPoE server performance and packet lose problems

Thu Mar 27, 2008 8:56 am

Pity, but after a day of uptime it began loosing packets and a minimal ping is 60ms :-/
 
rastod
Member Candidate
Member Candidate
Topic Author
Posts: 122
Joined: Sat Jun 04, 2005 11:35 pm
Location: Slovakia

Re: PPPoE server performance and packet lose problems

Thu Mar 27, 2008 12:00 pm

Yes, I can confirm this problem too. I have ping to localhost over 100ms. After restart it is ok. I've already informed MT, they are working on it.
 
rkorolev
newbie
Posts: 43
Joined: Tue Oct 23, 2007 1:49 pm

Re: PPPoE server performance and packet lose problems

Thu Mar 27, 2008 2:22 pm

And a new beta of 3.7 just arrived. Testing again... :)
 
gacopl
Frequent Visitor
Frequent Visitor
Posts: 63
Joined: Sun Jul 29, 2007 5:11 pm
Location: Poland

Re: PPPoE server performance and packet lose problems

Fri Mar 28, 2008 12:08 pm

Hope that "7" will be lucky number I have same problem, but i rolled back to 2.9.51. I can't afford this to happen in my net.

Cheers
Michal
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Fri Mar 28, 2008 2:31 pm

Hope that "7" will be lucky number I have same problem, but i rolled back to 2.9.51. I can't afford this to happen in my net.

Cheers
Michal
I have this same problem since version 2.9.50, so rollback didn´t solve my problem. But with older versions (2.9.4x) works fine.
 
rastod
Member Candidate
Member Candidate
Topic Author
Posts: 122
Joined: Sat Jun 04, 2005 11:35 pm
Location: Slovakia

Re: PPPoE server performance and packet lose problems

Fri Mar 28, 2008 2:46 pm

Don't wory, 3.7 seems to be really very good.
Running 24 hours, no problems yet.
 
gacopl
Frequent Visitor
Frequent Visitor
Posts: 63
Joined: Sun Jul 29, 2007 5:11 pm
Location: Poland

Re: PPPoE server performance and packet lose problems

Fri Mar 28, 2008 3:36 pm

I have this same problem since version 2.9.50, so rollback didn´t solve my problem. But with older versions (2.9.4x) works fine.
You see, this is what worries me for me packet losses stopped as soon as i rolled back to 2.9.51, OMG maybe i have different problem :(
Don't wory, 3.7 seems to be really very good.
Running 24 hours, no problems yet.
Is there any way i could get as well that beta to test it?

cheers
Michal
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Fri Mar 28, 2008 4:41 pm

Don't wory, 3.7 seems to be really very good.
Running 24 hours, no problems yet.
Glad to hear :D
 
EgyCom
Member Candidate
Member Candidate
Posts: 123
Joined: Thu May 31, 2007 9:47 pm

Re: PPPoE server performance and packet lose problems

Sat Mar 29, 2008 7:32 pm

Don't wory, 3.7 seems to be really very good.
Running 24 hours, no problems yet.

any update ??
 
rastod
Member Candidate
Member Candidate
Topic Author
Posts: 122
Joined: Sat Jun 04, 2005 11:35 pm
Location: Slovakia

Re: PPPoE server performance and packet lose problems

Sat Mar 29, 2008 7:46 pm

Still the same, working fine..., no problems as before.
 
airnet
Frequent Visitor
Frequent Visitor
Posts: 83
Joined: Thu Feb 09, 2006 12:46 pm

Re: PPPoE server performance and packet lose problems

Sun Mar 30, 2008 1:55 am

I recently upgraded a 2.9.26 PPPoE Server box to 2.9.51. Nothing has changed except the version.
What I see are users falling off one by one slowly with message logged 'peer not responding'. They log back in fine next time they retry after 30 secs (SOHO router PPPoE clients).
Over a 24hr period, maybe 50 users per PPPoE server willl fall off for no apparent reason.

The ability of 2.9.51 to 'hold' PPPoE sessions is not solid.

Can anyone confirm the last known good version of 2.9.x ?

Also, accoring to this thread, v3.7 should fix this problem. Does this mean they will also release 2.9.52 to fix the 2.9 branch ?
 
rastod
Member Candidate
Member Candidate
Topic Author
Posts: 122
Joined: Sat Jun 04, 2005 11:35 pm
Location: Slovakia

Re: PPPoE server performance and packet lose problems

Sun Mar 30, 2008 11:34 am

We use PPPoE servers for thousands of PPPoE clients and we never had such problem. Check your connectino timeout settigs may be. There were another problems on 2.9.5x.
 
denio
just joined
Posts: 3
Joined: Wed Aug 17, 2005 1:13 am

Re: PPPoE server performance and packet lose problems

Sun Mar 30, 2008 6:33 pm

When will this version 3.7 be available for download ?

We are having this packet loss problem in two of our pppoe concentrators (versions 3.1 and 3.6) and need to solve it really soon... We are having a lot of users compliants...

Thanks in advance.
 
gacopl
Frequent Visitor
Frequent Visitor
Posts: 63
Joined: Sun Jul 29, 2007 5:11 pm
Location: Poland

Re: PPPoE server performance and packet lose problems

Sun Mar 30, 2008 11:15 pm

yeah we could really get use of that, even of that beta version you say is working correctly, so please if someone can please share.

cheers
Michal
 
rkorolev
newbie
Posts: 43
Joined: Tue Oct 23, 2007 1:49 pm

Re: PPPoE server performance and packet lose problems

Mon Mar 31, 2008 9:36 am

We still have problems with 3.7 beta, it works good only a short period of time.
 
rastod
Member Candidate
Member Candidate
Topic Author
Posts: 122
Joined: Sat Jun 04, 2005 11:35 pm
Location: Slovakia

Re: PPPoE server performance and packet lose problems

Mon Mar 31, 2008 9:52 am

We do not have any problems with 3.7 build time Mar/27/2008.
We run it 4 days on 4 PPPoE server with 500-800 active connections each.
We have only little performance problems that are common for older versions 2.9.x
 
gacopl
Frequent Visitor
Frequent Visitor
Posts: 63
Joined: Sun Jul 29, 2007 5:11 pm
Location: Poland

Re: PPPoE server performance and packet lose problems

Mon Mar 31, 2008 10:00 am

Then please share this beta somewhere or give us link to it or whatever :)
 
santajosh
Frequent Visitor
Frequent Visitor
Posts: 69
Joined: Fri Dec 09, 2005 3:20 pm

Re: PPPoE server performance and packet lose problems

Wed Apr 02, 2008 9:47 am

Hi EveryOne!

We also have this problem with our PPPOE server (600 users). Every day once we have to reboot the server (packetlose, performance problem)...

Where can we download the 3.7 beta??? Or when will be 3.7 ???

Thanks in advice!

Josh
 
rastod
Member Candidate
Member Candidate
Topic Author
Posts: 122
Joined: Sat Jun 04, 2005 11:35 pm
Location: Slovakia

Re: PPPoE server performance and packet lose problems

Wed Apr 02, 2008 10:03 am

We tested the 3.7 package, from our point of view it is suitable to be released. But now it depends only on MT.
 
santajosh
Frequent Visitor
Frequent Visitor
Posts: 69
Joined: Fri Dec 09, 2005 3:20 pm

Re: PPPoE server performance and packet lose problems

Wed Apr 02, 2008 10:11 am

thanks!

So Normis???
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Wed Apr 02, 2008 5:35 pm

We do not have any problems with 3.7 build time Mar/27/2008.
We run it 4 days on 4 PPPoE server with 500-800 active connections each.
We have only little performance problems that are common for older versions 2.9.x
Just to know have kind of little performance problem you have?
 
rastod
Member Candidate
Member Candidate
Topic Author
Posts: 122
Joined: Sat Jun 04, 2005 11:35 pm
Location: Slovakia

Re: PPPoE server performance and packet lose problems

Wed Apr 02, 2008 5:58 pm

1. when there are more users (700-900), we can see that sometimes ping to localhost is higher, up to 2-5ms, but not every ping, only 10-30%
2. we have problem that users are not able to get higher speeds than 15-20Mbit for download and 5-9Mbit for upload. But this is common problem with older 2.9.x versions and it is only one unpleasant feature of PPPoE compared to DHCP
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Wed Apr 02, 2008 10:03 pm

1. when there are more users (700-900), we can see that sometimes ping to localhost is higher, up to 2-5ms, but not every ping, only 10-30%
2. we have problem that users are not able to get higher speeds than 15-20Mbit for download and 5-9Mbit for upload. But this is common problem with older 2.9.x versions and it is only one unpleasant feature of PPPoE compared to DHCP
Thanks for reply. At least for my case these are minor issues.
 
gacopl
Frequent Visitor
Frequent Visitor
Posts: 63
Joined: Sun Jul 29, 2007 5:11 pm
Location: Poland

Re: PPPoE server performance and packet lose problems

Thu Apr 03, 2008 10:17 am

Hello

Tell me guys is it so hard to put this build somewhere? Can't you see that there some desperate guys here, that really need to try this out? Please put it somewhere and share this can't be that hard.

Cheers
Michal
 
rkorolev
newbie
Posts: 43
Joined: Tue Oct 23, 2007 1:49 pm

Re: PPPoE server performance and packet lose problems

Thu Apr 03, 2008 2:57 pm

At least one more problem was found with test build.
Problem will be fixed in next build and at this point it looks like it will be
the last - so, please, wait for the release of 3.7
 
gacopl
Frequent Visitor
Frequent Visitor
Posts: 63
Joined: Sun Jul 29, 2007 5:11 pm
Location: Poland

Re: PPPoE server performance and packet lose problems

Thu Apr 03, 2008 4:43 pm

At least one more problem was found with test build.
Problem will be fixed in next build and at this point it looks like it will be
the last - so, please, wait for the release of 3.7
Yeah, tell that to my users ;)

I think i have a little more patience in me, hopefully my users too :)

Cheers
Michal
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Mon Apr 07, 2008 4:21 pm

3.7 was released today but changelog has nothing about PPPoE fixes!!!!!!!!!!!!!!!!
 
mietus
Member Candidate
Member Candidate
Posts: 122
Joined: Mon Jan 30, 2006 11:14 pm
Location: Poland

Re: PPPoE server performance and packet lose problems

Mon Apr 07, 2008 4:26 pm

pppoe fixes are included but i dont know why not described in changelog :>
 
whitlebitle
just joined
Posts: 5
Joined: Tue Feb 05, 2008 2:18 am

Re: PPPoE server performance and packet lose problems

Mon Apr 07, 2008 5:37 pm

pppoe fixes are included but i dont know why not described in changelog :>
Are pppoe problems really fixed?
 
User avatar
hulk-bd
Member Candidate
Member Candidate
Posts: 227
Joined: Mon Sep 03, 2007 7:19 pm
Location: Uttara, Dhaka, Bangladesh

Re: PPPoE server performance and packet lose problems

Mon Apr 07, 2008 9:16 pm

Yes pppoe problem is fixed take a look in this topic.

http://forum.mikrotik.com/viewtopic.php?f=2&t=23013

Thanks
Don't worry, be happy :) .......
 
EgyCom
Member Candidate
Member Candidate
Posts: 123
Joined: Thu May 31, 2007 9:47 pm

Re: PPPoE server performance and packet lose problems

Tue Apr 08, 2008 3:40 am

Today I upgraded one of our Access Concentrator to 3.7
I only load it with 10Mbps and 200 users, its work great, packet loss and latency issues vanished
I will try tomorrow with 40Mbps and 600-700 users
Good work Mikrotik team
 
rkorolev
newbie
Posts: 43
Joined: Tue Oct 23, 2007 1:49 pm

Re: PPPoE server performance and packet lose problems

Tue Apr 08, 2008 8:59 am

Currently 50-58mbit/s, 580 users (325 queues) - packet loss i still here :(
To RouterOS itself:
600 packets transmitted, 594 packets received, 1% packet loss

The absolutely same story via connected PPPoE:
Packets: Sent = 600, Received = 594, Lost = 6 (1% loss),

It's better than 2-5% loss in older versions, but our freebsd/pppoed server with 500 users online did not lose even 1 packet :(
 
rastod
Member Candidate
Member Candidate
Topic Author
Posts: 122
Joined: Sat Jun 04, 2005 11:35 pm
Location: Slovakia

Re: PPPoE server performance and packet lose problems

Tue Apr 08, 2008 9:59 am

Currently 50-58mbit/s, 580 users (325 queues) - packet loss i still here :(
To RouterOS itself:
600 packets transmitted, 594 packets received, 1% packet loss

The absolutely same story via connected PPPoE:
Packets: Sent = 600, Received = 594, Lost = 6 (1% loss),

It's better than 2-5% loss in older versions, but our freebsd/pppoed server with 500 users online did not lose even 1 packet :(
Do you have experiences with freebsd/pppoe? Is it good solution? Why are you trying MT PPPoE?
 
rkorolev
newbie
Posts: 43
Joined: Tue Oct 23, 2007 1:49 pm

Re: PPPoE server performance and packet lose problems

Tue Apr 08, 2008 10:27 am

that's not a topic for routeros forum ;)
you can write me a mail to Image
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Tue Apr 08, 2008 5:30 pm

Here are my results with version 3.7.

150 pppoe users, 8Mbits - 1000 packets - lost average 3 packets (it less than 1% but packet loss is still there).

I´m not happy with this result, old Mk versions never loose a single packet with much more users and traffic.
 
rkorolev
newbie
Posts: 43
Joined: Tue Oct 23, 2007 1:49 pm

Re: PPPoE server performance and packet lose problems

Tue Apr 08, 2008 7:34 pm

We also have packet loss 1% at least, but sometimes I've seen a 3% loss on a 600 packets
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Tue Apr 08, 2008 11:09 pm

I think we should report to Mk support that we are not happy with this solution, or else they will believe it´s fine.

I cannot accept a new version that works worst than an old version. And loose packets in LAN enviroment make this solution useless.
 
EgyCom
Member Candidate
Member Candidate
Posts: 123
Joined: Thu May 31, 2007 9:47 pm

Re: PPPoE server performance and packet lose problems

Wed Apr 09, 2008 4:57 am

I think we should report to Mk support that we are not happy with this solution, or else they will believe it´s fine.

I cannot accept a new version that works worst than an old version. And loose packets in LAN enviroment make this solution useless.
We didn't face any problem in 3.7 PPPoE
0 packet loss , with nearly 350 user
Packet loss only from 1 to 3 % for users with near there maxim speed limit and I think that is normal
We disabled compression and encryption, but still use Change TCP MSS
Server is 3.2 MHz, 512 Ram, multi-cpu=no
And we will try with more users ….
 
rpingar
Long time Member
Long time Member
Posts: 541
Joined: Fri May 28, 2004 2:46 pm
Location: Italy

Re: PPPoE server performance and packet lose problems

Wed Apr 09, 2008 10:01 am

is the cpu sage little bit lower then the previous releases?

regards
Ros
 
EgyCom
Member Candidate
Member Candidate
Posts: 123
Joined: Thu May 31, 2007 9:47 pm

Re: PPPoE server performance and packet lose problems

Wed Apr 09, 2008 12:25 pm

is the cpu sage little bit lower then the previous releases?

regards
Ros
No I think it’s the same.
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Wed Apr 09, 2008 2:47 pm

It seems they fixed a problem with traffic shape queue creation but it also lose packets when an authentication request is denied (so this is not related with queues). If you have users that try to authenticante entire day you will lose packets.

I have an old machine (Sempron AMD processor) with Mk 2.9.44 with about 300 users, more than 15Mbits and never lost a packet, same Mk configuration and user profile. With new versions even with a Pentium D 3.4Ghz 512M RAM (Intel Gigabit PCI-E interf.) with multi-core on or off I have packetloss.

At some point near Mk version 2.9.50 it starts to lost packets on user authentication request (denied or not).

And remember I´m talking about lose packets even at local interface (inside my local network).
 
rkorolev
newbie
Posts: 43
Joined: Tue Oct 23, 2007 1:49 pm

Re: PPPoE server performance and packet lose problems

Wed Apr 09, 2008 2:51 pm

Yes, we have exactly same situation.
 
rpingar
Long time Member
Long time Member
Posts: 541
Joined: Fri May 28, 2004 2:46 pm
Location: Italy

Re: PPPoE server performance and packet lose problems

Wed Apr 09, 2008 4:03 pm

Any experience on multi-cpu=yes about 3.7?
is it stable the smp verision about pppoe server?

regards
Ros
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Wed Apr 09, 2008 5:22 pm

Any experience on multi-cpu=yes about 3.7?
is it stable the smp verision about pppoe server?

regards
Ros
For us since 3.4 it was already stable with multi-cpu=yes, but it´s not helping to enable the second core since I continue to have dropped packets.
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Fri Apr 11, 2008 3:21 pm

Yes, we have exactly same situation.
Have you reported to Mk support you still have problem with it? We are trying to report to them but they insist it was solved (they really solved part of the problem). I know more people which continue to have this problem so I think everybody should report. Look like when you have a few more authenticantion request (even if all denied) it lose packets.
 
dsovereen
Frequent Visitor
Frequent Visitor
Posts: 53
Joined: Fri Oct 22, 2004 7:54 pm
Location: Michigan, USA
Contact:

Re: PPPoE server performance and packet lose problems

Fri Apr 11, 2008 8:48 pm

I sent an e-mail with a supout.rif, screenshot, and ping log of the problem.

I've found that doing:

ping 127.0.0.1 size=1500 interval=100ms

makes it really easy to see the problem. We run both PPPoE as well as DHCP. As Mikrotik indicated, it really does seem to be related to Simple Queues, because any time a PPPoE user disconnects/reconnects or a DHCP lease deassigns/assigned, both of which cause a Simple Queue to be dropped and added, we drop about 3 pings.

I haven't done enough testing to determine if it is the dropping of the queue or the building of the queue that causes the packet loss. Maybe it's both.

Everyone else who is seeing this, please send an e-mail to support@mikrotik.com so we can hopefully get this fixed.

Dave
 
muriachi
just joined
Posts: 3
Joined: Wed Jun 06, 2007 5:04 pm

Re: PPPoE server performance and packet lose problems

Sat Apr 12, 2008 2:19 pm

Yes, we have exactly same situation.
me too :shock:
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1723
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: PPPoE server performance and packet lose problems

Mon Apr 14, 2008 10:57 am

So you have a big QoS system and some little packet loss?
Bus as far as I know it is possible to limit traffic ONLY by dropping packets!!!

regarding the constant packet loss - did you really determined it?
if you say you have 3% packet loss thats mean you should have the same number with 1pps 10pps 100pps and 1000pps.

I just checked same of my routers (not only mikrotiks) with your unique method (ping 127.0.0.1) and almost in all I was able to see some pings missing.

So bottom line is - do you have real packet loss problem???? I know only one way how to determine it is - online gamers (Quake, Unreal tournament, Counter-Strike) - those guys will be first who will find out even the smallest disturbance in the network...
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Mon Apr 14, 2008 2:28 pm

So you have a big QoS system and some little packet loss?
Bus as far as I know it is possible to limit traffic ONLY by dropping packets!!!

regarding the constant packet loss - did you really determined it?
if you say you have 3% packet loss thats mean you should have the same number with 1pps 10pps 100pps and 1000pps.

I just checked same of my routers (not only mikrotiks) with your unique method (ping 127.0.0.1) and almost in all I was able to see some pings missing.

So bottom line is - do you have real packet loss problem???? I know only one way how to determine it is - online gamers (Quake, Unreal tournament, Counter-Strike) - those guys will be first who will find out even the smallest disturbance in the network...
Packet loss occurs on interfaces that doesn´t have any QoS or queues applied. We are not talking about packetloss for a user thru Mk route. It happens on 100Mbits or Gigabit interface with no QoS applied. It must not happen on a regular router, never! I use Mk as PPPoE server for years and it never happens before. Something is clearly wrong here.

In my case this happen with only 200 users online, which is not much.

And if problem is only related to queues why I lose packets even when I authentication request is denied?
 
dsovereen
Frequent Visitor
Frequent Visitor
Posts: 53
Joined: Fri Oct 22, 2004 7:54 pm
Location: Michigan, USA
Contact:

Re: PPPoE server performance and packet lose problems

Mon Apr 14, 2008 5:58 pm

As best I can tell, the problem occurs where all packets on all interfaces are dropped for this period of time, regardless if they are packets to be shaped by the simple queue being added/removed. All packets on all interfaces drop for that 250 ms window (roughly) while the simple queue is being added/dropped. Yes, this is a very bad thing.

Dave
 
rpingar
Long time Member
Long time Member
Posts: 541
Joined: Fri May 28, 2004 2:46 pm
Location: Italy

Re: PPPoE server performance and packet lose problems

Sun May 25, 2008 11:16 am

we are running 3.7 on our pppoe servers and are seeing packt loss and another strange issue:

when we run bandwidth test to any host directly connected to the switch about private network we are able to send only 10mbps of udp traffic at 1500.
If I run bandwidth test through a pc connected on the same switch we are able to send at full speed.

The pppoe servers are not configured with any qos system that can cut the bandwidth.
My idead is that on this scenario that could y a packt flow problem.

Regards
Ros
 
User avatar
eugenevdm
Member Candidate
Member Candidate
Posts: 208
Joined: Tue Jun 01, 2004 12:23 pm
Location: Stellenbosch, South Africa
Contact:

Re: PPPoE server performance and packet lose problems

Sun May 25, 2008 12:50 pm

Are you using simple queues? If so you should really be running a later version of RouterOS since a performance issue regarding simple queues was corrected in version 3.8.
The Snowball Effect
Superior Internet Solutions
 
rpingar
Long time Member
Long time Member
Posts: 541
Joined: Fri May 28, 2004 2:46 pm
Location: Italy

Re: PPPoE server performance and packet lose problems

Sun May 25, 2008 12:57 pm

i think it was corrected into 3.7
I don't see any change loc about simple queue into the 3.8

regards
Ros
 
User avatar
eugenevdm
Member Candidate
Member Candidate
Posts: 208
Joined: Tue Jun 01, 2004 12:23 pm
Location: Stellenbosch, South Africa
Contact:

Re: PPPoE server performance and packet lose problems

Sun May 25, 2008 1:37 pm

Sorry you are correct:

What's new in 3.7:

*) improved simple queue list updating;

but I still suggest you start using version 3.9 because very few people, including probably the folk at Mikrotik, will help you with prior versions. The problem is Mikrotik 3.x is new and changes very quickly so it's mostly better to use the latest version. Then again I also realize there are risks associated with upgrading routers.
The Snowball Effect
Superior Internet Solutions
 
hci
Long time Member
Long time Member
Posts: 601
Joined: Fri May 28, 2004 5:10 pm

Re: PPPoE server performance and packet lose problems

Sun May 25, 2008 6:28 pm

We are running 3.9 on a PPPoE server serving about ~800 users. We are seeing all sorts of weird things. Some times proxy arp just does not seem to work right. Sometimes users just cannot go anywhere. And an old bug we thought we got rid of when we moved too 3.x of queues not always being enforced on PPPoE users. A user with a 256k profile will be seen running at 4.5m or so.

Tempted to downgrade too 3.3 or 3.4 release. Hopefully 3.10 will be out soon and solve all our problems?

Matt
 
User avatar
eugenevdm
Member Candidate
Member Candidate
Posts: 208
Joined: Tue Jun 01, 2004 12:23 pm
Location: Stellenbosch, South Africa
Contact:

Re: PPPoE server performance and packet lose problems

Sun May 25, 2008 11:22 pm

We are seeing all sorts of weird things.
Ouch. Just for interest sake did you encounter any problems with 2.9.51? Is there a specific reason why you upgraded to 3.x?

I upgraded 2 x PC routers to 3.9 and both crashed...I didn't experiment much thereafter because they were production routers so I quickly downgraded. Since then no problems.

Personally I would like to implement 3.x routers because I need faster / larger SATA hard drives and more RAM for my proxy. But I can wait for stability.
The Snowball Effect
Superior Internet Solutions
 
hci
Long time Member
Long time Member
Posts: 601
Joined: Fri May 28, 2004 5:10 pm

Re: PPPoE server performance and packet lose problems

Sun May 25, 2008 11:40 pm

Ouch. Just for interest sake did you encounter any problems with 2.9.51? Is there a specific reason why you upgraded to 3.x?
The main reason for the move to v3 was to get the fix for queues not always being inforced and moving to v3 seemed to fix it. We are short on bandwidth right now waiting for our new circuits to be installed and not inforcing queues was a real issue. Everything actually seemed to work fine with v3 until something like 3.5 or somewhere in there. Not at all sure when, its hard to keep track. I heard starting with 3.5 Mikrotik has a new kernel. Not sure if thats true.

Matt
 
User avatar
eugenevdm
Member Candidate
Member Candidate
Posts: 208
Joined: Tue Jun 01, 2004 12:23 pm
Location: Stellenbosch, South Africa
Contact:

Re: PPPoE server performance and packet lose problems

Mon May 26, 2008 4:18 am

As far as I know the Kernel change was between 2.9.x and 3.x.

Have you actually even tried 2.9.x and did you have a queue issue with that version?
The Snowball Effect
Superior Internet Solutions
 
santajosh
Frequent Visitor
Frequent Visitor
Posts: 69
Joined: Fri Dec 09, 2005 3:20 pm

Re: PPPoE server performance and packet lose problems

Tue May 27, 2008 12:36 pm

Any stable version for PPPOE? We use 3.10 but we have queue problem...
2.9.51? Anybody use 2.9.51 to PPPOE? We downgrade now to 3.3 but now we have the packet loss problem before 3.7...:(

:(
 
rpingar
Long time Member
Long time Member
Posts: 541
Joined: Fri May 28, 2004 2:46 pm
Location: Italy

Re: PPPoE server performance and packet lose problems

Tue May 27, 2008 1:16 pm

has someone that is running latest releas sent supout to MT about the not working queue?

I had this bug on some of the latest 2.9 releases. Now with 3.7 I dont see this behavior but i am still getting some packet loss.
My experience is that it is more pronunced on Broadcom tigon3 gbit ethernet then Intel gbit ethernet cards.

In fact i have two pppoe server with the same exact configuration and i see this problem only on the server equiped with broadcom tigon3 driver, on the other one with intel i don't lose the packets.


regards
Ros
 
User avatar
eugenevdm
Member Candidate
Member Candidate
Posts: 208
Joined: Tue Jun 01, 2004 12:23 pm
Location: Stellenbosch, South Africa
Contact:

Re: PPPoE server performance and packet lose problems

Tue May 27, 2008 2:24 pm

Anybody use 2.9.51 to PPPOE?
I use 2.9.x and 2.9.51 on various servers with PPPoE client on high sites. The largest site has about 100 PPPoE clients. I have never had any problems with queues.
The Snowball Effect
Superior Internet Solutions
 
rpingar
Long time Member
Long time Member
Posts: 541
Joined: Fri May 28, 2004 2:46 pm
Location: Italy

Re: PPPoE server performance and packet lose problems

Tue May 27, 2008 2:57 pm

100 is a very small number.

You have to test them with 800-1000 clients.

Regards
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 24333
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: PPPoE server performance and packet lose problems

Tue May 27, 2008 3:09 pm

for so many clients, I hope you are using RADIUS authentication ...
No answer to your question? How to write posts
 
rpingar
Long time Member
Long time Member
Posts: 541
Joined: Fri May 28, 2004 2:46 pm
Location: Italy

Re: PPPoE server performance and packet lose problems

Tue May 27, 2008 3:26 pm

sure!

we have around 1200 on the intel gbit interfaces router and they are fine.
around 400 on the broadcom tigon and we are experiencing some little packet loss.......

same identical configuration to both machines. Supout sent two day ago to support.

Regards
Ros
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Tue May 27, 2008 5:00 pm

We have some packetloss with version 3.7 and Intel Gigabit NIC. I don´t know if with Broadcom is worst but we have this problem with Intel (PCI or PCI-E).
 
rpingar
Long time Member
Long time Member
Posts: 541
Joined: Fri May 28, 2004 2:46 pm
Location: Italy

Re: PPPoE server performance and packet lose problems

Tue May 27, 2008 5:24 pm

strange that the same hardware (dell poweredge 860) configured as core router so without any dynamic simple queue but with a queue tree is working great without any packet loss even pinging the pppoe server with broadcom ethernet nics.

The vice versa pinging from pppoe server to the core router I lose packets..........very strange

Ros.
 
hci
Long time Member
Long time Member
Posts: 601
Joined: Fri May 28, 2004 5:10 pm

Re: PPPoE server performance and packet lose problems

Tue May 27, 2008 5:35 pm

for so many clients, I hope you are using RADIUS authentication ...
Wouldn't an up to date PPPoE radius how too in the wiki be nice. Needs to cover IP pools, duplicate rejection, profiles and etc.

Matt
 
rpingar
Long time Member
Long time Member
Posts: 541
Joined: Fri May 28, 2004 2:46 pm
Location: Italy

Re: PPPoE server performance and packet lose problems

Thu May 29, 2008 12:05 am

We have some packetloss with version 3.7 and Intel Gigabit NIC. I don´t know if with Broadcom is worst but we have this problem with Intel (PCI or PCI-E).
Is it a multi-cpu box or an old fashion single core?

Regards
Ros
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Fri May 30, 2008 10:31 pm

We have some packetloss with version 3.7 and Intel Gigabit NIC. I don´t know if with Broadcom is worst but we have this problem with Intel (PCI or PCI-E).
Is it a multi-cpu box or an old fashion single core?

Regards
Ros
It´s a P4 3.4Ghz dual-core with intel server mobo.
 
rpingar
Long time Member
Long time Member
Posts: 541
Joined: Fri May 28, 2004 2:46 pm
Location: Italy

Re: PPPoE server performance and packet lose problems

Fri May 30, 2008 10:36 pm

uldis......

i think the actual packet loss is about an snm issue!!!!

this report from josefranco seems confirming my idea.

Regards
Ros
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Mon Jun 02, 2008 5:11 pm

uldis......

i think the actual packet loss is about an snm issue!!!!

this report from josefranco seems confirming my idea.

Regards
Ros
Sorry about my bad english but what´s a snm?

Just for information I had this same problem with a P4 3.0 single CPU on a totally different hardware. It starts to happen for me on 2.9.4x version.
 
rpingar
Long time Member
Long time Member
Posts: 541
Joined: Fri May 28, 2004 2:46 pm
Location: Italy

Re: PPPoE server performance and packet lose problems

Mon Jun 02, 2008 5:22 pm

it means multi core cpu.

Regards
Ros
 
josefranco
Frequent Visitor
Frequent Visitor
Posts: 95
Joined: Fri Mar 07, 2008 5:17 pm

Re: PPPoE server performance and packet lose problems

Mon Jun 02, 2008 5:40 pm

it means multi core cpu.

Regards
Ros
In this case I don´t think it´s related to multi-core cpu. I had this problem with single core CPU too.

Who is online

Users browsing this forum: No registered users and 78 guests