Community discussions

MUM Europe 2020
 
glucz
Member Candidate
Member Candidate
Topic Author
Posts: 123
Joined: Wed Jun 06, 2007 10:25 pm

Possible bug on 5.1 (memory leak, network or kernel?)

Sat Apr 16, 2011 10:39 pm

Hello,

Actually I have this problem since the 5.xRC versions: I have a few x86 installations running as VPN servers serving PPTP and L2TP. The 5.x versions will start loosing packets after a while. It all seems to depend on load / traffic. The heavier the load is, the sooner the packet loss will start. It will eventually happen on all of them. If look at the console, everything looks OK and no errors are logged. The only way to fix this is to reboot.

Network cards are usually realtek or intel. It also happens in virtualized environments (Xen, virtualbox, vmware) regardless of the architecture used.

I have a server with moderate load: 3mbit/s avergate traffic. This will start loosing packets usually every 24 hours. So I have to reboot every day.

Since I can reproduce this every time, I will be happy to help you track this, just tell me what to send.

GL

ps: I have 3.x and 4.x versions running the same service, in the same environment handling average loads of 30mbit/s without this problem - so I have a basis of comparison.
 
User avatar
omega-00
Forum Guru
Forum Guru
Posts: 1167
Joined: Sat Jun 06, 2009 4:54 am
Location: Australia
Contact:

Re: Possible bug on 5.1 (memory leak, network or kernel?)

Sun Apr 17, 2011 1:04 pm

Can't say I've seen the same problem on v5.0 but haven't done enough testing in v5.1 to confirm or deny what you're seeing.
Edit: bah just realised I cut off the packet error section..
mikrotik-5.0-x86.PNG
You do not have the required permissions to view the files attached to this post.
brightwifi.com | mikrotik-routeros.com | MTCNA,MTCWE.MTCTCE | Give karma where due
 
glucz
Member Candidate
Member Candidate
Topic Author
Posts: 123
Joined: Wed Jun 06, 2007 10:25 pm

Re: Possible bug on 5.1 (memory leak, network or kernel?)

Sun Apr 17, 2011 6:59 pm

Hello,

I imagine that it is related to a specific service or task, so the most common features won't bring out this problem. I noticed this very early with the RC 5.x versions - but there were so many other problems that got fixed quickly that I didn't pay much attention to this. I however started to see the same problem in the final releases - so I assume that casual testing won't bring out the issue.

I just had a server that started loosing packets. I attached the interface screenshot below. Most clients already dropped off, but there are some still connected, showing 0 traffic - due to the packet loss. This is version 5.0 . RX drops seems to be high, but I have that on other installations as well - so that may not be a factor.

I also created a supout.rif file in that state of the router. I'm sending that to support.
You do not have the required permissions to view the files attached to this post.
 
User avatar
omega-00
Forum Guru
Forum Guru
Posts: 1167
Joined: Sat Jun 06, 2009 4:54 am
Location: Australia
Contact:

Re: Possible bug on 5.1 (memory leak, network or kernel?)

Sun Apr 17, 2011 7:40 pm

What is CPU usage like on the device at this time?
brightwifi.com | mikrotik-routeros.com | MTCNA,MTCWE.MTCTCE | Give karma where due
 
glucz
Member Candidate
Member Candidate
Topic Author
Posts: 123
Joined: Wed Jun 06, 2007 10:25 pm

Re: Possible bug on 5.1 (memory leak, network or kernel?)

Sun Apr 17, 2011 7:53 pm

Hello,

I already rebooted, so I don't have an actual answer, but if my memory serves right, it goes to 0%. So it doesn't look like a deadlock.

GL
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1730
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: Possible bug on 5.1 (memory leak, network or kernel?)

Mon Apr 18, 2011 12:44 pm

Sorry to ask this, but do you have some actual problems with services or all the panic is just about counter counting numbers, btw in 3.x and early 4.x there was no such counter available :)

If it is x86 then "/system hardware set multi-cpu=no" (and reboot) is good thing to try, this will load Kernel without SMP support (similar to one that they use on RouterBoards)

Also if you write to support send them 2 supouts - "good" one and "bad" one.
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
glucz
Member Candidate
Member Candidate
Topic Author
Posts: 123
Joined: Wed Jun 06, 2007 10:25 pm

Re: Possible bug on 5.1 (memory leak, network or kernel?)

Mon Apr 18, 2011 1:11 pm

Hello,

There are no problems with the counters. The actual problem is that unless the load is light, routeros will start loosing packets at an increasing rate. It will eventually loose all network connectivity. This happened to me on all 5.x based x86 servers, both RC and final, but never on 4.X or 3.X

I'm only using these for DNS and VPN servers, so it might be related to those services.

Currently all servers are running fine, but I expect one to go down today, so I'll have the supouts.

GL
 
tchus
Member Candidate
Member Candidate
Posts: 111
Joined: Tue Jan 25, 2011 12:08 am

Re: Possible bug on 5.1 (memory leak, network or kernel?)

Tue Apr 19, 2011 7:15 am

I believe this problem is not specific to x86. Although, I haven't had the chance to try on one of my x86 boxes.
I too am having exact same issues (5.0x and above), along with packet loss, things start to go down hill.
Memory diminishes within 3,4-5 days of normal client usage, then progressively gets worse as loads increase.
A reboot always fixes it. I'm running (hotspot/RB750g)with absolute minimal services.
Frustrating. Back to 4.17
 
glucz
Member Candidate
Member Candidate
Topic Author
Posts: 123
Joined: Wed Jun 06, 2007 10:25 pm

Re: Possible bug on 5.1 (memory leak, network or kernel?)

Tue Apr 19, 2011 3:08 pm

Hello,

I was able to send the supouts to support today. I also found an actual error that could be related to this problem. Please see screenshot below ... there could be many other factors involved, but my money is on memory leak now.
You do not have the required permissions to view the files attached to this post.
 
babbage
Trainer
Trainer
Posts: 37
Joined: Mon Jul 12, 2010 5:55 pm

Re: Possible bug on 5.1 (memory leak, network or kernel?)

Sat May 07, 2011 10:19 am

I confirm having the exact same issue...
I have couple of x86 devices, same and different configs. Same and different HW. I have the issue happening in a mixed of them while some do not have this issue...

ip route cache becomes full and router starts fainting...

Any fixes yet?

Who is online

Users browsing this forum: Google [Bot], nz_monkey and 89 guests