Community discussions

MikroTik App
 
User avatar
irrwitzer
just joined
Topic Author
Posts: 23
Joined: Mon Apr 11, 2022 11:54 pm

Urgent: need help finding memory leak (RoS v6.48.6)

Thu May 11, 2023 5:41 pm

Hi Guys,

Unfortunately I need help rather quickly, otherwise this Router will continue rebooting every ~26hrs (almost exactly 26h!)
We are having a problem with a CCR1036-8G-2S+ (16G of RAM).
Starting two days ago, one of our routers started to reboot with OOM error messages.
RAM usage shows this pattern:
memoryleak.png
There have been no changes to the configuration at all according to config history/backup.

What's the best path to identify the process/configuration that is eating up all those memory?

Thanks for any help!

Irrwitzer
You do not have the required permissions to view the files attached to this post.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 11967
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Urgent: need help finding memory leak (RoS v6.48.6)

Thu May 11, 2023 6:49 pm

1) DoH
2) Firewall dynamic address list
3) SYN flood attack
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 2989
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: Urgent: need help finding memory leak (RoS v6.48.6)

Thu May 11, 2023 8:34 pm

as rextended says i think is not a memory leak i have some Routers with months of uptime with that version


some missconfiguration
or
you are being ddos attacked

to start check this:
/ip firewall connection tracking print
check the total entries value
 
User avatar
irrwitzer
just joined
Topic Author
Posts: 23
Joined: Mon Apr 11, 2022 11:54 pm

Re: Urgent: need help finding memory leak (RoS v6.48.6)

Thu May 11, 2023 8:46 pm

Thanks guys!
Will check ASAP.

It’s definitely not DOH and no dynamic address lists either, but I will look into the possible DoS.

I wasn’t thinking of a real bug in the OS (we have multiple tenth of them in the field), but thinking of bad configuration and wanted to identify the service/thing we misconfigured. On Linux top/ps would have told me the ressources used per service. On Mtk I don’t know anything like it.

Thanks, will get back here as soon as I know more.,
 
User avatar
irrwitzer
just joined
Topic Author
Posts: 23
Joined: Mon Apr 11, 2022 11:54 pm

[RESOLVED]: Urgent: need help finding memory leak (RoS v6.48.6)  [SOLVED]

Fri Oct 13, 2023 4:17 pm

Hi again,

I wanted to update this posting.

- No (D)DoS
- No DoH
- No dynamic address-lists
- Not debuggable with on router tools
- Ticket opened (SUP-129092, with at least two supout files of impacted routers) but didn't get any replies
- 5-6 routers that crashed regulary because of this in different cycles, so it's not just equal timing, it must be some kind of load/traffic/usage dependent problem

I'm thinking:
- Logging (storage)?
- IPsec stuff?

Regardless, I tried updating one of the routers to 6.49.10 and the problem disappeared. Memory usage doesn't climb above 7% now.
So I updated the other 29 routers still on 6.48.6 as well.

Everything is fine now. Still it bothers me, not to know what caused it.

BR,

Irrwitzer
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 2989
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: Urgent: need help finding memory leak (RoS v6.48.6)

Fri Oct 13, 2023 4:47 pm

thank you for sharing
 
jzhu47
just joined
Posts: 11
Joined: Tue Oct 08, 2019 8:53 pm

Re: Urgent: need help finding memory leak (RoS v6.48.6)

Fri Oct 13, 2023 6:34 pm

if none of above as well as ddos, then you have to consider any change before getting this issue? includes but not limit, adding new server, PC. installing new application on PC etc..

Who is online

Users browsing this forum: Google [Bot] and 65 guests