Page 1 of 1

SMP MultiCore fix update

Posted: Thu Jul 09, 2009 12:25 pm
by normis
People with problems on Multi-CPU and Multi-Core problems, please report to support to receive a special pre-release package for testing.

Thanks to people that are already helping us with reports to support, we have made quite a lot of improvements in this regard.

Re: SMP MultiCore fix update

Posted: Sat Jul 11, 2009 8:53 am
by hcidc
3.26 Pre rel

Q8200
2GB DDR2 800
ASUS P5BV

2PORT Intel 9402PT
2PORT BCM 5721

Version 3.26 CPU usage = version 3.25 CPU usage

Re: SMP MultiCore fix update

Posted: Sun Jul 12, 2009 11:48 am
by Chupaka
Normis, could you please give us some details on what was changed? =)

hcidc, please write to support@mikrotik.com - it's better to open a ticket with your request

Router-OS SMP (multi-core and multi-cpu) crashes part 2

Posted: Tue Jul 14, 2009 2:30 pm
by omega-00
I've been testing a pre-release copy of v3.26 for mikrotik over the last couple of days.

1. It works, the system did not crash over the 4 days it was running (with hotspot etc) whereas previously (pre-3.26) any multi-cpu device would crash within 24 hours of a reboot while running the hotspot
2. There WAS a memory leak issue in the prerelease which mikrotik have now fixed in the newest release however I haven't had a proper chance to test this myself just yet so I would advise anyone testing this on a multi-cpu system to CHECK back a couple of times after you install it.

Other than that, thanks heaps to the mikrotik team for fixing the issue and I'm looking forward to seeing some incredibly beefy multi-core routers in the future :-)

Update: Memory leak still seems to be present in the release version. Have submitted ticket and details.

Re: SMP MultiCore fix update

Posted: Wed Jul 15, 2009 7:40 am
by radioresearch
I d/l'ed and installed 3.26 this morning on a Dell PowerEdge 2970 quad-core with 2GB memory.

Multi-CPU now seems to be quite stable (yay!), however, I still have a memory leak. Tried three different pair of matched memory and it is still there. Leak averaged 250 MB in a 3.5 hour period. I will wait to see if it stabilizes over the next 24 hours.

I am running the optional Dude package as well, since this is optimum to work with the different subnets on this router.

More if and when it develops.

UPDATE: 20090715 0950L
After 5.5 hours, memory dropped to 651MB. First reboot since the installation of 3.26. Let's see if it was a glitch.

UPDATE: 20090715 2230L

After a little over 12 hours, we had leaked over a gig of memory. I have disabled the Dud(e) and rebooted the machine. Memory started out at 1757 MB and in 7 minutes is down to 1738 MB. Maybe it will stabilize. Let's see what the morning brings. Was the right version of the ROS posted on the website today? Seems to be still a memory leak problem even with Dude disabled.


UPDATE: 20090716 0520L

In almost 7 hours the memory leak caused about 150 MB of memory loss. This is much better than with Dude activated, but by no means acceptable. ROS 3.26 final still has a memory leak issue that really needs to get fixed.

Mikrotik, please fix up this memory leak, and things will be great with this ROS!

UPDATE: 20090716 1330L

Well, case closed. It will be fixed with the next release. Ticket No. 2009071666000112 elicited the following response from Janis:

"Hello,
It is DNS related problem. We fixed it already, will be included in the next release. to avoid this you can disable DNS caching.

Regards,
Janis Megis"

Well, that's not really what I was hoping to hear, and I hope this will be released fixed soon.

Re: SMP MultiCore fix update

Posted: Wed Jul 15, 2009 9:17 am
by omega-00
I'm also running the dude on my machine, haven't got one without it to test on at the moment. Maybe thats the source of the leak?

My leak seems a lot slower than yours, uses 250Mb in about 14 hours (out of 2Gb total) so I can afford to wait for a new MT pre-release :-)

Re: SMP MultiCore fix update

Posted: Wed Jul 15, 2009 9:24 am
by macgaiver
Did you write to support? with 2 supout.rifs - one with, other without memory leak (no reboots in between)

Re: SMP MultiCore fix update

Posted: Wed Jul 15, 2009 9:47 am
by omega-00
Did you write to support? with 2 supout.rifs - one with, other without memory leak (no reboots in between)
if you're talking to me, the 2 releases I've tried were:
1. 10/July - pre-3.26 release, multi-cpu working fine, memory leak over the weekend.
2. 13/July - pre-3.26 release, with some memory fixes, still has the issue.

I think the release I got then and the one that was on the mikrotik page a day later are different, so I'm going to re-downgrade / upgrade again to v3.26 and see how I go tonight.

Re: SMP MultiCore fix update

Posted: Wed Jul 15, 2009 9:54 am
by macgaiver
You can just upgrade - from 3.26 to 3.26final - have done similar several times.

Re: SMP MultiCore fix update

Posted: Fri Jul 17, 2009 1:15 am
by omega-00
Ok, upgraded again and the newest one still has the same issue.
(made sure I had the right one, checked the hash etc to make sure it was what was listed on the MT download page)

But yeah, still has a memory leak, sent another supout.rif through.

Re: SMP MultiCore fix update

Posted: Fri Jul 17, 2009 1:21 am
by timatics
We Have been Running 3.26 on or AMD athlon X2 box for about 32Hours. without a Full Crash.
However the All of my hotspot users stopped working. All DNS requests stop working.
the Solution was to disable and re-enable all the Hotspots on the Box.

So Hotspot on 3.26 is still not quite fixed yet. but it did not deadlock the box.

Re: SMP MultiCore fix update

Posted: Fri Jul 17, 2009 2:55 am
by omega-00
However the All of my hotspot users stopped working. All DNS requests stop working.
the Solution was to disable and re-enable all the Hotspots on the Box.
One thing I still notice is that when I reboot a box with multiple hotspots, sometimes not all the static dns entries appear.
You'll need to disable and re-enable the relevant hotspots for them to appear again.

Just make sure you check this when/if you reboot etc.. I've reported the issue to MT as it's been around since v3.23 at least

Re: SMP MultiCore fix update

Posted: Fri Jul 17, 2009 5:36 am
by Wavelan
Still Not stable as 3.26 X86 Final Release.

We had router restarting with multi-cpu enabled.

This router only used as l2tp concentrator and one rule of mangle (TCP SYN),
using ppp static secret entries.

We had send the supout to mikrotik.

Got : [Ticket#2009071766000085]

Hw Configuration :

- Intel Serverboard S3210SHLC
- Intel Quadcore Q9550
- 2x2GB DDR2 6400 Kingston
- Acbel TruePower 607W
- 2 Onboard Intel Gigabit Interfaces
- 2 Dualport Intel PCI-E Ethernet

Uldis,Janis,Normis.
Please Help.

Thx
//Andre

Re: SMP MultiCore fix update

Posted: Tue Jul 21, 2009 9:44 pm
by timatics
v3.27 seems to be stable with hotspot, Vlans, and simple queues. i now have a up time over four days. my dns problem in 3.26 was resolved in 3.27.

This is my AthlonX2 box. it had crashed within 24 hour on version 2.25.

IF this box lasts a few weeks i will upgrade multi-core boxes. I have about 10 more in the field running 2.9.X, waiting for a stable version 3.

Re: SMP MultiCore fix update

Posted: Wed Jul 22, 2009 10:09 am
by macgaiver
All 3.x was stable on multi-core, only it was necessary to "/system hardware set multi-cpu=no" :)

Re: SMP MultiCore fix update

Posted: Wed Jul 22, 2009 4:19 pm
by timatics
All 3.x was stable on multi-core, only it was necessary to "/system hardware set multi-cpu=no" :)
Why would you Post that. That kind of attitude is why it took over a F@#$%$g year to find the Problem with hotspot, if you had been Following this thread you would know that setting "multi-cpu=no" did not fix the Crashing.


I am up to Five Days on 3.27, i have a not seen a problem.

Re: SMP MultiCore fix update

Posted: Fri Jul 24, 2009 8:10 am
by macgaiver
IF this box lasts a few weeks i will upgrade multi-core boxes. I have about 10 more in the field running 2.9.X, waiting for a stable version 3.
Why would you Post that. That kind of attitude is why it took over a F@#$%$g year to find the Problem with hotspot

And did staying with 2.9.51 really allowed you to debug version 3.x properly? :). So if you ask me - "mine attitude" is way better than yours.

Re: SMP MultiCore fix update

Posted: Thu Jul 30, 2009 9:09 am
by Wavelan
All 3.x was stable on multi-core, only it was necessary to "/system hardware set multi-cpu=no" :)
Why would you Post that. That kind of attitude is why it took over a F@#$%$g year to find the Problem with hotspot, if you had been Following this thread you would know that setting "multi-cpu=no" did not fix the Crashing.


I am up to Five Days on 3.27, i have a not seen a problem.
It's still crashed on 3.27 after 1 day and 16 hours.

We've report to mikrotik and put new supout for them, but so far no reply.

WE're only use that machine for queue.

Got another problem ticket.. Hope they respond for this.

Thx
//Andre

Re: SMP MultiCore fix update

Posted: Tue Aug 04, 2009 10:41 am
by route
CPU : Dual CPU AMD QUADCORE (8CORE)

reboots at couple hours.

Re: SMP MultiCore fix update

Posted: Mon Sep 14, 2009 7:13 pm
by hcidc
Please under attention this question
:lol:
http://forum.mikrotik.com/viewtopic.php?f=2&t=34845