Community discussions

MikroTik App
 
keefe007
Member Candidate
Member Candidate
Topic Author
Posts: 125
Joined: Sun Jun 25, 2006 3:01 am

Interface Stats and CPU Usage not correct after upgrade

Wed Nov 12, 2008 6:37 am

I recently upgraded one of my routers from version 3.3 to version 3.16. Now with 3.16 I noticed that the Tx and RX rates for interfaces are not displaying correctly in Winbox. For example, Winbox is showing 10.5 Mbps TX on my LAN interface but I know that there's really about 30 Mbps on that interface. Also, the CPU usage in Winbox is showing 0% and I know that's not correct.

The interesting thing is that I'm graphing this router with Cacti and Cacti is showing the correct stats. Is this a bug?

Keefe
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 26379
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: Interface Stats and CPU Usage not correct after upgrade

Wed Nov 12, 2008 9:55 am

how do you "know" the real traffic amount ;) ?
 
keefe007
Member Candidate
Member Candidate
Topic Author
Posts: 125
Joined: Sun Jun 25, 2006 3:01 am

Re: Interface Stats and CPU Usage not correct after upgrade

Wed Nov 12, 2008 10:23 am

I know the real traffic amount based on the interface statistics on the devices on either side of my router.

I don't have an concrete proof that the CPU is wrong, however, I find it highly unlikely that I'm using 0% ;)

Torch is also showing the wrong stats...at least at the bottom where it adds up the total traffic. I haven't had time to actually drill down a single stream to see if that's correct or not.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8709
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Interface Stats and CPU Usage not correct after upgrade

Wed Nov 12, 2008 1:16 pm

what about clock?

do you use multi-cpu=yes?
 
keefe007
Member Candidate
Member Candidate
Topic Author
Posts: 125
Joined: Sun Jun 25, 2006 3:01 am

Re: Interface Stats and CPU Usage not correct after upgrade

Wed Nov 12, 2008 11:10 pm

The CPU frequency and CPU count is correct.

And I do have multi-cpu=yes.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8709
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Interface Stats and CPU Usage not correct after upgrade

Wed Nov 12, 2008 11:13 pm

try multi-cpu=no and see whether something change
 
keefe007
Member Candidate
Member Candidate
Topic Author
Posts: 125
Joined: Sun Jun 25, 2006 3:01 am

Re: Interface Stats and CPU Usage not correct after upgrade

Wed Nov 12, 2008 11:22 pm

Well I dont want to do that because I utilize dual-core...otherwise BGP convergence takes 100% CPU.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8709
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Interface Stats and CPU Usage not correct after upgrade

Wed Nov 12, 2008 11:26 pm

only for test... just to know =)
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 26379
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: Interface Stats and CPU Usage not correct after upgrade

Thu Nov 13, 2008 9:56 am

Well I dont want to do that because I utilize dual-core...otherwise BGP convergence takes 100% CPU.
multi-core is not stable yet, use single core. the speed gain is minimal anyway
 
analog
newbie
Posts: 41
Joined: Wed Oct 11, 2006 10:18 pm

Re: Interface Stats and CPU Usage not correct after upgrade

Thu Nov 13, 2008 11:24 pm

multi-core is not stable yet, use single core. the speed gain is minimal anyway
What is this?! We've had multi-core since version 3. You are saying speed gain is minimal going from a single core to a quad core?

Can you give more details on what is wrong with multi-core support? This is a major problem.

My new quad core router just crashed for no reason and was displaying wrong stats. Not very good!
 
analog
newbie
Posts: 41
Joined: Wed Oct 11, 2006 10:18 pm

Re: Interface Stats and CPU Usage not correct after upgrade

Fri Nov 14, 2008 12:11 am

Just a note on the "minimal speed gain". With multi-cpu enabled, my CPU usage is about 7-10%. Once I turned it off, I now have 30-35% usage. This is with about 400 PPPoE sessions and about 20Mbps traffic usage. That is not a minimal impact!
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 26379
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: Interface Stats and CPU Usage not correct after upgrade

Fri Nov 14, 2008 10:17 am

do you realize that CPU usage doesn't reflect SPEED, just available recources?

multi-core has not been stable yet, don't use it if it causes problems. on certain systems it works well, but we are still working on it (actually, it more depends on Kernel developers).
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8709
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: Interface Stats and CPU Usage not correct after upgrade

Fri Nov 14, 2008 11:59 am

Just a note on the "minimal speed gain". With multi-cpu enabled, my CPU usage is about 7-10%. Once I turned it off, I now have 30-35% usage. This is with about 400 PPPoE sessions and about 20Mbps traffic usage. That is not a minimal impact!
just notice: in both cases your router works equally, there's more than 50% free CPU time ;)
 
analog
newbie
Posts: 41
Joined: Wed Oct 11, 2006 10:18 pm

Re: Interface Stats and CPU Usage not correct after upgrade

Sat Nov 15, 2008 3:17 am

normis, thank you for taking the time to reply to this topic. Response from Mikrotik support staff is very much appreciated. This tells us you are listening to your customers, which seems rare in this industry.

I agree with you on the speed point. I did not describe my complaint accurately. The router is currently working just as fast as before (or maybe minimally slower as you described. I cannot perceive a difference).

However, there is significant reduced capacity. And, the whole reason for building this sized router was anticipated growth. I expect 800 PPPoE sessions (up from 400) within 2 months time. Also, I've learned to never run a router/server at even close to 100% capacity if you want reliability.

The biggest issue is that I can find no where in the documentation that mutli-cpu is unstable. Please direct me to the documentation if I have just overlooked it. If this feature is unstable, why is it enabled by default? Why is the feature even included in a "stable" release of RouterOS? I can understand an unstable feature included in a beta release, but you would never find me running a beta release on my core router!

Also, my router crashed late last night even though multi-cpu is off. Traffic was still passing, but no PPPoE sessions could negotiate, all SNMP requests were ignored, local queue graphs went blank. I could not winbox nor SSH into the server. I had to power cycle it to regain access. So, something is still up. The router I replaced with this server (an AMD Athlon), ran perfect for more than 2 years on multiple versions of ROS. The latest being 3.13. I am running 3.16 on my new router.
 
changeip
Forum Guru
Forum Guru
Posts: 3830
Joined: Fri May 28, 2004 5:22 pm

Re: Interface Stats and CPU Usage not correct after upgrade

Sat Nov 15, 2008 10:17 am

Also, my router crashed late last night even though multi-cpu is off. Traffic was still passing, but no PPPoE sessions could negotiate, all SNMP requests were ignored, local queue graphs went blank. I could not winbox nor SSH into the server. I had to power cycle it to regain access.
\

sounds like the memory leak I'm trying to figure out on 2.9.51. once the router is over 380mb memory it will stop responding to management. once its a little higher it will crash.
 
analog
newbie
Posts: 41
Joined: Wed Oct 11, 2006 10:18 pm

Re: Interface Stats and CPU Usage not correct after upgrade

Sat Nov 15, 2008 8:33 pm

The CPU, Memory and Disk Usage graphs did not stop recording during the event. And, it shows my max memory usage at only 56MB. I have 2G in the server. It could be the graph is inaccurate, but it does not appear my memory usage is to blame. My average usage is 47MB, so the high of 56MB does not seem to be abnormal.

I'd just like to point out that ~50MB of memory usage is quite nice considering the load on this router. Very efficient use of that resource!
 
User avatar
GWISA-Kroonstad
Member Candidate
Member Candidate
Posts: 111
Joined: Fri Nov 10, 2006 3:34 pm

Re: Interface Stats and CPU Usage not correct after upgrade

Mon Nov 17, 2008 9:57 am

Same issue here. Had always had problems with faulty speed reporting on interfaces in Winbox. Yet, by upgrading from 3.14 to 3.16 got much worse. Now, every time we reboot the PC-based MT, a different range of speed reporting occurs. At first upgrade, we got very high speed results such as 8Mbps on 1 Mbps lines. By rebooting, now we get only 30-60 kbps reporting. And so on with every reboot.

Actually, faulty reporting is affecting the queues for QoS purposes. This means real speeds are not only displayed improperly, but also perceived improperly by MT.

MT guys, you might have a bug here. How can queues work properly when the speed is not interpreted correctly?
 
User avatar
GWISA-Kroonstad
Member Candidate
Member Candidate
Posts: 111
Joined: Fri Nov 10, 2006 3:34 pm

Re: Interface Stats and CPU Usage not correct after upgrade

Mon Nov 17, 2008 10:08 am

Seems setting multi-cpu=no solved the problem. Thx.

Though a pity that MT has not mastered the multiple cpu technology yet.
 
analog
newbie
Posts: 41
Joined: Wed Oct 11, 2006 10:18 pm

Re: Interface Stats and CPU Usage not correct after upgrade

Mon Nov 17, 2008 6:17 pm

Over the weekend, my new pc based router has taken it upon itself to reboot at random times. It happend once each day on Saturday and Sunday. I expect the same thing again this afternoon. I log to a remote syslog server, and there is nothing in the logs that would point to why this is happening. No abnormal CPU usage, nor memory usage, nor traffic. I'll gather details and send in a support ticket later this morning and post here with results, if any.
 
User avatar
gmsmstr
Trainer
Trainer
Posts: 982
Joined: Fri Jun 04, 2004 2:22 am
Location: St. Louis, MO
Contact:

Re: Interface Stats and CPU Usage not correct after upgrade

Mon Nov 17, 2008 11:21 pm

from v3.6 to v3.11 we have had good luck with Multi-core, after that, hard power cycles are required. Sometimes it runs for days, some times weeks, but it will require a power cycle eventually. We have not been recommending more than 3.12 for a while.

Of course, I am with everyone else, come on guys. Just cause the Routerboard products do not have mutli-core does not mean your client base does not need multi-core CPU systems! When you start running 3 GigE BGP sessions, full routing tables, 8000 clients behind the router, with over 50,000 pps and avg of 200-300 meg of traffic, we need to have multi-core, as single core systems just can't keep up!

Let us know!
 
analog
newbie
Posts: 41
Joined: Wed Oct 11, 2006 10:18 pm

Re: Interface Stats and CPU Usage not correct after upgrade

Tue Nov 18, 2008 7:59 pm

so far, no response from support. My router rebooted itself twice last night, so I've had to pull it out of production. I'm very disappointed to say the least. I'm still hoping to get some guidance from MT on this problem. Normis, do you have any ideas?
 
User avatar
gmsmstr
Trainer
Trainer
Posts: 982
Joined: Fri Jun 04, 2004 2:22 am
Location: St. Louis, MO
Contact:

Re: Interface Stats and CPU Usage not correct after upgrade

Tue Nov 18, 2008 8:57 pm

We have been having good luck with v3.11 even with multi-core on. Have one PowerRouter 732 running dual cores moving peeks of 70 meg without issues, around 30-40% CPU load, and we are doing some traffic shaping.
 
analog
newbie
Posts: 41
Joined: Wed Oct 11, 2006 10:18 pm

Re: Interface Stats and CPU Usage not correct after upgrade

Wed Dec 10, 2008 10:39 pm

Still no response to my ticket. Almost a month now This is the first time in 3 years of usage that I've had to request support from MT, and quite frankly, I'm very disappointed. I purchased a Level 6 license for this server, and this is the support I get?!

Does anyone know of a way to simulate 400+ PPPoE sessions? I'm not very keen on putting this router back in as my core router until it proves itself to be reliable. I'm happy to downgrade to version 3.11 if that will fix it, but I'd like to test it before putting it back into production. Any ideas are appreciated.

On Edit:
I've had the router powered up here in my office, with no load of course, and it hasn't rebooted by itself at all in over 20 days. So, the hardware is fine.
 
dot-bot
Member Candidate
Member Candidate
Posts: 164
Joined: Tue Oct 11, 2005 7:05 pm

Re: Interface Stats and CPU Usage not correct after upgrade

Thu Dec 11, 2008 12:41 am

"Hardware is fine" and "with no load" do not fit into one sentence. You must STRESS that CPU and RAM. Last time I used Stress Prime 2004 Orthos. But that Router was a x86 PC and I was able to boot Win32 to run that test. Under MT ROS you should run a bandwidth test with UDP to stress NICs as well since they generate heat and need a lot of amperage to run in a typical big router.

With some Intel x86 dual-core systems I have found BIOSes to need flashing for updating.

Don't forget to post what you have tried as well as the results.
Last edited by dot-bot on Sat Aug 15, 2009 11:52 am, edited 1 time in total.
 
analog
newbie
Posts: 41
Joined: Wed Oct 11, 2006 10:18 pm

Re: Interface Stats and CPU Usage not correct after upgrade

Wed Feb 04, 2009 10:06 pm

"Hardware is fine" and "with no load" do not fit into one sentence. You must STRESS that CPU and RAM.
dot-bot, you are right. I stressed my system, ran some tests and found RAM to be throwing errors. MT, you have my apologies for blaming you for my router's instability. Once I put good RAM in, it has been stable for 30 days straight. I still have multi-core turned off though. Any word on it becoming stable?
 
User avatar
NetworkPro
Forum Guru
Forum Guru
Posts: 1376
Joined: Mon Jan 05, 2009 6:23 pm
Location: bit.ly/the-qos
Contact:

Re: Interface Stats and CPU Usage not correct after upgrade

Mon Feb 09, 2009 1:48 am

Good work mister dot-bot. 8)

To MikroTik most respected software developers: could we please request a better hardware testing tools, similar to what has been described by dot-bot? Thanks.

Who is online

Users browsing this forum: bertus, Bing [Bot], dervomsee, infabo, mgual, Mr47, pe1chl, pmcsill, unam83 and 121 guests