Community discussions

MikroTik App
 
User avatar
Hammy
Forum Veteran
Forum Veteran
Topic Author
Posts: 776
Joined: Fri May 28, 2004 5:53 pm
Location: DeKalb, IL
Contact:

6.37.1 and 6.36.4 100% CPU usage on Firewall

Fri Nov 11, 2016 5:46 am

I have a Xeon box that was humming along today, then became unresponsive. I drive to the data center and the firewall is using up all that the CPU has to give. Normally CPU usage is between 0% and 3% when under "load". The upstream interface now is using... 3 megs. I disconnect the upstream cable and it goes away. I reconnect and it comes back. I can't get a supout while the box is loaded. Can't seem to get one to fire even when the box isn't loaded.

I went from 6.37.1 down to 6.36.4 with no change and then up to 6.38rc25 and the problem went away. I don't like rocking an RC on my PE router, but I also don't like downgrading just to see if the problem is still there.

I added firewall rules to only accept DNS, Winbox and SSH, drop all else. No difference in the operation of the box.

What was going on? I can provide a 20M PCAP to qualified parties. I should have taken a much larger one.
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1764
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: 6.37.1 and 6.36.4 100% CPU usage on Firewall

Fri Nov 11, 2016 2:31 pm

this is guesswork at this point. in changelog there are at least few entries that can explain that. usually 100% is down to some kind of crash.
 
User avatar
Hammy
Forum Veteran
Forum Veteran
Topic Author
Posts: 776
Joined: Fri May 28, 2004 5:53 pm
Location: DeKalb, IL
Contact:

Re: 6.37.1 and 6.36.4 100% CPU usage on Firewall

Fri Nov 11, 2016 2:38 pm

this is guesswork at this point. in changelog there are at least few entries that can explain that. usually 100% is down to some kind of crash.
But then why would unplugging\plugging the upstream interface make a difference?
 
R1CH
Forum Guru
Forum Guru
Posts: 1101
Joined: Sun Oct 01, 2006 11:44 pm

Re: 6.37.1 and 6.36.4 100% CPU usage on Firewall

Fri Nov 11, 2016 6:39 pm

Are you sure you aren't receiving excessive traffic (potential DDOS) from your upstream?
 
User avatar
Hammy
Forum Veteran
Forum Veteran
Topic Author
Posts: 776
Joined: Fri May 28, 2004 5:53 pm
Location: DeKalb, IL
Contact:

Re: 6.37.1 and 6.36.4 100% CPU usage on Firewall

Fri Nov 11, 2016 6:42 pm

Are you sure you aren't receiving excessive traffic (potential DDOS) from your upstream?

My upstream interface was only showing about 3 megabit/s peak. It certainly wasn't a volumetric attack, but that doesn't rule out other kinds of attacks.

Also, it's an 8 core Xeon box. I only have 1 gigabit of upstream, so I'm not sure I could max out the CPU under regular or volumetric conditions anyway.
 
jarda
Forum Guru
Forum Guru
Posts: 7756
Joined: Mon Oct 22, 2012 4:46 pm

Re: 6.37.1 and 6.36.4 100% CPU usage on Firewall

Fri Nov 11, 2016 7:12 pm

Your badly organised firewall receiving huge amount of small packets can easily flood the cpu. See the profiler, watch the torch and you will get a clue... You have to know what kind of traffic is the reason in order to be able fight with it.
 
User avatar
Hammy
Forum Veteran
Forum Veteran
Topic Author
Posts: 776
Joined: Fri May 28, 2004 5:53 pm
Location: DeKalb, IL
Contact:

Re: 6.37.1 and 6.36.4 100% CPU usage on Firewall

Fri Nov 11, 2016 9:48 pm

Your badly organised firewall receiving huge amount of small packets can easily flood the cpu. See the profiler, watch the torch and you will get a clue... You have to know what kind of traffic is the reason in order to be able fight with it.
I did use Profiler. I used Torch to some degree. I did a packet capture for later analysis.

3 megabit of traffic can topple eight Xeon cores?
 
jarda
Forum Guru
Forum Guru
Posts: 7756
Joined: Mon Oct 22, 2012 4:46 pm

Re: 6.37.1 and 6.36.4 100% CPU usage on Firewall

Sat Nov 12, 2016 9:01 pm

Should not. So,what is the result?

Who is online

Users browsing this forum: GoogleOther [Bot], raiser and 90 guests