Serious bug casuing network DDOS in RouterOS v5.20 (and maybe others-didn't tested yet).

karwos · Sun Jul 05, 2015 5:53 pm

I've encountered several DDOS attacks in my network. The more interesting fact was that attacking device was Mikrotik RouterOS itself. I've successfully reproduced problem and confirmed that causative device was MT router....

Configuration:
RouterOS v5.20 on x86 platform,
around ten gigabit ethernet interfaces, lots of VLAN, running HotSpot

How to reproduce problem?
1. Configure Hotspot system
2. Go to Hotspot IP Bindings and add some static IP (of course in same subnets of RouterOS device) and very important - set static MAC address and type: bypassed. You can connect some working device (like other router) and set it's IP and MAC.
3. Now, go to Tools->Ping and PING previously added IP. It should respond and nothing spectacular should happen at this point.
4. Now - disconnect/poweroff this device, so it shouldn't be reachable by RoutetOS. Wait around 2 or 3 minutes, and retry pinging. As RouterOS couldn't reach this device, for some reason IT WILL BE FLOODING all interfaces - including ethernets, vlans etc with _BROADCAST MAC_ ADDRESS, and in response all devices will be flooding RouterOS itself with icmp type8 (echo-ping) in response to broadcast packet. This causing lots of problems-overloaded devices, slow network performance, and in some cases also server overload)

Same problem might be reproduced not with ping, but also with TCP connection... for example if you will try to connect to non-reachable IP added at IP binding to ssh or www port, server will flood all ports with SYN packets, and all devices will respond to broadcast-syn packets...

Workaround is to not use MAC address in IP binding, however i think problem should be investigated by MT engineers.

Best regards.

karwos · Mon Sep 21, 2015 11:56 pm

bump ...................... c'mon guys, if you are really too lazy to check this, get me access to kernel and OS and i will debug and find out this for you ...

geosoft1 · Tue Sep 22, 2015 9:01 pm

you must forgot Mikrotik support. i also found and send a few bugs and everytime they try to convince me that what i saw is not true. basically, Mikrotik are good tools but on the software part remember that you are on your own.

sorry Mikrotik support, but until now you never do something to change my opinion

Tue Sep 22, 2015 9:29 pm

It depends. My experience is that mikrotik solved so far all real bugs I have reported to them during the years. And they were not just few. They are willing to it, but it is very hard to reproduce the error sometimes and they needed several times repetitively to access my routers to see it. Once they was even directly making the corrections on my device and finetuning the development version on the go. I have never paid for it nor I have any agreement with mikrotik. Maybe they are discussing sometimes a bit around but when they see the real bug they try to solve it. Sooner or later. Sometimes it takes really long time, but that's the life...

angboontiong · Wed Sep 23, 2015 5:35 am

When this thing happened, only way to do with it is replace the routerboard.

And, useless talk to Mikrotik cause they don't know what happened on the routerboard.

Every year we'll happened few cause, just replaced it, then your problem solved.

Sent from my Che2-L11

geosoft1 · Wed Sep 23, 2015 10:16 am

jarda, depend of what they consider real bugs. i always send the steps to reproduce the bug and i spend a lot of time trying to explain to those slow guys that things was happen. also i found some misconceptions but they consider that are normal things. for me this is a lost cause.

Wed Sep 23, 2015 4:42 pm

Sure. Can be so. But it is not just pure black/white case. My experience also says that it is not so quick, easy and straightforward like it could be. Maybe your topics are much more complicated than mine. Maybe they could be partially right or didn't catch what you were trying to explain... Misunderstanding Can happen very easily. Sometimes you have to be very patient with them. They are just people like you or me so don't expect too much from them.

MrYan · Wed Sep 23, 2015 5:02 pm

User forum != support@mikrotik.com - did you open a ticket with them?

geosoft1 · Wed Sep 23, 2015 7:25 pm

no reason to have patience with they. personally, i handle well routeros to do almost all i need. when i reported a bug or a misconception i do it to help them to correct. not necessarily for me because i already correct that. but everytime the response was wrong or evasive or even what i saw is not true and i have many mails and tickets in that way. every time when i respond to a mail i talked with other consultant which resumed the problem from the beginning... and so on. i already said them that i will never report bugs anymore.
the last problem was the famous default ddos in home routers. unimportant of course. you remain without internet service, ok, who cares... initial is not true, ok, ok is true but is not a bug... ohh, maybe is a feature? of course are solutions... c'mon, next update has interface corrections...
no priority, no project management, no support... no way.

pe1chl · Thu Sep 24, 2015 11:03 am

Also I would recommend in any case to reproduce it with the latest version.
Most companies are a lot more reluctant to come into action when you report a software bug in an older version
and have not confirmed it still exists in the actually supported version.
In fact, when I recently ran into a bug and was ready to report it I thought "let's first try to update so I can mention
the latest release" and the problem was gone. Although nothing was written about it in the release notes...

Thu Sep 24, 2015 11:37 pm

no reason to have patience with they. personally, i handle well routeros to do almost all i need. when i reported a bug or a misconception i do it to help them to correct. not necessarily for me because i already correct that. but everytime the response was wrong or evasive or even what i saw is not true and i have many mails and tickets in that way. every time when i respond to a mail i talked with other consultant which resumed the problem from the beginning... and so on. i already said them that i will never report bugs anymore.
the last problem was the famous default ddos in home routers. unimportant of course. you remain without internet service, ok, who cares... initial is not true, ok, ok is true but is not a bug... ohh, maybe is a feature? of course are solutions... c'mon, next update has interface corrections...
no priority, no project management, no support... no way.

Everyone can have different expectations and experience with the support. Mine are almost contradictory to yours. I also think that your firewall settings is your responsibility. Not the responsibility of the device manufacturer.

geosoft1 · Fri Oct 23, 2015 11:40 am

Mine are almost contradictory to yours. I also think that your firewall settings is your responsibility.

are different things. we talk here about the home class of equipments (and Mikrotik says clear that some routers are designated for home use) which must had special firewall rules because home user don't have technical knowledge and is the manufacturer responsibility to provide a minimal protection against well known attacks.
any other network equipments manufacturer do this. am i right?
it's inadmisible to buy a home router and next day to have internet service suspended for abuse.
if you don't do a minimal protection you must inform the user through the user manual that you are responsible to write a firewall and a short description of what you must do in that way.
if we are talking about professional equipments, there is another story and the firewall must be created by users and is not the manufacturer responsibility but it seems that Mikrotik entered in the home segment of the market treating things as in professional segment.

anyway, starting with version 6.32.3 the problem was solved because that's what had to be.

What's new in 6.32.3 (2015-Oct-19 11:13):
*) quickset - create proper firewall rules when PPPoE is used for address acquisition;

Thu Nov 05, 2015 8:03 am

Ok. It can be. But go back, read the first post and say if someone who installed ros on x86 is unresponsive home user that needs to be protected against the lack of his knowledge. And anyway the core of my answer was about my experience with support, not about the original problem described here.