Page 1 of 1

RB4011 and RB1100 AHx4 "bricks" randomly

Posted: Mon Jun 03, 2019 1:18 pm
by marisspringis
Hi everyone,
since the last week of may, strange thing has happened to Router Boards which i manage.
issue - Router simply bricks, what i mean by that is - you cannot connect to router in any way (stays on logging in and nothing more happens), APs that are connected lose all config from dude, SNMP stops working and so on.
in the same time, from computers which are connected to switch, internet is working. also i can ping that router
this can be resolved only by hard reset (take power cable off/on)
issue has happened only on RB4011 and RB1100AHx4 Dude edition
Router Os - 6.44.3
previously this has never happened.
these RB are in different countries.

so far this has happened only once but with every RB4011 we have and one RB1100 AHx4
does anyone else has seen this?

Re: RB4011 and RB1100 AHx4 "bricks" randomly

Posted: Mon Jun 03, 2019 1:48 pm
by ccardenas
Hello! Do you have bridges in your network implementation? How many hosts are passing traffic among these "random bricking" devices? Could you provide us more info? So we can understand your problem and we'll be able to help you better.

We have a similar issue, and we are suspecting about the bridge host table size and (possible) memory exhaustion problem. It only happens in new arm devices (RB4011 and RB1100x4).

Symptoms are loss of connectivity, manageability and it's impossible to access the device in any way, but it keeps working as a switch. After a reboot (unplug power cable/ replug) all begin to work fine and we can see a lot log lines like: snmp, warning timeout while waiting for program XX (where xx is a variable two digit number)

Image

Regards

Re: RB4011 and RB1100 AHx4 "bricks" randomly

Posted: Mon Jun 03, 2019 2:43 pm
by marisspringis
Hi ccardenas,
yes, we have 3 bridges in RB4011 and in RB1100AHx4.
hosts in RB4011 most of the time are - 30-40
in RB1100 not more than 5 connected directly, it is used as dude server for monitoring.

one more thing to point that problem is in these RB4011 is that we have a lot of RB2011 with the same config, and they work perfectly, without any problems. also ROS version in all ar is the 6.44.3

symptoms are identical to yours.

Re: RB4011 and RB1100 AHx4 "bricks" randomly

Posted: Mon Jun 03, 2019 4:32 pm
by marcin21
Itr seems that cpu is getting exhausted over time.
this particular ARM based WAP60g is 149d up.

Re: RB4011 and RB1100 AHx4 "bricks" randomly

Posted: Mon Jun 03, 2019 5:45 pm
by ccardenas
one more thing to point that problem is in these RB4011 is that we have a lot of RB2011 with the same config, and they work perfectly, without any problems. also ROS version in all ar is the 6.44.3
Hello!! Yes, totally true. RB2011 and RB1100Hx2 in the same place, in the same network, in the same situation and nothing happens, they never block. We've opened a support ticket to Mikrotik and they told us to plug a serial cable and wait the device to block, then try to access it via console and make a supout, but we have a couple of them with cables attached and now they never block! :cry:

Other devices within the network keep blocking randomly. In the meanwhile we have scheduled a reboot (lame solution, but it saves the day) at nights a couple of times a week until we find the real problem, but it seems that some process inside the routerboard hangs or collapses the memory, making the another processes fall in cascade and block the access to the device.

If someone is experiencing the same problem, please share with us, maybe we can find a hint in the meanwhile, until I can get a good supout file and send to Mikrotik support.

Regards!!

Re: RB4011 and RB1100 AHx4 "bricks" randomly

Posted: Mon Jun 03, 2019 6:38 pm
by Dude2048
I have a RB1100ahx4 Dude edition which has the same behavior. What happens is that the memory hogs and the device will become inaccessible. I have a script that reboots the device when 70% is used. During the times that it is inaccessible I tried to make a supout, via console, but that didn't work.