RB5009 cannot enter the system after restart

Which implies that before it was not clear enough… :unamused: :laughing:

My understanding is/was that upgrading to 7.19 is without any risk (well, not more then usual…) as long as you didn’t poke around with these bootloader-images found in the articles mentioned.
So simple upgrades through Winbox from RouterOS + “Routerboard Firmware” towards 7.19 should be OK.
(…or did I just dodged a bullet here…?) :open_mouth:

I didn’t even do system/routerboard/settings/set protected-routerboot=enabled. I just wanted to upgrade the firmware to the latest. I’m used to using the latest software for my home devices, so I directly downloaded the firmware and uploaded it to the router to update it. :confused:

In all seriousness: that is correct.
Still some things need to be handled with extreme caution. Ask once, ask twice, ask again if needed.
The previous wording of that help page was not clear enough and this has been corrected (but to my liking it still is not alarming enough).

If so many users report a broken device, something must have been wrong with the instructions…
It´s not the users who should be blamed. Shooting back on users is easy, but not the way to make yourself popular.
Additionally there is something wrong with any commercial firmware, which has the potential to brick a device, but it just installs on the wrong device!

Ps. no, my RB5009 is still working :slight_smile:

@strods, the way you have it formatted here is much better than the way it is written on the help page. You might want to post this list as it shows here (with my edits) to replace the really long run-on sentence that precedes the instructions.

To address @normis’ response, I believe someone posted a link from either the 7.18.2 or 7.19 thread to the help page that talks about updating the backup firmware. Their post was written in such a way that made people think they needed it. I think that, combined with the way the instructions were originally written, got people in the mindset that they needed both to go past 7.18.2. I know I, as a native English speaker, was confused at first. (Remember, people don’t typically spend a lot of time reading before clicking. They scan the text swiftly, looking for links to click and buttons to push.)

Basically the way these backup (secondary) bootloaders work is that actually from the standpoint of the hardware, actually the backup bootloader is the one that is run on start. Then it does the following:

  1. Check if button is pressed (or the special flag for force-backup-bootloader is set) or I am actually the main bootloader
  2. If the above check is false, jump to normal (non-backup) bootloader
  3. Do everything/whatever it is the bootloader usually does, including booting the system, netintall, reset config, etc.

So basically for your device to function correctly, the backup bootloader only has to do steps 1 and 2 correctly. It is purposeful that the backup bootloader cannot be upgraded in the usual way, because any problem in the bootloader (except for step 1 and 2) can be rectified by updating the main one, and the backup bootloader is basically your chance to recover your device if everything happens to go wrong (e.g. the bootloader upgrade is corrupted, power is interrupted halfway, etc.)

Probably Mikrotik managed to provide an upgrade to the bootloader that actually fails in the first two steps. The LOL is deserved. But it still follows from the above: why mess with the backup bootloader as long as it is functional - it’s inherently risky.

I think that for those whose backup bootloader is borked, the only solution is to flash the 16mb flash chip with a known-good image using something like a bus pirate. Probably the license will have to reapplied.

http://forum.mikrotik.com/t/v7-18-2-stable-is-released/182200/605
It all started from here, but whatever happens the forum users are not responsible for the firmware/software in any way.

For the record: even my RB5009 where I did the tests (for 70x0 Armada), now that I tried to turn it on again, does not work anymore.

A simple sign like this would have probably helped :wink: .

But the base issue remains, it is the software that should check and NOT run unless ALL the conditions are met.
brick_warning.jpg

I do have the tools to do this, but it would be contingent on MikroTik providing a working image for me to do this.

Or any good soul who’s willing to dump theirs :slight_smile: And Mikrotik is kind enough to provide the original license to download if you’re willing to register to their customer portal.

I’d probably start by trying to get one from the openwrt guys. During their reverse engineering I’m sure they made a few images.

Good idea. Luckily my router is currently still running, so I was able to get serial number and software ID from it. I have my key in the MikroTik portal now.

I will see if I can find anyone that can provide a bootloader dump, unless MikroTik is willing to help in this case.

You are wrong. Bootloader stay at different chip and doesn’t affect licensing so no need to worry about rewrite.
I wish i had rb5009 but out of luck.

I advice you hire some PR consultant before you write any more posts. You are digging your grave here. You can put as many warnings as you like, this will never change the fact that you have provided signed firmware that bricks device 100% time and linked it in official documentation. The only proper way to address in disclaimer would be to write: “The above link is provided purely for educational purposes. We have not tested it on actual hardware and can not promise it will not destroy your device or kill your cats. All warranties are void if you follow these instructions!”. From the tone of your messages, I assume this is what you really meant by “DANGEROUS”? If so, you should likely make sure it is explicitly stated, both for your users as well for your legal protection.

On less aggressive note, I recommend re-wording current wiki again:

The protected RouterBOOT feature is supported by all modern MikroTik devices, but if you have and old device, if your factory-firmware version is lower than 7.18.2 and your device displays the message

should likely be:

The protected RouterBOOT feature is supported by all modern MikroTik devices, but if you have and old device and your device displays the message

The part about factory-firmware being lower than 7.18.2 serves only to confuse users.

OK, so it’s the weekend. I had some time on my hands.

I rebooted (i.e. killed) my still running but walking dead RB5009 for science. It sure was dead then.
Turns out, what that “universal” package did was universally erase the first 653295 bytes of the SPI NOR flash.

The good news is, the device specific information (mac addresses, serial numbers, etc. known as “hard_config” in the OpenWrt world) are intact.

The even better news is: I flashed back a known good ATF + Backup Booter and my 5009 is back and running. Config and all intact.
It still thinks it’s on the 7.18.2 “Factory Firmware” but the Backup Booter I wrote to the flash is from a device that has 7.0.5 as its Factory Firmware. I don’t really care. It’s late. It’s probably not the worst idea to netinstall the thing to get a clean start. I’ll maybe try that tomorrow.

For those daring enough:

  1. Acquire a SPI NOR flash dump from a working RB5009. If you don’t have another one around to serve as a donor check out https://forum.openwrt.org/t/add-support-for-mikrotik-rb5009ug/104391/32 for inspiration.
  2. Isolate only the needed parts from the full dump:
dd bs=1 count=$((0xaf000)) if=dump-from-router-not-ruined-by-the-universal-package.bin of=og-atf-and-bootloader.bin
  1. Write og-atf-and-bootloader.bin back to the flash chip. There are a few ways to do it. You can directly program the chip using a programmer. I was too lazy to unsolder the chip, so I spent way more hours with a custom OpenWrt build (booted via the SoCs UART recovery mode, courtesy of the mrvl_uart.sh script) that I modified so I could reprogram that section of the flash.
  2. Bob’s your mother’s brother.
  1. You are a amazing and thank you for being as forthcoming with a fix as you’ve been. Wish it had been Mikrotik but whatever.

  2. For those of us that are not computer wiz kids, would you entertain fixing mine if I paid for round trip shipping and some money for your time? OR could you put together some idiot proof instructions as well as the required hardware? I sure would love to have my RB5009 back in action. My CCR2004..+PC works just fine but I truly love the 5009.

Wow, that’s a very cool feature in the Marvell! I wish all SoCs had ability to take a boot payload via UART! It would make fixing bricked devices across the board so much easier!

You mentioned that the first 600kB of your NOR flash was wiped. I can see from the partition layout diagram on OpenWRT’s website that the first partition on NOR is 600kB, and is the ARM trusted firmware. So is the bug here that the MikroTik factory firmware upgrader is writing to the wrong NOR partition on this particular model (it should be writing to mtd2 but wrote to mtd1 instead)?

It makes me wonder: when recovery-booting via UART, could you have simply substituted in a valid copy of backup RouterBOOT in place of your custom OpenWRT build in order to at least boot the device “normally” (RouterOS from NAND), then? Obviously this would not get you closer to fixing the contents of the NOR so that you did not have to depend on UART-booting every time (unless you followed that with a RouterOS jailbreak…hmm…)

What did you need to change in your OpenWRT build?

How do you pass both the TF + the boot payload to the UART recovery booter? Do you just concatenate them together? Did you just take the copy of the ATF from the “good” 5009 NOR dump, and that worked to boot your custom OWRT image?

In any case, well done in discovering that this was possible, and coming up with a working implementation.

I just have to ask. I have a RB5009 with 7.18.2, Factory firmware 7.6, current firmware 7.16. I never uploaded anything else than official packages on it. Can i update to 7.19 safely or not. It’s my main route to the net, so it would be good if it doesn’t get bricked.

The issue is not with the new RoS, it is with the protected bootloader update, just DO NOT attempt to install the latter.

Turns out, what that “universal” package did was universally erase the first 653295 bytes of the SPI NOR flash.

That’s pretty bad and confirms what I feared and voiced in other thread. This could’ve well happen with normal fwf upgrade distributed alongside ROS itself (let’s hope they actually test those!). It seems they absolutely lack any bound checks or sanity checks when programming firmware and just bang bits directly. Ouch.