Why would anyone want to do this? I am sure that other people can come up with their own good reasons for doing this, but in my particular case, I have been using MetaROUTER for some time to run third-party software alongside RouterOS on the same device, and I have unfortunately been fighting tons of MetaROUTER and RouterOS bugs and instabilities lately that have been frustrating both to me as well as to our clients, and at the present time, there does not seem to be any hope for a fix in sight, which leaves me in a very bad position. I know that OpenWRT has been ported to many RouterBoards and that I could just completely delete RouterOS and exclusively run OpenWRT on one if I chose to, and in a few of the cases where I have been having problems, I probably will end up going to 2 devices: one running RouterOS, and one running OpenWRT that performs the tasks that I normally run inside of a MetaROUTER. But I guess I must be an optimist, because I am hoping that this will be a temporary situation and that MikroTik developers will eventually get their act together on MetaROUTER. When that day comes, I would like to be able to boot RouterOS back up and fire up OpenWRT inside of MetaROUTER as if nothing had happened, without going through the pain of reinstalling RouterOS, re-loading my MetaROUTER image(s), and restoring the configuration on both. Being able to do this will also give me a good way to test and verify that problems I am experiencing are able to be blamed on MetaROUTER and not on OpenWRT itself. (There is also no reason why you couldn't install two copies of OpenWRT side-by-side on a RouterBoard and switch between them, just like you currently can with multiple versions of RouterOS.)
Essentially what I have done is built a single Linux kernel that is aware of MikroTik's in-memory NAND partitions, as well as a single kernel binary that can boot both inside of MetaROUTER as well as bare-metal on a RouterBoard, just like MikroTik's own kernel. So you can literally run the same exact OpenWRT install bit-for-bit either inside of MetaROUTER or directly on a RouterBoard.
Details and disclaimers about my kernel and OpenWRT image: it is based on an old OpenWRT version (Kamikaze 8.09.2) because that is what I initially based my custom MetaROUTER images on, and so far I have had no need or reason to change it because it has been very solid for me. Eventually I do plan on moving to a more current version of stable OpenWRT, but for right now, I am trying to change as few variables as possible and so avoid running the risk that updates to either my userland or my build environment/toolchain will introduce new bugs or regressions (the devil I know instead of the multiple devils I don't). And rather than try to shoehorn a Linux 3.x kernel onto Kamikaze, I elected to take MikroTik's own patches to 2.6.35 that they wrote for RouterOS 5.x, integrate them into the Kamikaze build, and then backport the various MikroTik drivers from their 3.3.5 kernel patch set for RouterOS 6.x. This gives me support for recent RouterBoard hardware without me needing to change anything in my custom image except for the kernel itself.
There are a couple of quirks to be aware of:
- I have really only been testing this so far on RB450G, and the only builds I have done so far are for MIPSBE. In theory, this kernel can probably run on other RB400/700/900 RouterBoards just fine (and it will certainly work inside of MetaROUTER on all MIPSBE RouterBoards), and in theory I can probably do basically the same thing on PowerPC, but none of this has been done or tested yet outside of 450G support.
- Hardware ethernet support when directly booting this build on a RB450G does work, but you will only see 2 ethernet interfaces: eth0 and eth1. eth0 corresponds to ether1, and eth1 is connected to the remaining 4 ports (ether2-ether5), which are switched together without any VLANs by default. I believe there is a utility included with OpenWRT that can be used to configure the switch chip, but I have not tested or played with it yet.
- I built and included the drivers for other hardware (such as, for example, USB support), but they have not received any testing yet and may not currently be in a working state. Also, ethernet ports on RouterBoards other than the 450G may not work, and no wireless support has been included yet.
- When it comes to NAND partitions in RouterBOOT, there is one proprietary MikroTik "feature" that I haven't been able to tame yet: by default, RouterBOOT employs some kind of "dead man switch" that RouterOS has to toggle in order to indicate to RouterBOOT that it has successfully booted. If it doesn't toggle it, RouterBOOT assumes that something is wrong, and as a failover measure, the next time the RouterBoard boots up, RouterBOOT will follow whatever it is configured to do, and the default action is to change the active partition to the next one in line. Since my kernel doesn't know how to tell RouterBOOT that it has booted up just fine, when you shut down or reboot the RouterBoard, it will boot up RouterOS next time instead of OpenWRT, even when there is no problem that prevented OpenWRT from booting. This can be worked around, but doing so effectively disables this partition failover protection feature.
- http://www.nconx.com/~nathan/openwrt-rb ... rootfs.tgz (for booting from NAND or MetaROUTER)
- http://www.nconx.com/~nathan/openwrt-rb ... tramfs.elf (for netbooting)
Do note that this is image is only provided for demonstration purposes! I will most likely use this as the future base for my Asterisk MetaROUTER images, but I have not built any OpenWRT packages for any apps, nor do I plan to run an OpenWRT package repository. If you want a specific application, you will either have to compile it yourself, or wait for somebody else to build packages and set up a repository.
I will try to carve out some time to write up some step-by-step installation instructions later, but basically what you do is configure one or more new partitions, netboot the INITRAMFS kernel on the device you want to install to, mount the new, second Main partition, download and extract the tarball to that partition over the network, mount the corresponding Boot partition, and copy the kernel to it. After that, you should be able to boot either OpenWRT or RouterOS by setting whichever partition you want to boot as the active one in either RouterBOOT or RouterOS (cannot be done from within OpenWRT at this time).
Since the exact same kernel binary can boot just fine either from RouterBOOT or from within MetaROUTER (and has all of the necessary driver and filesystem support for both environments), another trick you can accomplish is to install my OpenWRT build to a partition, mount that partition within RouterOS, and boot your OpenWRT partition within a MetaROUTER instead of importing the OpenWRT tarball...so, kind of like VMware Fusion's support for directly booting your Windows Boot Camp partition rather than booting Windows from a VMDK image file. I have done this and it works great, but I will probably have to leave figuring out the exact steps for making this work as an exercise for the reader, and it is DEFINITELY not supported by MikroTik (no way to accomplish this using standard commands from RouterOS shell or Winbox). Basically, you will need to modify the main RouterOS system package's read-only squashfs image so that the startup scripts mount your OpenWRT partition to the right location when RouterOS boots up. In theory, there is no reason why this exact same procedure couldn't be applied to an external disk, which would allow you to run and store MetaROUTERs on an SD card, for example, which is something that MikroTik themselves has sadly made no move to support (even though it was mistakenly advertised in early promotional materials). If you do this, you will have to re-do your modifications after every RouterOS upgrade.
RouterBoards, RouterOS, MetaROUTER, and the RouterBOOT bootloader are all very powerful and very flexible platforms and tools, especially when considered together; in fact, because these are basically general-purpose computers/SoCs running Linux, they are much more flexible than most people probably realize or give them credit for. This is a rough technical proof-of-concept, but if the value and utility of these kinds of features is recognized, then perhaps someday we will see some of these features become officially supported. Hey, a guy can dream, right?
-- Nathan