Device-mode changes hit or miss? Mikrotik strategy?

Yes, I prefer enter a secure key or something else than press a button to switch devices mode.

As I already suggested, wouldn’t it be enough to use the factory password for security?

for new devices with a default one OK…

As dang21000 points out, not all devices have default factory password.

For the ones that do, if they have already been deployed and the factory password was changed, and we didn't keep a record of the factory password...then what?

I guess the crux of my complaint is I just want to set device-mode via netinstall (or even flashfig or new tool), so that there can be an automated way to setup a new/factory router with our custom config. The device-mode around container has already BLOCKED adding a container as part of default configuration which has been a long-standing grip even before the recent more restrictive device-mode setting.

Mikrotik did eventually add some variables to control “default passwords” via netinstall or branding kit + defconf - but there NO similar scheme for device-mode as of yet.

In a perfect world there would be cheap, reliable devices similar to the “finger bot”, but with an ethernet port, i.e. something that can be remotely be commanded to press a (the reset) button.
Though the reset buttons on Mikrotik devices don’t seem to be particularly solid/tough, while needing anyway a not so light pressure to make contact.

Using a web relay to control the power supply of the device (so that It can be remotely hard rebooted) is possible but in theory has much more serious consequences in case of malfunctions.

Connecting/soldering two wires to the contacts of the reset button would be much safer, but besides the complication of doing It on the field, It would likely void warranty.

DIY solutions (not suitable for professionals anyway) seem like possible using el-cheapo sg90 or - better - mg90 RC model servos, but making the control circuit Is not straighforward.
Maybe something based on these tiny/pico/whatever Wiznet boards (Raspberry pi compatibile with integrated ethernet) would do.

The device itself stores the factory password somewhere, if I recall correctly. When you reset the device, the factory admin credentials and password are restored. If you didn’t save the factory password, removed the password label without noting it down, then you’ll need to request it from the distributor or Mikrotik. Otherwise, you won’t be able to access your factory-reset device, regardless of the device mode.

So why not offer an optional feature, where devices can be registered "somewhere" - like a Mikrotik Devices Controller? If a registered device needs a device-mode change, instead of pressing a button or power cycling, users could authorize the change in their controller. This would only work for devices with active connection to the controller, but it would also ensure that once a device is linked, it can't be taken over by another. This would prevent malicious actors from hijacking devices.

@infabo
Sorry, but this is exactly why they say that the road to hell is paved with good intentions.

First of all, this device controller at this point is just a dream.

Second, companies really like this stuff because then the devices can be tied to subscriptions and their resale can be limited. Many companies playing this trick won’t allow you to put away your old devices somewhere and then get them out of the closet if they’re needed somewhere. (Yes, really: some vendors require you to pay licensing for all of your devices or all of your devices will stop working. If you decommission a device, either you keep on paying or you de-register it from your account. It will never work again - either for you or for someone you would sell it to as used - until you re-register it, which the vendor only does “according to their discretion”)

Even if you accept this, I have deployed many devices where they have no access to the public internet (either for technical or policy reasons.) Even if it is allowed to host your own device manager, you are severely restricted in doing one-off installs like this.

Then there is the other problem with hosting your own controller: in order for a device to trust the controller, it must have some sort of pinned certificate for it. If it’s not built in - and for self-hosted it can’t be, you have a problem very much like the original device-mode problem in how to transition a device to a new controller. Button press/timed reboot?

My preferred solution would be to have a separate device-mode password, that is randomly generated on netinstall, and is only available for viewing on the first boot (and maybe even on the first boot only for the first 15 minutes of uptime.) This way the following is fulfilled:

  • only people who are serious about changing device mode remotely have this password
  • because it is regenerated on each netinstall, if you purchase a used device and netinstall it, you can be sure that you’re the only one who knows it
  • it is generates randomly (not set to a specified value), so there is no additional danger in someone using the same code for all devices (so if this password is read from a device e.g. with a JTAG programmer, only the code for the single device is compromised, and even then only until it’s netinstalled again)
  • it’s accessible to the netinstall script, which can auto-insert it into some database (linking it to its serial number) in mass deployment scenarios

As long as I’m imagining random solutions: why not have a special user with super-duper device-mode setting powers that can only be created from a netinstall script and only with ssh key authentication, and is restricted to logging in via ssh. Just to add to it, it’s only allowed action would involve setting the device-mode so it’s not over used as a superadmin user.

My point is not that my proposed solutions are the most wonderful and perfect, just that there is a wide gap in terms of possibilities between “yeah, the device is compromised, traffic-gen away!” and “you need to physically be present at the device at just the right time the command is issued to it”

Not if you replace the default configuration so that a reset-configuration / 5-10s button press reset uses a custom default configuration. See:

And that’s what’s missing for device-mode: some method to set/remove it on new units as part of their netinstall/flashfig/branding-kit schemes.

I’m not talking about that (I agree that removing the label is a bad idea). I’m talking about, as I said before, already deployed devices. Implying, I am not physically in the same room with said device, nor would it be easy nor convenient for me to be so. Looking at a sticker in that scenario is not happening. I would have needed to have the foresight to record the default password in advance, before deploying. I haven’t really had a need to do this to-date since at least in our case, the use-case is almost always a router that is directly attached to a network under our control, so I have no qualms about remotely issuing “/system/reset-configuration no-defaults=yes” on such devices when the situation calls for it and then MAC-Telnetting into them to reconfig. Of course, maybe lack of recording these default passwords on our part needs to be re-evaluated, but that doesn’t address the many devices that have already gone out the door. I doubt we are unique in this sense, either.

If, as you said, there is a way to extract the default password from the device (though without restoring the default config entirely, which would just block us from being able to get back into it remotely!), though, that would be great.


I love this idea.

Sounds like a kind of “secret key” (rather than a password). Good idea. But won’t help for already deployed devices.

This is actually what concerns me the most. One of the key features of device mode is flagging. According to the documentation:

flagging-enabled (yes | no; Default: yes)

The device will perform a configuration analysis, and if traces of suspicious code are found, flagged mode will be triggered. This sets `flagged=yes` and enforces the restrictions described under `flagged=yes`. See the "Flagged status" section for details.

I already suggested in the 7.17beta topic that it might be better to refine the configuration analysis. For example, if ROS detects something like “user ‘admin’ suddenly moved to group ‘read’ and traffic-gen has been running continuously for hours”, then yes, it should flag the device and disable certain features.

However, what I find unclear is why these restrictions are enabled by default, even before any actual suspicious activity is detected.

AFAIK, Flagging just catches if RouterOS internal files are tampered or replaced — not config. Similar to macOS Gatekeeper/SIP or Windows File Protection (WFP).

A rose by any other name. The problem around the whole thing is:

  • The attacks/botnets have become increasingly weaponized and are not one-off any more, but supported by either pure financial or involve state interests. At this point this is not the usual give-us-more-money security industry fearmongering any more, but a fact of life on the Internet. More and more critical infrastructure and things necessary for out daily lives have become dependent on and exposed to attacks on the telecom infrastructure.
  • Realizing this, the US, EU (and particular EU members on their own), UK, Australia and others have enacted rules like banning of certain manufacturers, ensuring stricter crypto, and the rule to be “secure by default” rule (which is famous on this forum, because the usual implementation of this is default random password)

Mikrotik routers are of particular interest in this because:

  • Their routers are often deployed in data centers and other places that have a higher level of potential access to critical infrastructure, and/or access to high bandwidth for DoS scenarios
  • Their normal software includes features like traffic-gen, socks proxy, containers, etc. that enable all sorts of nasty attacks. This is in stark difference to other (especially affordable) routers, because most of these have to be rooted and special tools developed by the attackers, while with Mikrotik “simple” admin access is enough and the features are available with manufacturer support.
  • Mikrotik devices have a long useful lifespan (very good!), have a long support life (very good!) and are often preferred by users who keep these devices in service for a long time. I can’t find it not, but i actually saw some research on RouterOS versions in the wild, and it was staggering how many 10+ year old (and not updated) devices they found. (I don’t know the methodology, probably they just looked at exposed webfig interfaces that report the version),

So basically Mikrotik had to do something about this. And for it to be effective in any meaningful amount of time, it had to be done to already deployed devices as well. Clearly this causes disruption to some operations that have a larger number of devices that are difficult to access physically. Not to dismiss this - and this is only my very biased take on this - but in my view this was unavoidable.

Again, what I find difficult to understand in why a strategy for handling similar situations in the future was not developed.

And similar situations will come. With the level of hysteria around security, and how (not just tech) media likes to pick these things up, with some attention seeking security researchers (of questionable talent) all too ready to feed the buzz, to me it is not a question of whether new restrictions on functionality (already present or newly introduced) will come, but when. And whether one likes their products of not, Mikrotik devices are widely deployed all over the world, and given this sort of pressure, inaction will not be a feasible strategy for them.

I think that’s actually the “check-installation” command (which might well be incorporated in the flagging process.) That flagging “analyzes the configuration” is clearly stated in the docs (and quoted in the thread). I think flagging is some sort of signature-based detection, which while not a bad thing, is ultimately reactive in nature.

Default disabling of dangerous functionality is actually a valid defense that addresses exploits at a fundamental level. Like a web server on Linux shedding permissions and running as “nobody” vs. running a virus scanner. Or having restrictive firewall rules vs. detecting possible threats with signature-based traffic inspection. Both have a place.

Not in all scenarios, but definitely in certain ones, it absolutely can. One such scenario would be, as I mentioned earlier, if you have end-to-end control of the network that the router in question is attached to. In that case, it is 100% feasible to conduct a Netinstall to the device completely remotely, provided your physical connection to the device is via ether1 (or whatever port RouterBOOT will try to netboot from on a given model).

Unfortunately, some of the new default device-mode restrictions also make this more difficult than perhaps it needs to be (for understandable reasons, but still…aggravating). A concrete, real-world example: we recently pushed out ROS updates to a whole slew of CPE, one of which happened to be a hAP ac2. This particular one was getting upgraded from a pre-7.13 version to 7.18.2. This meant that “wireless” was now separated out as its own package. However, something hidden on the skimpy 16MiB NOR flash was chewing up tons of space, preventing the new wireless package from installing (not enough free space), so after the upgrade, the customer’s WiFi stopped working. Since the problem was not with a file visible via “/file/print”, the only option to work around it was to reformat and reinstall with Netinstall. To do that remotely requires setting “/system/routerboard/settings/set boot-device=try-ethernet-once-then-nand”. UNFORTUNATELY, since it was now running 7.18, device-mode was now restricting access to RouterBOOT settings from within RouterOS, so this could not be done!!! The work-around? Downgrade the stupid thing to a version that allowed me to finally change the boot-device, then instruct it to netboot, and Netinstall 7.18 back to the device. This indeed solved the problem and freed up the nearly 1.5 megs of “phantom” usage, allowing me to finally install the wireless package. But, …grrrr. Dumbest. Problem. Ever.

The dumbest problem was this hidden 1,5mb flash usage. This is the second time here in last days I read such an experience report. Mikrotik put 16MB flash in their devices and then forgets to include garbage collection mechanisms. It is their job to do that, as - different to a regular Linux - we can’t see actual files on their root fs. And /files/print is apparently very abstracted and buggy.

I don’t disagree. (Well, maybe that combined with the fact that 16MiB of fixed nonvolatile storage was ever a thing on any model.) Really, though, I was annoyed by the entire problem-set as a whole (and not one any specific part), which is what I was trying to convey: the initial underlying issue (missing space), the impact to the customer post-upgrade (‘routeros’ and ‘wireless’ packages both uploaded simultaneously, the first succeeded in installing, the second didn’t, router booted back up without any WiFi support), that fixing the problem required breaking out Netinstall, and that a new feature designed to “protect” me actually ended up screwing me over as it prevented me from fixing the customer’s problem easily.

It was just a perfect storm of bullcrap all around.


I’m not sure I would go that far. If you have looked into RouterOS internals at all, then it becomes clear that the “root” directory of /file/print is just a particular subdirectory within the writable filesystem (sometimes ext2/3/4, sometimes yaffs, sometimes ubifs, depending on model) on the router (at least on models with larger NAND flashes…these 16MiB NOR flash models have the “root” of this as a RAM drive, and then they hang the same subdirectory normally exposed under “/file” to a directory called ‘flash’), while the actual router config is stored in a proprietary binary format spread across multiple files in a completely separate directory elsewhere on the underlying filesystem.

I’m 1,000% sure that the missing storage was not being taken up by anything that would have normally appeared under “/file”, and was rather likely one (or more) files in the always-hidden config directory. In fact, in retrospect, I probably should have originally tried to see if a “/system/reset-configuration” would have wiped it out, instead of immediately jumping to Netinstall…I guess when I run across my next hAP with 16MiB of storage where a bunch has gone missing, I can give that a try.

Im just a silent reader of this post, but it gets to the point where i want to share my experience.
Generally spoken, i have the exact same problem as @Amm0

We are managing a lot of mikrotik devices and generally want to keep them mostly up to date.
Being sometimes up to nearly 1000km away from a remote site, is not something we can deal with when upgrading the router and device-mode kicks in.
For the flagging, i can say that this is something that concerns me to. When i was messing with some settings on the router, the router flagged itself up to 2 times. Only one flag is needed to completly lock down the router. I couldnt get it to get 3 flags but i still concerns me.

The thing that angers me the most is, when updating a hap ac2 from a ROS 6 version, to >=7.17.x the cpu-frequency is set to something around 730mhz…
And i cant change it to cpu-frequency=auto, because i am not allowed to change settings in /system/routerboard

The same goes for our deployment process of multiple hundred new devices per month, this is something we cannot deal with. Setting the device-mode via a script seems buggy and not working on all mikrotik devices. I am still into that issue an are trying to solve it. For the hap ax2 it works perfectly fine, but for the ac2 im just running into issues.

Ive made a ticket at mikrotiks help desk last year in april, asking for a simpler way to activate protected routerboot. Because activating that option needs a button press in 1 minute. Ive got a response that due to security reasons physical access is needed. After i wrote back a message to stand my point again (because it is not very reasonable for a person to press 20 buttons in 1 minute, while configuring 20 devices at the same time), mikrotik support noted that if there are similar requests from other users, they will consider changing the process.
In january this year ive reopened the ticket again (that was the time the first versions of the device-mode got introduced into ROS), asking for a way to configure device-mode & protected routerboot in flashfig or netinstall or by any chance with a special package for big customers. Ive got the answer that they are currently looking into a solution, but cannot give an ETA.

Hopefully they will find a suitable solution in near future, cause just right now for us its an absoulute pain and prevent us from keeping our devices up to date.

Not to derail the conversation too much, but I’m pretty sure that the CPU freq default setting on RouterOS 6 / RouterBOOT 6 was to fix it at 716MHz. In fact, originally, there WAS no “auto” freq option for the ac2 CPU. So I don’t think it is the case that upgrading to v7 is setting or changing that value. It’s merely keeping it at the existing setting that you already had it configured for, which is the default.

I am not sure what version introduced it, but at some point during 6.x, the “auto” setting was added. I haven’t tested it, but would be willing to bet that if you were running a 6.x version that added the “auto” freq setting, and then you changed it to auto while running 6.x, that it would still be set to “auto” after upgrading to 7.17+.

EDIT: found it…it was in 6.48:


What’s new in 6.48 (2020-Dec-22 11:20):

*) arm - added support for automatic CPU frequency stepping for IPQ4018/IPQ4019 devices;

I am fairly certain that upgrading to 6.48 or 6.49 did not change the CPU freq setting for you from 716MHz to auto. You would need to have done that yourself, by hand, if you wanted to use it.