Are there plans (and is there sufficient community interest) for RouterOS to implement something equivalent to `configure replace` on Cisco/Arista or `load override` on Juniper?

I was able to find at least one reference to this from 2016 [1], but I'm hoping that there is more interest in this feature in the 7 years since.

The ability to do transactions to change the system configuration would also be welcome if the system got overhauled. The main use case for having partial or complete transactions (like 'configure replace') is to enable network automation workflows where the configuration is templated externally (via Ansible/Jinja2/some other method), copied to the device, and replaced while the device remains operational.

I appreciate the fact that this might be difficult to accomplish since unlike many of its competitors, RouterOS' configuration syntax isn't so much a declarative configuration as it is a scripting language. A given configuration file can have global side effects that produce different configuration outcomes under different circumstances.

Because of this, an approach to provide the same effect within the RouterOS framework might be to provide us with a new verb (along with `add`|`remove`|`get`|`set`) called `assert`. The purpose of this verb would be to have the effect of `add` if no such element exists or `set` for all of the fields whose values differ from the configured values.

Here is a concrete example. Right now, if I generate a configuration like `/interface/bridge add name="foo" protocol-mode="none"`, it will work correctly the first time. If I apply the configuration again, it would fail with the error that a bridge already exists with that name, even though the intent of the statement was to change nothing. Likewise, if I wanted to change the value of a field, I could instead do `/interface/bridge set [ find where name="foo" ] protocol-mode="rstp"`. This isn't too cumbersome, but if that was what my config was generated as originally, it would fail because there would be zero matching rows for `set` to operate on since the bridge doesn't exist.

This can of course be fixed with something like: `: if ([ /interface/bridge print count-only where name="foo"] > 0) do={ /interface/bridge set [ find where name="foo" ] protocol-mode="rstp" } else={ /interface/bridge add name="foo" protocol-mode="rstp" }`. But this starts to get unwieldy when the entire configuration has to be defended with all of this bespoke logic in order to be applied multiple times and produce idempotent results.

If we had a verb like `assert`, the configuration could simply be `/interface/bridge assert name="foo" protocol-mode="rstp"`. Any values that aren't provided after the `assert` will be set to their defaults (even if already set). Any values that are provided after the `assert` will be updated to match. I assume that RouterOS behind the scenes uses a hidden primary key in the operational datastructure to keep track of which entries are unique, so that can continue to be leveraged to figure out which of the provided values are supposed to indicate which item to operate on. So for example, two bridges are identical if their names are the same. There are some tricky cases like firewall filters which have an order within the chain but not one directly exposed in the configuration syntax (unlike, say, Cisco IOS sequence numbers). I'm sure a solution could be found.

The other major gap to close would be what to do with lines that were intended to be deleted. Today, the only way to accomplish this would be to `/bridge remove [find]` before adding everything back (disruptive therefore not an option) or to carefully track everything that needs to be removed (also problematic). In the second case the templated configuration would have to generate all of the conditional logic like above and then construct a complicated `where` clause to remove every other item. This is fine in theory except that adding a new `assert` to the templated configuration would require modification to the cleanup `remove [ find where <gigantic compound match clause> ]` statement. This quickly fouls up the complexity of the templating logic because the last line has to depend on the state of the previous lines. Very ugly and a big part of the reason to ask for this feature on the RouterOS side.

Fortunately, this too could be addressed by adding an `assert-only=true|false` (default: `false`) option to the `import` command. With `assert` and default `import` behavior, the device would behave somewhat like a configuration merge. In this case, the applied configuration using `assert` would only add and modify. With `assert-only=true` and a configuration full of `assert`, the parser would automatically keep track of items not evaluated by `assert` clauses during the `import` and at the end, automatically remove them. The result would be equivalent to `config replace` or `load override` in that the changes made would leave the device configured in a way exclusively described by the `assert`s.

There are probably more gotchas that would need to be addressed, but to me this seems like it would be a foundation on which Mikrotik could design this feature in a way that minimizes the amount of overhaul to the configuration/parsing mechanism and preserves backward compatibility.

Thanks for reading!

[1] viewtopic.php?t=110380

:if ([ /interface/bridge print count-only where name="foo"] > 0) do={
/interface/bridge set [ find where name="foo" ] protocol-mode="rstp"
} else={ /interface/bridge add name="foo" protocol-mode="rstp" }

One of the correct ways is:



When you write too much, you get lost along the way.

1) Making an import by replacing everything (reset config) that exists before is already possible. (ignoring the description of how for now)

2) Simply add a "method=[ *** ]" option to "import" if everything must not be deleted first, but the import in case of duplicate objects replaces what is written on the" import".
*** The options are the same of the point 3)
Is equivalento to set the default method for each "add" on .rsc
(the logging is another point)

3) In "add" simply add "method=[add (default) | ignore | overwrite | update | default]"
add = the same as now
ignore = same as now, but DO NOT DO ERROR... simply ignore...
overwrite = overwrite (delete) previous item (if exist) with one new, like a "delete (if exist) before add", obviously do not throw error if already not exist.
update = if not exist: create, if exist: update the existing object with new specified attributes, keeping the previous as are.
default = the same as update but set as default all not declared values, defaulting what already is set to not default †

† why this last: if the bridge is deleted, also the port binds are lost, because internal .id change,
instead with "default" option, the bridge settings are restored to default, the wanted are set, and the port binds are not lost, because is still the same bridge.

Code: Select all

/interface bridge
:if ([:len [find name="foo"]] = 0) do={add name="foo"}
set "foo" protocol-mode="rstp"

That works, too. My point was that whether you write it your way or mine, if you do the entire config like this, there's a lot of extraneous logic to fake `configure replace`.

1) Making an import by replacing everything (reset config) that exists before is already possible. (ignoring the description of how for now)

Is there? I'm only aware of `/system/reset-configuration run-after-reset=config.rsc`. The goal here is not to impact the configuration if it is the same, so a reboot doesn't meet the requirement.

2) Simply add a "method=[ *** ]" option to "import" if everything must not be deleted first, but the import in case of duplicate objects replaces what is written on the" import".
3) In "add" simply add "method=[add (default) | ignore | overwrite | update | default]"

Sure, this would work too. Whether we have a `add method=` or a new verb like `assert`, it would meet the first need. There does still need to be a mechanism for removing config lines that are not present in the new candidate config.

I like the proposal. It'd be a handy addition. Will it happen? Probably not unfortunately.

I get the request. Not saying some "update" (or "assert" or whatever) wouldn't be per se be handy.

But wholesale "configure replace" kinda ignores the broader picture of RouterOS configuration, and a bad model. It isn't a cisco or Juniper. RouterOS is more like SQL — so it's tables and relationships all the way down. So like SQL, it's essentially INSERT and UPDATE, and there are relationships and primary ID – it the "transactions" part they miss. And that be the first step to some monolithic replace-without-reboot, since you'd need be able rollback a failure otherwise.

Also the ":export" today isn't really the whole "config" (e.g. no container, certificates, etc.). Additionally it could persist sensitive password in plain text, so not ideal for recurring use as a whole. The idea is that items not expressed (like certs) in config are created when the configure/default/reset-configuration script is run, at BOOT. So the file you see with :export isn't really the "whole config", is a new router creation script, by design. And, if it was the same router...you just save a backup file, and restore that on same router – which does get you everything.

Now I'm all for some additional operations for admin tasks to avoid the set/add, specific Bridge VLAN table. And there may be other places too. But "configure replace" is "-1" from me.

RouterOS is more like SQL — so it's tables and relationships all the way down.

Every network OS I know of uses a configuration database under the hood, including Cisco/Arista/Juniper. I agree that Mikrotik made the right choice exposing the database semantics as the configuration language rather than parsing a hierarchical plaintext configuration. I suspect we are both against an interpretation of this request that takes RouterOS closer to `:export` being the configuration as opposed to just a script that configures the database.

On the other hand, if this has database semantics, then where is the MERGE capability? The 'configure replace' problem also rears its head in SQL if you don't have MERGE or some other "upsert" capability. I agree with your assessment that transactions are the biggest thing that is missing. What I am calling the `configure replace` problem is really a combination of lacking both transactions and upsert.

Also the ":export" today isn't really the whole "config" (e.g. no container, certificates, etc.). Additionally it could persist sensitive password in plain text, so not ideal for recurring use as a whole

I see what you're saying. Frankly, the things that don't make it in `:export` really have more to do with device management than they do the control/data plane configuration. I don't mind if they would have to be touched separately.

If we disagree, it might be over whether we think that the current state of affairs is acceptable in terms of network automation. Without transactions and upsert, RouterOS will never be able to be automated in a declarative fashion without a huge amount of middleware. If, for example, you look at the Mikrotik community Ansible module: the authors didn't go to the effort of trying to make the module idempotent. It only executes commands as plays. Presumably they stuck to this because doing more would require writing all of the missing middleware to "query the database" with ROS syntax, calculate all of the adds/sets/removes, perform all of them, check to make sure there were no errors, and if there were, calculate the inverse of the adds/sets/removes to set everything back.

The way I see it is that the most people will benefit if Mikrotik is the one that implements the transaction and upsert logic. That way it will be a first-party feature of RouterOS and we (network operators) each won't have to reinvent the wheel in our own development pipelines. I suspect it could also have a marketing and sales impact for selling Mikrotik devices into organizations that are moving towards an automated configuration lifecycle with contemporary tooling (Ansible, Jinja2, etc).

I may have sound harsher than needed. I guess largely accepted the reboot & use VRRP where downtime is a big concern. It not just config replacement that necessitates a reboot. e.g. V7 has a lot of updates (and two reboots if you want matching firmware). And two Mikrotiks is less one cisco by far.

In fairness, where this problem rears it head is something like Terraform and the like. I don't use, but familar enough to know the models are pretty incompatible. Or, yeah, reboot with new entire config. But even here, each device has different things too, so "config replace" doesn't help that much here. And more sophisticated plugin likely help here, but still be a fair amount of RSC built-in to that for it work.

But the limitation here isn't the "assert" operation, it "dynamic" variable/script/function in the attribute values that what's missing in the syntax IMO. For example, where the $$LAN_INTERFACE (just for example) is stored with the "add" & if it changed, the "row" be updated based on the variable changing. You need some "super" or "persistent global" that's stored with the "config". JavaScript or Python call them a generator, but idea being is you'd be able to link an attribute to value (that's updated when the variable changes). NOT just have variables in the config resolved once at :import.