[Discussion] MikroTik configuration abstraction complexity

RouterOS General
1

This is my first forum post, that I am making in genuinely trying to get MikroTik to see pain points of most MikroTik users, especially SOHO/Home users and even professional network engineers too.

Every vendor in the network world has their own flavours and implementation for configuration abstraction front-end for exposing the UI/UX to the human operator, whether it’s CLI or some JSON over APIs.

There’s the ugly Cisco CLI, there’s the tactical CLI from Juniper JunOS & VyOS, and there’s the modern-day DevOps/Linux-y friendly CLI/API from Nokia SR-Linux.

MikroTik is obviously a unique network vendor in both the type of hardware they sell and produce and support in the last 20+ years. And due to this large portfolio of devices, it’s understandable why MikroTik has excessive tech debt in their software stack that prevents and overnight migration to JSON-like CLI like JunOS or DevOps styled SR-Linux.

The challenge however is that there are too many ways to configure different hardware models from MikroTik, especially on layer 2, 2.5 and 3 (VLANs, bridge, VPLS to bridge or no bridge etc), this leads to confusion as evident on this forum itself, every day, every month every year, we see even professional network engineers struggling with MikroTik’s bridge/VLAN configuration or many other aspects of configuration in general, like BGP on RouterOS v7, it’s simply not readable-friendly through the CLI, with the “.” weird syntax.

I’m well familiar with Linux Netfilter framework, Linux bridge VLAN-aware and worked with Cumulus Linux as well, but never have I seen so much confusion as MikroTik’s implementation.

Of course, I’m known on this forum to be a bit of an asshole, but this time I decided to look at this issue with an open mind — The reality that I have to accept is that, the abstraction model for RouterOS configuration is too heavy with tech debt, leading to issues for users, even those who are experienced network engineers and have worked in the industry for 20 years+. While I personally managed to avoid bridge/VLAN debacles by following the official docs, it doesn’t mean it’s a great experience, it’s way too much inconsistency and lack of resilience across hardware upgrades etc:
https://help.mikrotik.com/docs/display/ROS/Basic+VLAN+switching

This is probably because MikroTik supports almost ALL of their hardware, leading to excessive tech debt that impacts their modern hardware products, I think it may be time for MikroTik to potentially consider ending such excessive long-term software support in favour of avoiding tech debt OR have two different versions of RouterOS, RouterOS Legacy for older hardware with the old config abstraction model, RouterOS Modern for newer Marvell hardware with proper 2024-grade CLI/API abstraction models like other vendors.

You can also look at VyOS, which is open-source, yet it doesn’t have config abstraction tech debt/complexity that leads to excessive confusion or misconfig for users.

It’s also unclear if MikroTik is using switchdev, or some NIC offloading with Linux bridge VLAN-aware implementation on some models, or purely Marvell SDK on Marvell chips? Lack of in-depth CCIE-level documentation on RouterOS is also another pain point.

It’s probably easier for MikroTik to implement something like SR-Linux, considering both SR-Linux and RouterOS are both Linux kernel underlay for the control and management plane:
https://learn.srlinux.dev/blog/2024/vlans-on-sr-linux/

@normis/@MikroTik management, would it be possible for you guys to some day remove legacy RouterOS CLI/Abstraction model and do something new? One where configuration of bridge VLANs/VLANs in general etc are easy just like Juniper, Cisco, Nokia etc? On other vendors, misconfig doesn’t lead to performance issues (issue unknowingly routing/bridging traffic through CPU).

Side Note: MikroTik uses Linux Kernel for RouterOS, it would be nice if they contributed back to the upstream Kernel (patches/fixes etc) like Nokia does, even better, let’s move away from legacy bridge/VLAN implementation CPU-only products/paths and implement DPDK for maximum line-rate performance, let’s move away from legacy iptables of MikroTik RouterOS and move to XDP for packet filtering, let’s move to nftables for IPv4/IPv6 NAT and NPTv6.

It would be nice if MikroTik is open-source for certain components like Nokia, the community could help patch things faster, improve things faster, along with root access/shell login for RouterOS:
https://github.com/nokia/srlinux-yang-models

2

Some latest examples of the issue with abstraction complexity:
http://forum.mikrotik.com/t/trying-to-use-vlans-l3-hw-offload/172696/1
http://forum.mikrotik.com/t/vlan-tagged-untagged-on-same-router/173280/1
http://forum.mikrotik.com/t/vlans-not-talking/173263/1
http://forum.mikrotik.com/t/cap-ac-vlan-switching-hardware-offload/126415/1
http://forum.mikrotik.com/t/map-lite-not-able-to-configure-very-simple-use-case-lan-to-wlan-bridge-neither-does-manual-describe-such-a-basic-thing/155902/1
https://forum.mikrotik.com/viewtopic.php?t=204083

3

(slightly off-topic)
Not going to contribute a lot in the context of this post.
But I do get your point even though I nowhere have the same experience as you have with other vendors. So it may be interesting to follow where this leads to.
I certainly would like to see more from you with this style.

Nice way to start a discussion !

4

Yeah, but it looks like nobody cares. They must love the abstraction complexity of tech-debt ridden RouterOS.

Heck, can’t even get MPLS/VPLS to work on the ASICs on CCR2k models, it’s still CPU-only in 2024.

5

It’s not that we don’t care it’s just that I doubt that Mikrotik will change something here.

In another topic I proposed to them to create RouterOS lite for SOHO devices as there is really no need for BGP, MPLS etc. All of this advanced routing functions that will never be used in such devices. So leave just basic functions that could be useful in such device.

Also if they want to expand to such market then much better GUI is needed (for eg look at Ubiquiti, TP-Link etc.)

And I agree with you, Mikrotik have too much older device that they supports, to much different hardware they need to adapt ROS to.

6

MikroTik lacks software engineering expertise, it’s clear as day based on the facts we all know and the bugs and the lack of features (MPLS on hardware etc).

Why they never scaled up and take VC money like Juniper is beyond me. Look up founded date/year of Juniper and MikroTik, both started in the same era, one’s rich, one’s broke…

7

Lacking software engineering expertise. I certainly would agree.

Too much legacy hardware? Indeed.

Where are the examples of the ROS complexity?

I looked at the VyOS docs. Their config management is very interesting in terms of features like versioning.

But VyOS seems to be a full blown Linux OS. Why do we compare with embedded device OS like ROS?

A fair comparison would be OpenWrt. But their CLI and config management is crap TBH. And the luci webui isnt much better. Too sluggish.

8

Well, for eg. configuring VLANs. I think that regular users would be much more happier if they had some kind of wizard or something to do that. Take a look at ubiquiti and how they have that solved.

I mean you can’t satisfy all users but if Mikrotik is doing a push towards regular home users then I think that GUI needs redesign.

9

From the very little I know of this OS and from the view point of a non-professional/casual user there are surely issues well before the “technical debt” or “abstraction complexity”.

The documentation is scarce and extremely incomplete, the little that exists is poorly written and in some cases confuses more than helps.

The impression is that the help pages are written casually by people that either cannot or don’t like to explain the way the commands work or how they should be used (and why).

There is an almost complete lack of “sane”, “complete” examples, everything is fragmented and scattered throughout the Mikrotik site, most of what you can learn (as a newbie) comes from the forum (where the info is also scattered, but at least with some patience, lots of search, countless side deviations, it can usually be found).

At the level of the home/soho networking is not (should not be) brain surgery or rocket science, there can be what 3/5/10 types or common setups, it shouldn’t be too difficult to provide a complete commented template for each one of those cases, reproducible in (say) CHR/gns3, tested and verified.

But, looking at the more complex setups/questions on the forum, asked by people that (seemingly) do have a networking experience before and besides Mikrotik’s ways of doing things and at the replies by experienced people in both general networking and specific Mikrotik OS it seems like there is often no “canonical”, “documented” solutions, but rather this or that “trick” coming from own personal experience or extrapolated from this (or that) MUM presentation.

If you look at the releases of new versions they are essentially an endless list of mostly vague one liners describing the change without any logic or order of relevance, either they do have a proper documentation for each change (but they don’t publish it) or they don’t have it at all, tertium non datur.

Besides calling “stable” what is actually at the most “beta” and calling “beta” what every other software house would call “pre-alpha” (after all what is in a name) the fact that in 1 1/2 month 7.14 beta had 6 releases, smartly called 3/4/6/7/8/9, each one touching almost every aspect of the OS, in many cases fixing new bugs introduced in the previous version the impression I have from the outside is that there is a push for publishing, no matter testing even in simple, basic test cases and an attitude to touch everything at the same time without any apparent priority.

10

One was so rich it sold out to HPE (who will ruin it). The other probably makes decent money for the founder/owner and staff.

11

The last feature that had some special attention was BTH. The rest is a lots of fixing and fixing of fixings.

But back to topic: I would wish to see a unified way to configure things one way. No more fooling around and reading all the - when they exist - warnings in doc: “if you have switch chip XYZ then DONT DO IT THIS WAY. then beware and set this FOO here and avoid BAZ here”. This is a very error prone approach MT had chosen. why does ROS not resolve the caveats behind the curtains magically without having the user to know every aspect of any platform and what is wrong and right depending on just a piece of chipset/hardware.

12

Because MT obviously lacks a few developers to do something from start to end and not stop half way. They started great by hiding switch peculiarities behind L2 HW offloaded bridge. They are sticking to it for new products, but not all of them received offload (yet). But: they did not come around to do it for all capable older devices either (CRS1xx, CRS2xx, smaller devices with Qualcomm switch chips). And the new bridge is around already for ages. The way I read lack of HW offload on these devices is that they de-facto decided they were EOL but they are not willing to admit it yet.

13

Any network engineer who’s worked with MikroTik, Cisco, Juniper, Arista, Huawei, Cumulus Linux will know what complexity. This forum itself is full of it. I’m not going to point out the obvious if you can’t see it.

RouterOS supports MPLS/VPLS/BGP/OSPF etc, it’s on par in terms of functionality with Cisco IOS variants, JunOS, VyOS, Nokia SR-Linux. No engineer would compare RouterOS with consumer-grade OpenWRT. How the hell do you run MPLS on OpenWRT?

14

+1000 to @jaclaz comments:
http://forum.mikrotik.com/t/discussion-mikrotik-configuration-abstraction-complexity/173293/1

15

Doesn’t take away the fact that Juniper hardware + software are carrier-class in the industry, even better than Cisco. Can you call CCR2216 as carrier-class?

16

If MikroTik cannot afford an end-to-end software engineering team who can handle NOS development from front-end CLI/API to back-end dataplane, then perhaps they should become like Edge Core and sell hardware with ONIE, let us install our own NOS instead.

As I highlighted in OP, VLAN-aware bridge is not the problem, it is easy to configure on Cumulus Linux or any Linux based NOS that uses switchdev/Linux DSA etc.
It is not “easy” on MikroTik even for someone like me who has Linux networking skill set. In fact, there’s a lot of things Linux bridge can do, such as per-VLAN STP etc, that MikroTik cannot.

17

I don’t agree on one thing. MikroTik creating “RouterOS Lite” or some crap, this will be dumb down their hardware capabilities, check Cisco, Juniper, Nokia “enterprise” APs etc, they are dumb as hell and can’t do crap beyond basic functionality.

But, yes, WebFig/Mobile app overlay abstraction model (like TP-Link?) could be made for "lite” SOHO/End users who can’t even differentiate BGP from OSPF, leaving full SP features intact in the core of the OS for SOHO users who create home labs etc and make use of MPLS/BGP etc. I run BGP in my own home as well, static routing is for peasants!

18

I am a peasant. :open_mouth:

This said, real world anecdata, a few days ago I had the need to work on my laptop at the same time as with a workstation PC (don’t ask) in a room where wi-fi is suboptimal, and I have only one ethernet connection (that goes in the PC) and - since I had literally in my hands a spare Ax lite (I was putting it on a shelf) I had the brilliant idea to use it as a dumb switch.
But then I realized that I had not any free mains socket free near the two PC’s and had the even more brilliant idea to use it as a repeater for the wifi, placing it near a free socket a few meters away.
After more than one hour fighting with the stupid thing (and failing miserably) I took off the shelf an old TP-Link (TL-WA901N) and in no more than 10 (ten) minutes I had everything working for what was needed[1].
This probably means something.

[1] the quick setup guide is a two page .pdf:
https://static.tp-link.com/2019/201912/20191230/7106508576_TL-WA801NTL-WA901N_QIG_V6.pdf
you setup a password, choose the desired mode (Range Extender in my case) and follow instructions.

19

We all are peasants when comparing with for eg. @DarkNate.

For someone who isn’t into networking Mikrotik was my first real touch with networks and i got used to it so i don’t really know how other vendors have GUI sorted. (Juniper, Nokia, Cisco)

But one feature that i would really love to see is PPSK.

I want to have one SSID no 5 of them… I used user manager and radius but problem is that i run out of sessions due to license limitation and it’s really unnecessary complication…

20

The real issue here is MikroTik lacks software engineering expertise, may be financial reasons, may be understaffed, maybe both, maybe more, but they don’t seem to care. Who the hell still sells NEW hardware with 32-Bit software in 2024, even? See below:
http://forum.mikrotik.com/t/l009-and-zerotier/172938/11

Even tiny embedded industrial electronics in 2024, tends to be arm64 with 64-Bit Kernel/Code base, but nope not MikroTik they will innovate the world of networking with 32-Bit OS.

Clear sign here that MikroTik has software engineering issues, not much hardware issues. MikroTik hardware/block diagram etc is actually open and nice IMO.