MikroTik Devices Controller

[pe1chl]

It will probably be released right after the scripting function library

[infabo]

This thread is for brainstorming. Throw in some buzzwords that will end in a tin.

[pe1chl]

Check this topic and what came of it: http://forum.mikrotik.com/t/built-in-function-library/117288/1
I mean, brainstorming is alright but when you do it as a company you may get some expectations…

[infabo]

Can you summarize what was added to the function library from the long topic?

[pe1chl]

Nothing at all. The function library never came into existance, 4 years of discussion going to waste.

[rextended]

Is not all correct, on 7 is added the random number generator…

[infabo]

Nothing at all. The function library never came into existance, 4 years of discussion going to waste.

Basically what I said.

[pe1chl]

Is not all correct, on 7 is added the random number generator...

The way I understood "function library" there would be an optional package that you could install, or some script file you could download to the router and call from your scripts, containing a collection of utility functions as a layer on top of (i.e. written in) the scripting language.
Basically anyone can write that in the latter form, and some people indeed did so.
But there never was an official MikroTik supported version, not even an endorsed one.
Of course in many cases it would be much more efficient (or it would even be the only practical way...) to have new functions available as built-in functions in the scripting language itself. I would not call that a library, but still it would be welcome.

[senseivita]

I think I might be just Mikrotik’s target audience, and I think I’d thread carefully in its place.

Something like the UniFi Controlller is pretty to look at but it ain’t very useful. It’s slow, it’s got so many problems with from adoption, to disconnections, to being unable to handle consecutive (not “too many-”, just ·“consecutives”) updates, all of which Mikrotik doesn’t have plus accessibility issues for installers (that thing when you come down the white bright rooftop into an air conditioned relatively dark server room and you try to read fonts too thin) which are a staple for this mgmt utils. I’d document better instead. The documentation is written for the CLI, but the CLI isn’t what’s encouraged to use, there’s this myriad of admin UIs but only documentation for the one without any graphics, often with graphics/screenshots/suggestions for the other ones. It’s also in a needlessly technical language most of the time but it’s not technical enough where it matters so there are no ambiguities–there are way too many of those. Sure it’s hard to contemplate all possibilities in computer networking being nearly endless, but it can be done because it’s been done, there’s another vendor who’s managed.

I’m referring to pfSense. If I had to improve Mikrotik’s , I’d take a look at what used to be called The pfSense Book (now just its documentation) for guidance. It does an outstanding job of explaining why things work as they do and even why decisions of the UI were taken in some instances addressing straight on their shortcomings.

You don’t need to dumb things down the way Apple, Ubiquiti, Cloudflare, (..) and others do, it’s kind of infuriating when yet another vendor discovers minimalism and your settings are gone. This is how we end up with these annoying certificate warnings in every browser on non-routable addresses, if we were taught, we probably wouldn’t be needing to be treated like children, I’m sugar coating there, and end up with lack of options, and expensive, limited and cumbersome controller device or alternatively “management as a service”, what UniFi turned into. UniFi still can’t multi-WAN properly, basics like DHCP reservations are kind of there, it’s over a decade old. Also there is telemetry; every device, even outdoor wireless radios are consistently trying to contact external web, STUN servers, IP addresses in China, were Ubiquiti hosts some of its UNMS or whatever it’s called lately. The forum, is mostly complaints now and not a link easy to find anymore, maybe it’s related.

Ubiquiti for years have been trying to get rid of support for “legacy” (Wi-Fi 4) devices and has deleted the needed apps to access the old self-hosted NVRs, another form of controller. People got angry because you could only access your cameras if you maintained a Play Store, Apple ID account in which you got the apps earlier before or trust the company won’t have another change of heart and upgrade. As for the APs, they’d become unmanageable just because they didn’t feel like maintaining that support, which wouldn’t be required if the APs had a built-in admin UI like Mikotik’s do. It seems they reversed that somewhat, you can still manage the APs in the latest controller but not group them to newer ones. It was too late though, a lot customers called it quits.

Focus on documentation, build-it right in Winbox or one of these great UIs you already have which are as powerful as you know how to make them do things, make documentation offline, not links to a wiki which aren’t helpful when you’re setting up a device–when you need it the most. Finding the default IP management address shouldn’t take half an hour. Things like Mikrotik “Home” or whatever dumb things down way too hard and don’t provide a learning/evolving path to follow. For those of us coming with advanced, already set up networks on platforms a little more straightforward, reaching the level of knowledge necessary to deploy the same infrastructure can be cost-prohibitive in terms of time/downtime. The first time I tried a Mikrotik router, I ended up returning it because it was going to take way too much time to set up. The last time I tried Mikrotik (it’s been like 4 or 5) I couldn’t find how to set up full cone NAT on a dynamic IP interface without first learning pretty high level scripting.

[suhl]

Do you think it would be possible to get something similar to Cisco High Availability?
https://www.cisco.com/c/en/us/td/docs/routers/access/4400/software/configuration/xe-16-12/isr4400swcfg-xe-16-12-book/configuring_high_availability.html
There’s this other Mikrotik community project I found here:
https://github.com/svlsResearch/ha-mikrotik
Basic idea is to be able to change the configuration in one place and all the routers/switches in the HA group will be affected by that change. Versioning would help a lot as well.
Good luck. I love MikroTik from the early beginnings in the late '90. You are destined for a big success with your approach.

[realmark]

Native support for push metrics / streaming telemetry!

Support for pushing data to influxdb or similar. We’ve moved away from “network monitoring” tools towards grafana dashboards for all server monitoring, firewalls, and are attempting to do the same on routers/switches. No SNMP inbound, proxies, agents, etc. Just a clean feed of desired details streamed off to a target of choice.

http://forum.mikrotik.com/t/feature-request-streaming-telemetry-native-metric-export-for-influxdb-or-similar/154872/4

[spippan]

My number one suggestion and highest priority is to build in strong security from the start, not as an afterthought. MikroTik could show the rest of the network equipment industry how to establish best practices for securely maintaining network devices, and that should be the goal!

Some suggested approaches:

• Incorporate a robust management facility for establishing and maintaining administrator and user accounts. Ideally, this should also support “machine” accounts that could be used for automated status queries and management of devices. The controller itself should use a “machine” account for its purposes, and this account must be customizable by the customers. Integration with enterprise systems like Microsoft’s Active Directory would likely be desirable for some customers.
  
  
  
• Build in a Certificate Authority (CA) for issuing certs to network devices. The new controller should incorporate hardware protection for private keys (especially CA’s own private keys), along with the ability to securely clone the CA’s private keys to a backup controller using multi-party controls. Network devices should be able to request renewal of certs from the CA using automated methods. There should also be automated tools for installing certs in network devices. Certificate revocation must be supported using a dynamic protocol like OCSP with ability to push out revocations immediately (e.g., via CRL update). An optional approach could be to support integration with a third-party Certificate Issuing/Management system, but these days the tools to implement the subset of services required for network devices are readily available from multiple Open Source projects, including OpenSSL itself.
  
  
  
• Use CA in new controller to also issue client-side certs for network administrators. Client side certs would be used with mutual authentication to handle logins to Winbox and other device-specific services, including an option for providing SSH keys via certs. Automated client cert renewal should be supported, and it must be possible to revoke client certs with immediate pushing of revocation notices to devices along with dynamic cert checking. (Aside: WinBox might directly support requesting administrator certs from new controller’s CA.)
  
  
  
• CA should also issue user certs for Wireless access, PPP/VPN remote access, HotSpot services, etc. This implies that there should be specialized access to the controller from users to handle cert requests or update their account details, such as email addresses, phone numbers, workstation details, mobile device details, etc. In an enterprise environment, it might be possible to pull this sort of information from a central service.
  
  
  
• Fully support the latest cryptographic algorithms and measures, including the widely accepted elliptic curve algorithms (e.g., ED25519). Provide policy controls to limit/restrict use of cryptographic suites in the network devices from a network-wide perspective.
  
  
  
• Provide a complete implementation of a robust RADIUS service for legacy devices and services. For extra credit, support RADIUS integration with Microsoft AD NPS/RADIUS facilities.
  
  
  
• Implement an SSH key management system that would support pushing administrators’ SSH public keys to network devices and rolling keys as appropriate. Immediate removal or disabling of SSH public keys for administrator login is also necessary. One possibility would be to use SSH and SSH key management to handle securely pushing updates to devices, along with invoking of scripts and automated retrieval of device information, including device configurations.
  
  
  
• Provide an encrypted storage system for maintaining sensitive information at rest, especially for device configurations and any other sensitive information.
  
  
  
• Build in a software repository for redistributing RouterOS (and possibly other software/firmware packages) to network devices in a controlled manner without requiring that individual network devices have access to the Internet. This could be an adjunct to the requests from others on this Post for RouterOS bulk updates. Ideally, this system should support two or more storage partitions on devices that support this option to make it easier and safer to rollback an update. For devices that are not equipped (or configured) with multiple partitions, a rollback facility would still be a valuable capability.
  
  
  
• Implement support for redundant device deployments, including for the new controllers. For example, support measures to independently update RouterOS in each member of a redundant device pair thereby allowing the other member to maintain services during the upgrade. This capability could also allow staging of firmware in redundant systems to confirm stability before completely updating all devices. Similar capabilities would also be necessary for redundant controllers. Resilience is an often-overlooked essential security requirement.
  
  
  
• Support RANCID or an equivalent service for maintaining network device configurations in a source control system (e.g., Git). This could be an add-on package for users dealing with larger networks or complex support requirements. (Aside, my own experience using RANCID with a complex network involving devices from multiple vendors illustrated that this is an invaluable tool for not only tracking configuration changes, but also monitoring changes made by multiple administrators, which in turn provides further security controls with the added ability to recover from unapproved or ill-conceived changes.)
  
  
  
• Support management of security credentials for SNMPv3, including the ability to update credentials periodically in a controlled and automated manner. Provide methods for pushing SNMPv3 credentials to network management systems (e.g., via secure upload of an exported dictionary of credentials).
  
  
  
• Provide tools for automated responses to DDoS attacks using parameter-driven approaches for invoking mitigation measures.
  
  
  
• Implement a comprehensive system logging facility. This could be optimized for MikroTik devices to leverage enhanced features. The system logging should support TCP logging, as well as optional support for logging via encrypted links (SSH, IPsec or other VPN). It should be feasible for customers to implement redundant syslog servers for resilience as well as protecting logs from being modified by attackers. The logging system should be capable of relaying log records to more advanced enterprise-oriented logging systems (e.g., Elastic Search).
  
  
  
• Since DNS is one of the most essential services and also one of the most sensitive from a security perspective, centralized management of DNS services in network devices would be a valuable service. This could include the ability to maintain static DNS caches across some or all network devices to improve availability of essential DNS resolution during periods of degraded operations, such as network outages or partitioning.
  
  
  
• Provide robust NTP services, ideally supporting authenticated access. The new controller would ideally provide an option for GPS time sync so that it could operate as a Tier 1 NTP server. This would also be an underlying security facility for supporting certificate management and use of time-based authentication services.

Yes, this is a lot. However, everything listed above is readily available and supported in the Open Source realm. What is important is to build these capabilities into the product plan, and build other controller features and capabilities on top of a secure base. Not everything needs to be in version 1, but everything (and more) needs to be in the product plan and resulting design. Security is just too important an issue these days to not be the primary objective for anything that purports to control network devices and maintain a network system.

perfect.

[spippan]

Yes that would be nice to have for “the general public”.
I have this function for ages as I export my configs to a locally hosted git versioning system with the additional “gitweb” program that shows output similar to the above via a web interface (only accessible locally).
Very useful to be able to look back at what you changed before. Also useful when having more than one admin.

For anyone interested in a solution like this, there’s Oxidized.
https://github.com/ytti/oxidized

…can also be coupled with, e.g. LibreNMS
had 2 of those running with oxidized (which pushed config versions to a local git repo)

[lcohen999]

This would be fantastic

It would be nice to :

get a Tik online, register with my account, then I can push any configurations I like (IPSEC/LAN, etc) or “copy from another mikrotik”

Have it run in Mikrotik’s cloud, or make it open source so I can spin up my own linux server and use that instead.

Past that, the usual stuff, remove management behind NAT, push packages, etc.

We are a small business with 150 units. I would love to one click push a ROS and Firmware update. Push wifiwave2 as needed, whatever.

Really though, at minimum, remote management behind NAT is the biggest PITA at the moment

[spippan]

Really though, at minimum, remote management behind NAT is the biggest PITA at the moment

you could setup a public CHR to which your to-be-managed tiks would connect via ovpn/sstp/wireguard (with a /30 subnet for instance or framed /32), establish a routing-protocol for automatic route exchange with route filters in place so every device has its own Lo0 address and you also connect to that CHR and route your management subnet (in which the Lo0 address reside) to that CHR

so NAT is no problem any more in fact. and with sstp/ovpn on tcp port 443 it even would be possible to maybe deploy tiks in china xD

[realmark]

Following up, the devices should reach out to the controller, not the other way around. Push metrics to the controller is a good start. Controller keeps a git (or other version control database) for configs, and endpoint devices pull the latest config.

Yes, you could still potentially compromise the controller and use it to deliver compromised configs, but the controller has no inherent path to the sites. And in metrics-only mode (config pull disabled / manual on the device), there’s zero path in. We’re using this sort of strategy with a number of clients for whom data security is paramount. We provide remote monitoring of systems, but have provably zero access to the data within. A breach of our systems cannot get back to the customer, but we still provide a lot of value from metrics monitoring and analysis.



I know this sorta gets into the user facing end of things, but Aruba is doing a nice job of the config part with NetEdit. Other guys keep mentioning RANCID. This is the opposite flow. Config builder / git repo at the head end, and network devices pull their configs down.

[lcohen999]

Really though, at minimum, remote management behind NAT is the biggest PITA at the moment

you could setup a public CHR to which your to-be-managed tiks would connect via ovpn/sstp/wireguard (with a /30 subnet for instance or framed /32), establish a routing-protocol for automatic route exchange with route filters in place so every device has its own Lo0 address and you also connect to that CHR and route your management subnet (in which the Lo0 address reside) to that CHR

so NAT is no problem any more in fact. and with sstp/ovpn on tcp port 443 it even would be possible to maybe deploy tiks in china xD

I can also zerotier each router with my own hosted ZT and access it that way. Point being...it is an extra step which would be nice to avoid

[robins]

We are a large 802.11 WISP with a managed BYOD wireless client service. We are in a phase of transitioning heavily to MikroTik products for our backhaul. We are interested in your CAP access points to install in homes and businesses, but your CAPsMAN controller currently has an issue which is fatal to our use case:

If the controlled AP loses contact with the CAPsMAN controller, it kills the radio and stops broadcasting entirely. This will suffice for local CAPsMAN management, but we require central management of thousands of separate sites (multi-tenancy). In order to do that with your controller with uninterrupted failover, we would need to run VRRP and script an automated sync. All our last-mile wifi services would be at unacceptable risk of total catastrophic failure. Our current vendors’ APs cache their config and continue broadcasting with its last known settings if it loses access to a central controller.

Is there an effort to add a feature like this for multi-tenancy and non-distributed CAPsMAN control?

[mjezierski]

In regards to the ‘Cloud’ solution.
Not everything I have in now >100 devices touches the public internet.
I would prefer a solution I can spin up on a Virtual Machine in a closed environment.

I understand that other people could benefit from a cloud controller, but not in my current use case.

^^^^^^^^^^^^^^^^^^^^^^^
THIS!!!

[nkourtzis]

In regards to the ‘Cloud’ solution.
Not everything I have in now >100 devices touches the public internet.
I would prefer a solution I can spin up on a Virtual Machine in a closed environment.

I understand that other people could benefit from a cloud controller, but not in my current use case.

^^^^^^^^^^^^^^^^^^^^^^^
THIS!!!

…Or the best of both worlds: a cloud-based service, with the option for a local “proxy controller”, located on the LAN edge.