 CHR using Apple Virtualization & QEMU via macOS UTM

RouterOS Virtualization
61

My plan is to build three version:
UTM with Apple (Intel)
UTM with QEMU (Intel)
UTM with QEMU (ARM64)

I also want to figure out how to update the UUIDs in the .plist per build too. I’ll hopefully get it this week.

But I’ve gone from waiting 5 minutes for VMWare Fusion to just start… to going from reading the “new release thread” to have CHR running in a 2-3 minutes for entire process. I’m some AppleScript away to UTM from automating some testing.

If anyone had input on a good default mem/cpu’s to use, I’m open. I left UTMs defaults, but perhaps CHR’s min specs.

62

You can try to automate with PlistBuddy (/usr/libexec/PlistBuddy) in script. PlistBuddy is macOS provided CLI tool for editing .plists

63

Oh plenty of tools on macOS for plist. Issue is the UTM ZIP files are built at GitHub Action “in cloud” on Ubuntu… And thus no PlistBuddy (or even defaults) on Linux…

I’m sure some sed could work… but the nitty-gritty details are I wanted to try Apple’s new config language, pkl-lang.org since that does read a plist on Linux, and supports “templating” so the GitHub build can provide the variables to pkl to generate a new .plist as part of build. For automation, it’s actually more useful to set serial to a “Psuedo-TTY”, and stuff like cpu/disk. And pkl script should work both at GitHub and a local macOS to “clone” as UTM VM. But I actually only got as far there wasn’t some thing similar to plistbuddy for linux… And… there are really no pkl-lang examples, other than: https://pkl-lang.org/main/current/language-tutorial/02_filling_out_a_template.html.

64

Ah, forgot it’s under Linux, maybe with plistutil for conversion + some XML edit tool like xmlstarlet? plistutil comes with libplist-utils package. plistutil is not even needed if .plist is not in binary format or it can be used to convert to JSON format and such can be modified with jq tool (instead using xmlstarlet) and then convert back to .plist XML format.

65

I finally got around to streamlining this. It’s actually in a new GitHub — mikropkl.

Since it has a better framework, and I plan to retire the chr-utm version at some point - since mikropkl has same Apple, plus now QEMU images. Basically I can create ready-to-use UTM virtual machine based on a “manifest” file written in pkl (https://www.pkl-lang.org) that deal with UTM’s .plist and other packaging details. So creating any new variants of QEMU or Apple CHR is now pretty straightforward - it just a new half dozen line file in ./Manifests. The Makefile and build will do the rest.

For example, I now have “ROSE” images that include 3 small 100Mb disks that can be used to test ROSE features, without having to manually add disks to config. In the new scheme, QEMU images are always based on unmodified CHR images from Mikrotik (while Apple images still require processing to convert ext2 to fat inside the .img file). And all VM packages come standard with TWO serial ports, one going to UTM window console, and another wired to some pseudo-tty device (i.e. a “file” like /dev/stty00X). But still Apple Virtualization support is limited to Intel.
mikropkl-packages-in-utm.png
FWIW, even the icon style (solid or line) and color are customizable in the manifest file, and selected for now based on gaudiness - in solidarity with the WinBox4 pro-color&contrast movement.

I have tested all these images on Intel macOS with UTM, but not Apple Silicon based macOS. If anyone want to try the aarch64 QEMU image on ARM-based Mac, and report back that be good info to know if it just worked (although it’s based on @optio’s example above) - but it possible some tweak may be need for ARM macOS, dunno.

The download CHR images for UTM are here: https://github.com/tikoci/mikropkl/releases/tag/chr-7.18.1

With the following images currently being built:

CHR.Apple.x86_64.utm.zip
CHR.QEMU.aarch64.utm.zip
CHR.QEMU.x86_64.utm.zip
ROSE.QEMU.aarch64.utm.zip
ROSE.QEMU.x86_64.utm.zip

66

Great work. I will try them on Silicon Mac when I find some time and report back.

67

And the UTM packager now deals with versions, so there is a 7.19beta4 release, with QEMU using “real CHR” (and the Apple Virtualization version still needs custom FAT boot image):
https://github.com/tikoci/mikropkl/releases/tag/chr-7.19beta4

Also, renamed the generated package/ZIP names to include the version. While long, you can rename it once import or alias’ed in UTM.

I’ll probably split the new UTM package into a new topic at some point. But to use the new “mikropkl” UTM packages of CHR for any macOS, it’s basically:

  1. Install UTM (via “brew install --cask utm” or https://mac.getutm.app) if you don’t have it.
  2. In Terminal or in Safari’s URL, use the correct UTM URL from https://github.com/tikoci/mikropkl/releases - for example for Apple Silicon with 7.19beta4, with 3 spare disks, can be launched from Terminal using:
open 'utm://downloadVM?url=https://github.com/tikoci/mikropkl/releases/download/chr-7.19beta4/rose.chr.aarch64.qemu.7.19beta4.utm.zip'

(note the ‘single quotes’ around the URL)
3. UTM will open and prompt to download and add the new CHR
4. If accepted, hit the “Start” icon on “rose.chr.aarch64.qemu.7.19beta4” shown on left in UTM
5. A terminal window will appear connected to CHR. Login is “admin” without password, speed in free version is 1Mb/s.
6. After login, “/ip/address/print” will show you the “shared” network IP, and you can use that in Winbox or Webfig from the local computer only.

All “.qemu.” version use the same CHR .img from Mikrotik. The “.apple.” version uses one that get repackaged by GitHub to convert ext2 to FAT (as well described in 60+ posts above).

The ready-to-use QEMU images for UTM on macOS - that use “real” image – are what’s new here.

68

Hi!

I tried to run those machines from https://github.com/tikoci/mikropkl/releases (chr.aarch64.qemu.7.19beta4.utm and rose.chr.aarch64.qemu.7.19beta4.utm - used the utm:// links for both) on my Mac Mini with Apple M2 Pro processor and on anything I try I get only such error.
utm chr error.png
What am I missing here to run it successfully.

69

Thanks for testing! I had not tried it on Apple Silicon, I’m a bit of a dinosaur on that front and still use Intel Mac. So the aarch64 images are based on @optio’s working configuration provided a few post above.

In diff’ing @optio’s config from the GitHub generated config, I suspect the issue is the generated config.plist needs to have both setting below set to “false” - while GitHub/pkl are using “true”:

		<key>Hypervisor</key>
		<false/>




		<key>RNGDevice</key>
		<false/>

Also, the CPU type is wrong, it’s using “default” which may not be “cortex-a710” which is what @optio uses. So this also need to change:

		<key>CPU</key>
		<string>cortex-a710</string>

I’ll re-generate an image with those settings for 7.19beta4 today, to exactly match @optio’s configuration which is known to work.

You should be able to edit these “manually”:

  1. In UTM, right click on the CHR machine, select “Show in Finder”.
  2. Close UTM app to allow modifying the configuration without potential conflict.
  3. In Finder, right click on same machine, select “Show Package Contents”
  4. Select “config.plist” and right-click and open in an editor. If there are no default editors, select “Other”, then find “TextEdit”.
  5. Change the above RNGDevice and Hypervisor from to
  6. Change the above CPU from “default” to “cortex-a710”
  7. Save file and close editor.
  8. Restart UTM and try machine again.
70

@Amm0, yes issue is with enabled QEMU Hypervisor for aarch64 CHR, it should be disabled, I had same issue as @monk when I was setting this up before, for X86_64 doesn’t have affect since it’s different architecture. Regarding RNG device, it is also working when it is enabled, maybe it will approve performance when enabled for aarch64 CHR. Also aarch64 works on cortex-a710, cortex-a72 (Default) and cortex-a76 CPU series.
machines1.png
machines2.png
Also QEMU Hypervisor can be disabled over UTM GUI for VM setup:
hypervisor.png
When rebooting aarch64 CHR noticed this error, it’s working anyway, could be CHR ROS image issue:
reboot-aarch64.png

71

I re-built the 7.19beta4 and 7.18.1 images to exactly match @optio’s plist, so cortex-a710 and no RNG for now. I do think the RNG shouldn’t be a problem. But I recall the “no hypervisor” part in thread, but missed that in diff’s.

Images with above changes:
https://github.com/tikoci/mikropkl/releases/tag/chr-7.19beta4

Or directly to ROSE + QEMU + 7.19beta4 for Apple Silicon (aarch64) from Terminal app:

open 'utm://downloadVM?url=https://github.com/tikoci/mikropkl/releases/download/chr-7.19beta4/rose.chr.aarch64.qemu.7.19beta4.utm.zip'



Yeah I don’t know how the CHR software ID is generated, nor if it’s correlated with UTM “machine identifier” UUID in config. Something could go wrong there, and “cloud lic” part is some clue I think. I have tested with all images running on Intel. If running code locally, with “make utm-install; make utm-start” which add all built images and starts them with AppleScript from Makefile)


EDITS: I changed the URL to releases to include a “chr-” & rebuilt 7.18.1 (stable) and 7.19beta4 (testing) to use revised “no hypervisor” and other settings for Apple Silicon

72

This new image works like magic! Thank you so much! :slight_smile:

73

Good to hear. Yeah it pretty cool to see them just open up, without going through dozen steps.

CHR 7.19beta4 x 4 (Tiled Terminals).png
I could write a bit more notes on using them, but happy to hear it works since I can’t easily test aarch64 macOS.

74

Nice work Amm0. I opened the URL on my Mac Studio and it came right up. I’m able to browse into and it works well. Just need to license it for more bandwidth.

Also testing CHR on an Ampere box (Q80-30) and loving all this ARM64 stuff.

75

FWIW, I updated the README.md with a lot more information. I’ll summarize some key things below.

While all generally works fine with Wi-Fi adapter… it is not the quickest path. This is NOT a CHR thing, as it effects all virtualization on Mac. So wiring up a Ethernet dongle is recommend if you’re looking to see more native-like performance from the network interface. Or… if anyone wants to learn about /queue, bring up 2 x CHR images, both “bridged” to a Wi-Fi adapter, and run a /tool/bandwidth-test…you will quickly see variability. TL;DR is if you are using the Wi-Fi interface, using fq_codel queue in UTM CHR does clean up the performance somewhat.

In particular on further automation options, which I left unexplored. But the same utm:// URL scheme has a few more simple commands for automation. So you can use some

open 'utm://start?name=chr.aarch64.qemu.7.9beta4'

to start a VM. UTM’s docs are good here in describing some simple usages on macOS with Automator and Shortcuts:
https://docs.getutm.app/advanced/remote-control/
So you can in a more few steps, you can be starting/stopping CHR from Siri if desired. And the same Automator and Shortcuts can be used with the REST API to go beyond starting/stopping in those tools.

Also, it now trivial to add new UTM CHR VM configurations for download, so if anyone wants a more “customized CHR”… feel free to post the suggested configuration. I added the ROSE images to both test the “customizability”, but also it handy to not have to modify UTM setting after download.


The current images all default to “help.mikrotik.com CHR recommandation” memory of 1GB and 2 CPU cores. On a Mac Studio, you might be able to more generous. :slight_smile:

On licensing, I added a section in README.md about it. But they do offer a 60-day “trial” mode. So if you have/create an account at www.mikrotik.com/client, then to enable a 10Gb/s license on any CHR, the command is just:

/system/license/renew account=$myaccount password=$mypassword level=p10

and enabling it does not need a credit card or anything. @normis has pointed out the only restriction is upgrade is not allowed.

Since I think a primary use case be testing… you would not want to upgrade & likely use a new version before 60-day even. Now for long-term use, yes, you should buy a license. But I cannot see Mikrotik objecting to MORE people testing the “beta” or “rc”, or new features like “ROSE” - the later @normis is also asking folk to “test” – but how was no described. So the “ROSE” CHR image published, at least for macOS, allow someone to try the ROSE features safely. But I do not speak for Mikrotik, and not a lawyer on licensing rules.

But.. essentially, it’s one command and your password to move it from “free” to “trial” mode.


While the underlying GitHub re-packaging is as transparent as possible, It is worth repeating a previous commentary:

And to clarify this part for the “mikropkl” images:

For the chr..qemu VM downloads*, all use unmodified CHR .img files download as part of a GitHub Action. Anyone curious can see the build logs here: https://github.com/tikoci/mikropkl/actions.

For the chr..apple VM downlaods*, these use images that are originally download from Mikrotik.com, but have a script changes partitioning table (as discussed in dozen of post above). Specifically, the .apple. images on mikropkl’s GitHub, use CHR images build in another project, tikoci/fat-chr, with those build logs here: https://github.com/tikoci/fat-chr/actions

At the end of the day, a lot of trust is placed in Microsoft / GitHub, since I cannot make that more transparent than it already is… Flip side is GitHub does have various security scanning tools to look for possible supply chain attacks that are enabled in tikoci projects. And if the supply chain or other attack was within Mikrotik’s image (e.g. in their build process), nothing to be done about the risk either, nor any transparency on that process. It’s a tricky world on these things.

And as alternative, it is totally possible to build everything locally. The README.md describes how to this in Build locally on macOS section.

Please comment if anyone has idea to make the process more secure.

76

For anyone messing around with Apple Virtualization and ARM-based Macs, I saw this on YouTube today:
https://www.youtube.com/watch?v=r3wpXhL3iA0
while about Slackware linux, it was interesting to see the kernel options he used aarch64 / “Apple Silicon” with Apple Virtualization. One thing I noticed was using console=hvc0 as a kernel option for Apple Virtualization on aarch64. And there was another article about Arch and Apple Virtualization that also suggest using “console=hvc0” too.

At some point I’ll break down and get a new ARM Mac, since I still think it’s possible to get RouterOS to run on Apple VZ using right ARM64 image or options. So mainly cataloging the clues.