Community discussions

MikroTik App
 
poisons
just joined
Topic Author
Posts: 17
Joined: Wed Sep 18, 2013 3:50 pm

Regular docker inside ROS container, what going wrong?

Wed Jul 26, 2023 11:52 pm

What I looking for:
1. Simple regular docker inside CHR.
2. Handy web ui interface for maintaining nginx/mariadb and some useful apps like librenms with sidecar containers, nextcloud and etc.
I found nesty project that can help me with launch regular docker without mounting /var/run/docker.sock from host system(sure, I know that the mikrotik containers not the same as docker).
So, what I do:
1. Create mount point like /disk1/containers/docker.sock /var/run/docker.socks
1. Launch container from docker hub nestybox/ubuntu-jammy-systemd-docker
2. Launch container with portainer
3. All containers work with mount point with docker.socks.
But....when I go to test docker container with something like "docker pull busybox", I got error "can't connect to Unix socket and bla-bla-bla".
Looks like docker don't start inside container....and I stuck.

My question is:
1. Is it possible in theory to use regular docker inside ROS container
2. How to debug why dockerd not start inside nesty container?
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 3647
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: Regular docker inside ROS container, what going wrong?

Thu Jul 27, 2023 12:17 am

Why not just run docker as another VM along side CHR, instead of instead the three-levels of virtualization for your "docker-inside-container-inside-CHR-inside-VMHost" idea... Or just use multiple containers directly, instead of via a large linux image and "docker pull"... I guess I'm missing the value added by all the levels of indirection.

I'm guessing docker doesn't work (even with the .sock trick) since it can't acquire the enough rights to the kernel. e.g. docker itself has to support allow hosted container to use —-privileged and --cap-add – but those would be blocked inside of Mikrotik's container support since there not allowed.

So I view this as a feature, not a bug.
 
tangent
Forum Guru
Forum Guru
Posts: 1446
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: Regular docker inside ROS container, what going wrong?

Thu Jul 27, 2023 7:40 am

Hear, hear. Containers inside CHR atop a hypervisor is lunacy. Put something like Flatcar atop the hypervisor and be done.
 
poisons
just joined
Topic Author
Posts: 17
Joined: Wed Sep 18, 2013 3:50 pm

Re: Regular docker inside ROS container, what going wrong?

Thu Jul 27, 2023 10:48 am

Are you serious? I have a cheap digital ocean virtual machine, it has been running chr for several years. I have enough resources of this virtual machine for all the tasks that I assign to it.
But the implementation of containers in Mikrotik makes me cry, it looks good for a pi hole container, but dancing with crooked rights and persistent data is beyond my strength.
It is impossible to modify the container on Mikrotik, there is no compose, uploading data via sftp will most likely cause problems with reading / writing this data from the container.
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 3647
Joined: Sun May 01, 2016 7:12 pm
Location: California

Re: Regular docker inside ROS container, what going wrong?

Thu Jul 27, 2023 5:22 pm

FWIW, I'm not so adverse to the containers on CHR, especially if paying per-machine. And the ability "orchestrate" multiple container exists only through scripting and certainly less convenient than say a "docker compose".

Your underlying issue is that RouterOS tries very hard to prevent "true" root access, and access is gated through config/policy. So a RouterOS doesn't allow a container to avoid this philosophical design principal. More practically, direct access to raw kernel interface from a container is tricky... since those same interfaces may be used for RouterOS things and start breaking things outside of the container's scope...which won't be good.

It's likely best to think that they add "--security-opt no-new-privileges" to "docker run" (even if they're not actually using docker internal, same idea). The result is some containers won't work (e.g. ones that need root to the RouterOS kernel). So yeah network services like DNS, HTTP, etc shouldn't need root, work fine as you point out.

If you have software that needs root, flip the relationship around so CHR is not main virtualized host, and use Proxmox/VMWare/etc as the instance your hosting, run CHR and Flatcar/Kerbantes/etc under that. You'd still have multiple layers of virtualization, but without the restrictions that come from a root-less RouterOS being at the "top of the stack".
 
tangent
Forum Guru
Forum Guru
Posts: 1446
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: Regular docker inside ROS container, what going wrong?

Fri Jul 28, 2023 5:39 am

Are you serious?

Yes, quite. RouterOS is not a full-fat container engine or an orchestrator. It is bare-bones, thinly-documented, and sparsely-featured.

And that's fine for a router or a switch that merely needs to have an extra feature or two tacked onto the side. If you want more than that, you're using the wrong tool.

RouterOS's container feature is closer to how systemd.nspawn or runc work. It is less like Podman or Docker Engine, much less Kubernetes and the like.

But the implementation of containers in Mikrotik makes me cry

The pain will go away when you stop trying to bash that nail in with the butt of your screwdriver. :)

Who is online

Users browsing this forum: No registered users and 0 guests