Running linux server on ROS7.4 (using docker containers)

pi0 · Sun Jun 19, 2022 10:47 pm

Router OS 7.4 finally added back support for Linux Containers (Docker) and I couldn't wait a minute longer to give it a try! [main topic]

For an introduction, you can follow the Container Wiki Page including a tutorial to enable containers and set up a Pihole DNS server.

I've decided to create a generic container image to be able to SSH into it and play around. It is probably not the best use of Docker containers but gives you a tiny handy isolated Linux server running directly inside the router to try different ideas and do benchmarks without the hassle of externally creating and testing images each time.

Prebuilt containers include common utilities such as vim, curl, speed-test and iperf3 with an enabled open-ssh server. You can install additional packages using apt-get (debian) or apk (alpine) or build a customized version from the Docker files provided.

Here are some screenshots of the Debian and Alpine containers running on my RB5009UG+S+IN.

Step 0: Backup and Requirements

Make sure to backup your router configuration and be aware this is still an experimental feature. And make sure to have:

RouterOS device with RouterOS v7.4beta or later and installed Container package

Physical access to a device to enable container mode

Attached hard drive or USB drive for storage - formatted as ext3/ext4

Step 1: Upgrade and enable containers

Download the latest 7.4beta version of router os + extra packages (container). Then move them to the router using Winbox > Files and reboot to install.

After reboot, enable container mode. You need physical access to turn off or reboot the router to enable container mode.

/system/device-mode/update container=yes

Step 2: Create network

Add veth interface for the container:

/interface/veth/add name=veth1 address=172.17.0.2/16 gateway=172.17.0.1

Create a bridge for containers and add veth to it:

/interface/bridge/add name=dockers
/ip/address/add address=172.17.0.1/16 interface=dockers
/interface/bridge/port add bridge=dockers interface=veth1

Setup NAT for outgoing traffic:

/ip/firewall/nat/add chain=srcnat action=masquerade src-address=172.17.0.0/16

Step 3: Download image

Download the docker image and transfer it to the router.

If you prefer to build the images yourself for a different architecture use Docker files and entry from this gist and build against your router's CPU architecture. I've hosted some prebuilt images for easy use.

We the fetch tool to download into the router directly but you can use your preferred method to fetch the links.

[ARM64] Debian (300MiB): (if you have enough space on an external disk, go for it!)

/tool/fetch url=https://dl.home.pi0.io/mikrotik/docker/debian.arm64.tar dst-path=disk1/images/linux.tar

[ARM64] Alpine slim (7.8 MiB): (super-slim build. you can add other packages using apk)

/tool/fetch url=https://dl.home.pi0.io/mikrotik/docker/alpine-slim.arm64.tar dst-path=disk1/images/linux.tar

[ARM64] Alpine (40 MiB): (a moderate build with alpine base)

/tool/fetch url=https://dl.home.pi0.io/mikrotik/docker/alpine.arm64.tar dst-path=disk1/images/linux.tar

Step 4: Configure the container

Set an env to autoconfigure SSH root password (value is an example. use something more secure!)

/container/envs/add list=linux_envs name=PASSWD value="letmein"

Create persisted data volume (ssh keys and home dir):

/container/mounts/add name=linux_data src=disk1/docker/linux_data dst=/data

Create the container:

/container/add file=disk1/images/linux.tar interface=veth1 envlist=linux_envs root-dir=disk1/docker/linux_root mounts=linux_data hostname=mikrotik

What for status from "extraction" to be "Stopped". It might take few minutes based on your storage speed.

/container print

Finally, start it:

/container start 0

Wait and check for the status to be "running":

/container print

Note: If you have more than one container, the second might not be starting. I'm still trying to figure out why but removing the first one using `/container/stop 0` and `/container remove 0` fixed the problem for me.

Step 5: Connect to the container

You can ssh into the container using `ssh -v root@172.17.0.2`. Use the password defined in earlier steps (default is letmein).

Happy hacking and please share your feedback and ideas in the forum!

pi0 · Sun Jun 19, 2022 10:54 pm

Screenshots of the Debian and Alpine containers running on my RB5009UG+S+IN.

pi0 · Sun Jun 19, 2022 10:58 pm

Benchmarks (on RB5009):

$dd if=/dev/urandom bs=1M count=256 | md5sum 256+0 records in
256+0 records out
268435456 bytes (268 MB, 256 MiB) copied, 3.64405 s, 73.7 MB/s
fe42ea91bd4c480d5cfbc59d750d0409 -

Bench.sh:

-------------------- A Bench.sh Script By Teddysun -------------------
Version : v2022-06-01
Usage : wget -qO- bench.sh | bash
----------------------------------------------------------------------
CPU Model : CPU model not detected
CPU Cores : 4
AES-NI : Enabled
VM-x/AMD-V : Disabled
Total Disk : 234.2 GB (736.7 MB Used)
Total Mem : 994.1 MB (187.8 MB Used)
System uptime : 0 days, 0 hour 38 min
Load average : 0.01, 0.05, 0.09
OS : Debian GNU/Linux 11
Arch : aarch64 (64 Bit)
Kernel : 5.6.3
TCP CC : cubic
Virtualization : Dedicated
----------------------------------------------------------------------

I/O Speed(1st run) : 6.8 MB/s
I/O Speed(2nd run) : 29.5 MB/s

I/O Speed(3rd run) : 9.8 MB/s
I/O Speed(average) : 15.4 MB/s

vps bench:

Total amount of RAM: 994 MB
System uptime: 55 min,
I/O speed: 12.2 MB/s Bzip 25MB: 13.61s Download 100MB file: 66.3MB/s

Busylog.net

Test openSSL speeds (openssl signatures speed)....
Doing 512 bits private rsa's for 10s: 54928 512 bits private RSA's in 9.93s
Doing 512 bits public rsa's for 10s: 639965 512 bits public RSA's in 9.94s
Doing 1024 bits private rsa's for 10s: 10489 1024 bits private RSA's in 9.93s
Doing 1024 bits public rsa's for 10s: 208632 1024 bits public RSA's in 9.95s
Doing 2048 bits private rsa's for 10s: 1557 2048 bits private RSA's in 9.97s
Doing 2048 bits public rsa's for 10s: 58247 2048 bits public RSA's in 9.97s
Doing 3072 bits private rsa's for 10s: 504 3072 bits private RSA's in 9.95s
Doing 3072 bits public rsa's for 10s: 26351 3072 bits public RSA's in 9.76s
Doing 4096 bits private rsa's for 10s: 223 4096 bits private RSA's in 9.97s
Doing 4096 bits public rsa's for 10s: 15132 4096 bits public RSA's in 9.96s
Doing 7680 bits private rsa's for 10s: 30 7680 bits private RSA's in 10.24s
Doing 7680 bits public rsa's for 10s: 4388 7680 bits public RSA's in 9.96s
Doing 15360 bits private rsa's for 10s: 5 15360 bits private RSA's in 10.59s
Doing 15360 bits public rsa's for 10s: 1106 15360 bits public RSA's in 9.98s
OpenSSL 1.1.1n 15 Mar 2022
built on: Tue May 10 18:37:36 2022 UTC
options:bn(64,64) rc4(char) des(int) aes(partial) blowfish(ptr)
compiler: gcc -fPIC -pthread -Wa,--noexecstack -Wall -Wa,--noexecstack -g -O2 -ffile-prefix-map=/build/openssl-ZoudvR/openssl-1.1.1n=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
sign verify sign/s verify/s
rsa 512 bits 0.000181s 0.000016s 5531.5 64382.8
rsa 1024 bits 0.000947s 0.000048s 1056.3 20968.0
rsa 2048 bits 0.006403s 0.000171s 156.2 5842.2
rsa 3072 bits 0.019742s 0.000370s 50.7 2699.9
rsa 4096 bits 0.044709s 0.000658s 22.4 1519.3
rsa 7680 bits 0.341333s 0.002270s 2.9 440.6
rsa 15360 bits 2.118000s 0.009024s 0.5 110.8
-----------------------------
Disk seek rate test (ioping)....

--- . (ext4 /dev/sda1 234.2 GiB) ioping statistics ---
464.0 k requests completed in 1.78 s, 1.77 GiB read, 260.7 k iops, 1018.4 MiB/s
generated 464.0 k requests in 3.00 s, 1.77 GiB, 154.7 k iops, 604.1 MiB/s
min/avg/max/mdev = 2.52 us / 3.83 us / 438.0 us / 2.42 us

pi0 · Mon Jun 20, 2022 12:25 am

(Reserved)

cklee234 · Mon Jun 20, 2022 4:57 am

Thank for the good works.

Any chance to build a x86 image?

lspzj · Mon Jun 20, 2022 10:04 am

Thank you

Looking forward to your x86 works

Mon Jun 20, 2022 10:38 am

why would anyone download image from unknown source when this can be done in few minutes locally and safely and to any supported architecture?

make Docker file with contents -

FROM ubuntu:latest

RUN \
apt-get update && \
apt-get install --no-install-recommends --yes openssh-server && \
echo "PermitRootLogin yes" >> /etc/ssh/sshd_config
RUN mkdir /run/sshd
RUN echo "root:root" | chpasswd
CMD ["sh" , "-c", "/usr/sbin/sshd -D && sh && tail -f /dev/null"]

run:

docker buildx build  --no-cache --platform arm64 -t test_arm64 .
docker save test_arm64 > test_arm64.tar

Znevna · Mon Jun 20, 2022 12:56 pm

That wasn't his intention, he only left the links for the lazy Bob.
He explained that he used this:https://gist.github.com/pi0/a7f734a69d3 ... c862cfa3ac to create the images.
Don't be so mean

And thank you for the guide.

pi0 · Mon Jun 20, 2022 1:23 pm

Thanks for the feedback @cklee234 @lspzj! Publishing x64 and arm 32 images shortly.

> why would anyone download image from unknown source when this can be done in few minutes locally and safely and to any supported architecture?

Thanks for the feedback @antonsb. I understand your concern and fully agree that it can be simply achieved by locally building a docker image (used docker files are in the gist from guide).

But there are several points:

- When you are statically building an image, both root password is Server SSH keys are hardcoded into the image. If image is shared or leaked, one can attempt a MITM attack on the SSH server, considering has access to the pre-generated private keys of the server. Prebuild images I've provided generate them once during container startup and persist them on the router. And root password is also not static.
- Considering the security aspect of downloading images from remote sources, while in general is a valid concern, it is always the case when we support package registeries. Whether using a container registry (such as docker hub which is supported and documented in wiki) or a remote URL there is not much difference. Dockerhub allows any arbitrary users to push into It. (besides i'm not that random

)
- Considering user experience, it is much easier to use a prebuilt and tested image rather than using an external matching to build from scratch. Unless you have plenty of free time it is wasting time to build, test, and maintain images. Plus the fact that simplicity can implicitly misguide to use not secure practices such as the ones mentioned earlier above. And the purpose of this article is basically to make it as easy as possible to try containers not how-to for building images (which is probably worth a dedicated topic). Prebuilt images also provide persistency for storing (more secure) ssh-keys to use instead of initial password and also keeping `/root` files.
- If you are concerned that the isolation of router containers is not enough to run an untrusted container (which I would agree with since router containers are exposed to the network and can leak information by their local network reachability - unlike normal container daemons that are only reachable by their host), i think we should emphasize it in the wiki documentation about security implications of pulling an image from docker registry.

Moving this forward, I will try to move Docker files and build scripts to a public GitHub repository for further transparency and trust and also easier local builds and probably using Github actions. Also planning to work on some new ideas such as auto-starting daemons (like nginx) also probably an automated script to set up the container for router (of course also public and available to inspect)

Mon Jun 20, 2022 2:13 pm

for lazy Bobs, there is remote-dist to dockerhub, where you can be sure that what you see is what you get.

running sshd into container is bad practice from start and is not correct use of containers all-together.

pi0 · Mon Jun 20, 2022 2:32 pm

@antonsb I've mainly avoided docker hub and used CDN since docker hub has rate limits to the personal namespace images. There are no differences in terms of security! and also the fact that you have mentioned in the wiki about ram restrictions for when pulling layers. I Will provide registry images as well btw when moved to the Github repo. It was a quick tutorial to share

> running sshd into containers is bad practice from start and is not the correct use of containers altogether.

Are you sure read the tutorial other than the title?

This is also mentioned there that this is probably not the best use of docker containers but the purpose is quickly trying RouterOS containers without the hassle of image building to try things such as performances and do benchmarks. Containers in RouterOS are in the really early stage and I'm afraid many other things are non-standard for time being. Without a way to try them easily, it is really hard for the community and users to give you feedback about this feature. Even if examples like PiHole work out of the box, we need to try it better.

Mon Jun 20, 2022 2:34 pm

I think Antons just wants to make people wary of trusting unknown people on the internet, nothing personal about your example

Thanks for the effort, Pi!

pi0 · Mon Jun 20, 2022 2:40 pm

Thanks for the kind words @normis <3 and I fully understand the security concerns of @antonsb. No worries. I'm just trying to say, pulling from the hub registry and a URL, have almost the same security concerns and would be more than happy to contribute to this for early detection of possible router container flows (such as local network exposure with untrusted images) and document them.

Znevna · Mon Jun 20, 2022 3:02 pm

[...] you have mentioned in the wiki about ram restrictions for when pulling layers.

Those aren't RAM restrictions mentioned, "main memory" refers to the internal little memory some devices have (as little as 16MB of SPI flash).

Mon Jun 20, 2022 3:45 pm

Thanks for the kind words @normis <3 and I fully understand the security concerns of @antonsb. No worries. I'm just trying to say, pulling from the hub registry and a URL, have almost the same security concerns and would be more than happy to contribute to this for early detection of possible router container flows (such as local network exposure with untrusted images) and document them.

yeah for random containers. But when I use docker hub, I stick with verified images with high community rating.

pi0 · Mon Jun 20, 2022 3:55 pm

> yeah for random containers. But when I use docker hub, I stick with verified images with high community rating.

Maybe something worth mentioning in the wiki? And also to emphasize running trusted code inside containers? (Because otherwise, docker containers are usually assumed to be fully isolated whereas router containers are exposed to the local network)

cklee234 · Thu Jun 23, 2022 3:27 pm

Thank for the good works.

Any chance to build a x86 image?

I try to build a Debian x86 docker image but it never gets start in routers

Znevna · Thu Jun 23, 2022 3:30 pm

In what router?

padow · Thu Jun 23, 2022 6:35 pm

Thx, I already did and works very well, I actually have the pihole container running in the mikrotik, and start running it doing /container/start 0, but when the router reboots I need to start it manually, therefore I need to make and script to do that auto or there is another way ?

/system/script
name="run pihole" owner="admin"
policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon
dont-require-permissions=no run-count=0 source=/container/start 0
/system/scheduler>
0 Autostart Pihole startup 0s /system script run "run pihole"

Thx
Patrick from Argentina

padow · Thu Jun 23, 2022 8:26 pm

the schedule does not run but the script is ok, something wrong ?

script correciton working

{
:log info "Starting System Startup script"
:delay 00:00:20

/container/start 0
}

Znevna · Fri Jun 24, 2022 10:18 pm

/container/start 0

will never work without a previous

/container/print

Because otherwise "0" means nothing. This behaviour is documented somewhere.
The correct way is to start containers by name

/container/print 
 0 name="25d5281b-8e6c-460c-a683-87c21c6e41fd" tag="" os="linux" arch="arm" interface=veth1 envlist="pihole_envs" root-dir=zdisk/containers/pihole mounts=dnsmasq_pihole,etc_pihole dns="" hostname="PiHole"

Like this:

/container/start [find name=25d5281b-8e6c-460c-a683-87c21c6e41fd]

Of course you replace the name with the name of your container.
Cheers.

cklee234 · Tue Jun 28, 2022 11:13 am

Thank for the good works.

Any chance to build a x86 image?

Finally make it work

FROM debian:bullseye-slim
RUN \
apt-get update && \
apt-get install --no-install-recommends --yes openssh-server && \
echo "PermitRootLogin yes" >> /etc/ssh/sshd_config
RUN mkdir /run/sshd
RUN echo "root:root" | chpasswd
CMD ["sh" , "-c", "/usr/sbin/sshd -D && sh && tail -f /dev/null"]

DrGeek · Thu Jul 14, 2022 1:23 pm

Hi everyone !

I want to launch an ndppd proxy in container so I need multiple interfaces, is it possible ?

mehdi1980 · Thu Sep 01, 2022 8:58 pm

as mikrotik not support shadowsocks,does it possible to install openwrt ?

morynet · Tue Oct 25, 2022 1:50 pm

Very very thanks...

only
fetch the links not work ...

Tue Oct 25, 2022 2:48 pm

I appreciate the effort, but the tips in this thread are now outdated and no longer necessary.

Consider using /container/shell instead:
https://help.mikrotik.com/docs/display/ ... sandtricks

Running linux server on ROS7.4 (using docker containers)

Running linux server on ROS7.4 (using docker containers)

Re: Running Linux container server on ROS7.4

Re: Running Linux container server on ROS7.4

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Re: Running linux server on ROS7.4 (using docker containers)

Who is online