I got a python-based ModbusTCP logging module that I intend to run on this hAP ax lite router, which has internal 128MB flash storage (95MB in practic)
I managed to get the image size down to 55MB using podman & alpine, & it runs perfectly on my PC
But the problem is, I still can’t get it to run on the router since the image needs to be put on the router & then extracted there when added, which means at minimum it requires 2x size of the image.. which clearly the router doesnt have.. Anyone got advice? can I just put the .tar on my PC and get the router to extract it via network? or maybe use USB-C splitter so that I could add a flashdisk dangling with the power supply? (is it even possible). Or perhaps I should give up and use the built-in iot package and write a script with it (which only has read-holding-registers function, so much for a “modbus package”)
Ugh I already told my colleague i’m doubtful of this idea and preferred using external SBC, but the guy insists it’s “technically” possible, last time I caved in
I doubt that USB-C connector can be used for anything else then powering the device.
It’s not even mentioned on block diagram (whereas for other devices where I know it works, it clearly is indicated as USB).
I even seem to recall Normis (MT staff) mentioned it somewhere (would have to look for it).
So much for being technically possible.
You could try with ROSE package, that would allow to use network storage but it might become SLOOOOWW. Can’t tell, I never used it.
Another option is to focus on slimming down that container further. 55Mb still sounds like a lot.
E.g. tangent was able to make a container running iperf3 using less then 1Mb.
Another user used alpine with iperf3 and came out on 7Mb.
That podman makes up for 48Mb then ??
Or …
With the time you’re probably already spending on it (and how many of those will be used ??), it might be a LOT cheaper simply to go with another SBC board as base. Or a bigger beefed routerboard to build further on without any of those restrictions. https://mikrotik.com/products/group/routerboard
I messed around with routerOS scripting and managed to made a rudimentary logging script
# Load JSON parser
/system script run "JParseFunctions"; global JSONLoad; global JSONLoads; global JSONUnload
# Read, Parse & Extract individual fields
:local devices ([$JSONLoad "modbus_config.json"]->"devices")
# Unload JSON parser
$JSONUnload
# Loop through each device
:foreach device in=$devices do={
# Get the current device info
:local deviceId ($device->"device_id");
:local deviceAddr ($device->"device_addr");
:local devicePort ($device->"device_port");
:local slaveId ($device->"slave_id");
:local regs ($device->"regs");
# Start building the JSON for the device
:local jsonOutput {"StatusSNS"={"TIME"=" ";}}
# Build device payload
:local payload;
# Loop through each register for the device
:foreach reg,regConf in=$regs do={
# Poll for each register (inefficient, but simple)
:local response [iot/modbus/read-holding-registers ip=$deviceAddr port=$devicePort slave-id=$slaveId reg-addr=($regConf->"addr") num-regs=1 as-value once]
# Append to the payload
:set ($payload->$reg) [:tonum ($response->"values")];
}
# Finalize the JSON for current device
:set ($jsonOutput->"StatusSNS"->"ENERGY") $payload
# Publish to MQTT
/iot mqtt publish broker="mqtt" topic="stat/$deviceId/STATUS10/data" message=[:serialize to=json value=$jsonOutput] qos=0 retain=no;
}
It works relatively good, but still some potential issues/headaches if I develop it further :
Unit conversion / format (integer, unsigned, float, etc)
Endianness
Only holding registers, biggest limitations
Nice, how small did you get it to? I’m kinda new to container, that seems to be pretty advanced honestly..
is it still possible to install 1-2 packages libraries with that image?|
My dockerfile btw (it’s clearly made by a noob sorry )
FROM python:3.14.0a2-alpine3.20
WORKDIR /app
RUN pip install --no-cache-dir pymodbus paho-mqtt
COPY . ./iot-datalogger/
ENV TZ="Asia/Jakarta"
CMD [ "python", "-m", "iot-datalogger"]
just checked again with simple app - you will require a lot out of /usr/lib/python modules.
my docker file ballooned up to 27.2MB
FROM alpine as builder
RUN apk add --no-cache python3~=3.12 curl
WORKDIR /usr/lib/python3.12
RUN python -m compileall -o 2 .
RUN find . -name "*.cpython-*.opt-2.pyc" | awk '{print $1, $1}' | sed 's/__pycache__\///2' | sed 's/.cpython-[0-9]\{2,\}.opt-2//2' | xargs -n 2 mv
RUN find . -name "*.py" -delete
RUN find . -name "__pycache__" -exec rm -r {} +
FROM scratch
COPY --from=builder /bin/sh /bin/sh
COPY --from=builder /usr/bin/curl /bin/curl
COPY --from=builder /lib/libz.so.1 /lib/libz.so.1
COPY --from=builder /usr/bin/python3 /
COPY --from=builder /lib/ld-musl-aarch64.so.1 /lib/ld-musl-aarch64.so.1
COPY --from=builder /usr/lib/libpython3.12.so.1.0 /usr/lib/libpython3.12.so.1.0
COPY --from=builder /usr/lib/python3.12/ /usr/lib/python3.12/
COPY varda_dienas_new.json ./
COPY vardi.py ./
ENTRYPOINT ["/python3"]
CMD vardi.py
i also needed to add libz.so.1, i planned using curl instead of requests to save on size, but it also requires some additional modules to parse python input, i probably also need libcurl as well
Good news! managed to reduce it to 25MB, I used the Dockerfile.scratch-full rather than the minimal since I was overwhelmed (it doesnt even have json)
Still require to add some libraries tho, namely libz, libssl, & libcrypto (probably for paho-mqtt)
It’s not exactly 8MB like yours, but it’s definitely an achievement for me , now it runs on the router with 59MB of 128MB used (53% free, after .tar removed), not bad!
I’ll probably just use the scheduler to run a script that’ll start this container, much simpler than implementing scheduling inside the python itself w/ library (probably gonna inflate it much more)
Dunno if I can reduce it further using the minimal file, guess I’ll try it later when I got time
This has been quite a learning experience, was forced to learn multi-stage build like this (I literally don’t know much about container before) not to mention routerOS scripting
thanks!
Make your container image smaller, you can try tools or methods to improve compression. If your images have files that can be compressed more, tools exist which can help reduce unnecessary file sizes before adding them
)
