I was a bit surprised to find that the L009 uses 32-bit RouterOS rather than the 64-bit version that should be native to the platform. For the most part, it doesn’t matter, except for encryption acceleration. Does the 32-bit ZeroTier package take advantage of the hardware acceleration provided by the 64-bit processor, or is it reduced to software encryption only?
Unfortunately still software encryption on all platforms. Hopefully it will be addressed in future releases of ROS.
Hm. ZeroTier has had hardware acceleration on the ARM64 platform since version 1.6. RouterOS’ package is currently at 1.10. Did MikroTik actively disable this on their build?
Only AES-NI.
And why would they do that ? If it can be done, they will add it.
Nope. From the 1.6 release notes:
AES hardware acceleration is not yet supported on 32-bit ARM, PowerPC (32 or 64), or MIPS (32 or 64) systems. Currently supported are X86-64 and ARM64/AARCH64 with crypto extensions.
Since the L009 has an arm64 CPU, why would they not be using 64-Bit RouterOS on it to begin with?
Then their faq page is inconsistent.
https://docs.zerotier.com/faq/
While ZeroTier will operate on very low-power hardware (e.g. a single core 32-Bit ARM running at 600Mhz without AES hardware acceleration (AES-NI)), its performance will suffer.
That’s a question unanswered from the very first time this device was releases.
EDIT: or was it ?
http://forum.mikrotik.com/t/32bit-os-on-64bit-cpu-architecture/169613/14
But given that reply, why 64 bit on RB5009, AX2, AX3, … ? They don’t have 2Gb RAM.
That reply from Normis is confusing, we regularly use 64-Bit lightweight OSes (which is what RouterOS is, it’s not Debian or Ubuntu) on 64-Bit embedded devices with 512 MB RAM or less.
Am I missing something here? 32-Bit software (outside MikroTik) is phased out is phasing out even for low-power 64-Bit hardware.
Be nice if Mikrotik clarified the HW encryption stuff with ZT.
The ZT docs suggest there were issues with 32-bit ARM to @DarkNate’s point:
ARM
If you’re planning on embedding ZeroTier in a product such as a router, network attached storage or some other IoT application we see that our partners have a better experience with 64-bit ARM as opposed to 32-bit ARM since the core speeds are typically higher and the chips are more likely to have AES-NI.
Now whether 32-bit OS matter IDK – that’s more about memory addressing. Encryption primitives would still be in the processor - dunno if accessible with 32-bit OS. But only MT know.
As far as encryption/ciphers goes including ZeroTier and many others, please correct me if I’m wrong, but isn’t 64-Bit CPU/Kernel/Host OS the De facto industry standard across the board?
Mikrotik marches to their own beat. I’d like to think they tested it.
While dunno know enough about ARM assembler to say much (nor IPQ5018’s specifics), ARM’s docs suggest encryption extensions are in the 32-bit:
https://developer.arm.com/documentation/ddi0501/f/programmers-model/register-summary
It’s kinda hard to check if ZeroTier/etc are using them. I’d like to think so… e.g. never seen ZeroTier cause sustained high CPU… now never systematic tested with large traffic volumes either.
I’m not sure why they’re throwing the Intel-specific AES-NI term in there for ARM stuff, but they’re saying that the 32-bit ARM stuff that doesn’t have the AES instructions that they need to accelerate things isn’t going to perform well… hence the question. I suspect that the 32-bit zerotier.npk is assuming a 32-bit architecture and not running with acceleration, but it can’t hurt to ask.
Good catch. Looks like the acceleration is in there for 32-bit code on 64-bit processors even if it isn’t there on the 32-bit units. I’ve got a couple of customers with L009 units now. Time for a speed test comparison with one of them on my RB1100AHx4 and on my RB5009 to see what differences come up.
I got a message from Zerotier a while ago that they are changing the pricing model pretty severely. More than tripling the price.
Have not put anything new on Zerotier since. Waiting to see if they kill my subscription at renewal in March.
But seriously… check their new price structure before designing any solutions around it.
Enter Mikrotik BTH …
(or setup your own WG server and work from there)
Maybe. But you’d need EoIP with WG to mimic ZeroTier. Since more generally BTH/WireGuard has no multicast or Layer2 stuff like RoMON… while ZeroTier does.
But if the BTH desktop clients supported terminating GRE/EoIP as a psuedo-interface on the computer/device, that be very close to ZT.
@gotsprings, I noticed the Reddit discussion speculating about a possible new licensing model but haven’t seen any official statement regarding this. Do you know where to find it?
Thank you, but it looks like a rather old post from Sep '23. I can’t find any statement from ZeroTier regarding a new license model and it seems more like two customers have complained about incorrect license quotes.