I have an existing preformatted disk crypted with cryptsetup (metadata version 1) and containing an ext4 filesystem with data on it. I’d like RouterOS to decrypt the volume and mount the filesystem (so I can use that, for example, for an SMB share). All these pieces are supported through the ROSE feature already. Though, I can only see options to format the disks afresh with a new filesystem. Is there an option to import existing preformatted disks without actually losing existing data?
Even with “plain” file systems (e.g. ext4) it’s sometimes necessary to reformat drive (possibly due to lack of support for some type of partition table or some such). But with encrypted drives I’d say it’s even more necessary to reformat drive … it would be logical to me that encryption key for drive(s) is completely private to OS who started to encrypt it … the whole idea behind drive encryption is that plugging drive to another system should not give access to contents. And hiding encryption key (as good as possible, hopefully to extent which prevents from extracting it from OS) is a very good thing.
I’d be very disappointed if things can be done in ROS easier than this.
Well, I know the encryption key as that is what I set when I formatted the drive earlier on a Linux distro. I think from this perspective, it doesn’t make any difference whether I use the existing key and explicitly provide that for ROS to use it or whether I format the drive providing the (possibly same) key.
The ROS uses Linux kernel underneath anyway and I assume the very same tools underneath as any usual Linux distro (the usual dm-crypt driver and ext4 driver in Linux kernel, the cryptsetup, mkfs, mount and other utilities in userspace), so it has all the resources necessary to see the device, its filesystem and the content the same way as when I would create a crypted device and a filesystem on it on one Linux distribution and then move the disk to another installed distribution.
The changes in properties and representation of the partitions (basically taking into account the two kinds used in 99% in Linux world - DOS type and GPT type), crypted devices (if it’s versioned standard like LUKS) and filesystems (if it’s standard one like ext4) doesn’t change so much that it would require reformatting. Usually, these standards are created so that we can move the disks around without a need to reformat if changing the machine and/or the kernel version. Unless there are incompatible changes, but in that case, the kernel would simply not accept such device and that error can be simply returned to the user. Such incompatible changes are always versioned then.
I just need to use existing drive created with standard and basic tools found on any Linux distro. Sure, I could format a new drive and then copy across the terabytes of existing data, but I simply don’t have a spare drive to do that.
So I think this is just about exposing this functionality to users. Honestly, I was a bit surprised I couldn’t find this possibility, it seems very basic functionality to me when something like ROSE storage is introduced. But maybe it’s just planned for the future? With the "need to be formatted on ROS " being first to avoid support burden at the beginning (the ROSE storage is still relatively new). I don’t know, that’s the reason I’m asking…
Here you assume wrong, so nothing you wrote after this point doesn’t necessarily reflect reality.
ROS indeed runs linux kernel … but if we can believe MT guys around here (and I don’t see why we shouldn’t), similarity with “usual Linux distros” ends here. MT developed their own “busybox” and implemented many standard protocols from scratch (e.g. ssh, IPsec, etc.). I wouldn’t be surprised if their ROSE was full of their own developments and/or stripped-out functionalities (they really strive to squeeze as much functionality as possible in 16MB of storage space, available on some of their devices … even flagship ones).
Just as illustration: “usual Linux distro” for x86_64 ships with kernel file of almost 15MB and additional 72MB initrd file. Compare it to 16MB available, which needs to hold also userland utilities.
OK, I can understand that the situation with MT devices is a bit different when compared to usual desktops (mainly when it comes to available storage/memory size etc.).
In that case, the Linux kernel is GPL licensed code and as such we should be able to see the sources with modifications from MT (is there an official repository anywhere?). Then we should be able to compare that to the vanilla Linux kernel drivers (for dm_crypt, ext4, partition handling) and see directly if there are any major changes. I’ve no idea what is used in userspace as that is all closed source unless they reuse existing open source tooling (the cryptsetup, mkfs, mount) where the license would also mandate them to reveal the sources. Would be great to know though.
I guess at this moment, interesting information for me is whether MT is changing particular drivers related to storage and how. And then if they use standard userspace tooling for storage or they use their own homebrew ones (or some exotic minimized ones), if that is possible to reveal. I’m not actually interested in network side of things here (where, logically, MT would probably be doing lots of their customizations), just the storage. And what are the arguments for not supporting pre existing formatted devices. But that would need to be answered by someone from MT directly. Just interested…
(However, the output that is logged when formatting a disk with a filesystem looks just like the output from the usual mkfs tools from util-linux sources. That’s why I assume that the standard tooling is used.)
It’s closed source, so you cannot diff the modules.
But you need ROSE installed for encryption support, and drives need be Opal complaint: https://help.mikrotik.com/docs/spaces/ROS/pages/259031065/ROSE-storage#ROSEstorage-Self-EncryptionDrives
Does this encrypted disk mount on another plain Linux system (other than the one it came from)?
I don’t think that MT linux kernel contains many changes … apart from some specific device drivers and some patches. I think it’s more about missing features. E.g. dm kernel support can be in modules … and MT might decide not to include them. Or they might decide to omit support for GPT partitioning. Etc.
What I’m trying to say: don’t expect to see each and every functionality, available in usual Linux distros. ROS provides what it provides, nothing more.
And my personal opinion: with your attitude you might get better off if you simply forget about ROS.
Yes, I do have the ROSE package installed. Also, I’m using software encryption here (dm-crypt), not the “hardware” SED (self-encrypting-drive), so for that, we can use any drive (block device). It works for me well if I format the encrypted drive and create a filesystem on a test drive.
The sequence of steps I did when I created the volume was (considering the “sda” is the device):
1.
cryptsetup luksFormat /dev/sda
Then opening the crypted device with:
2.
cryptsetup luksOpen /dev/sda mycrypteddevice
Then creating the filesystem on it with:
3.
mkfs.ext4 /dev/mapper/mycrypteddevice
Then mounting with:
4.
mount /dev/mapper/mycrypteddevice /mnt/mymount
-
Then using the device, writing data to it, reading data from it…
-
Then unmounting it with:
umount /mnt/mymount
- Then closing the crypted device with:
cryptsetup luksClose mycrypteddevice
Now, I can move the device to any other Linux distribution/installation (with dm-crypt driver in the kernel and cryptsetup and mount tools installed) and I can open (step 2.), mount (step 4) and use the device (step 5) as well as cleanly deactivate it (step 6 and 7).
I’m just curious if it would be possible for ROS to skip step 1 (crypted device initialization) and step 3 (filesystem creation) and only open the crypted device (step 2.), mount it (step 4) and make it usable that way. I wouldn’t mind, if for any reason (e.g. some cryptography algorithm not included in the kernel) the opening of the crypted device fails. It’s just that ROS would try to use the existing one if possible. The ROSE can already format a new drive for encryption and create an ext4 (or other) filesystem on it, so the drivers are already there, surely including the basic encryption algorithms (which are probably already used on my existing disk because I didn’t request cryptsetup excpliticly to use anything exotic, just defaults).
It seems that ROS does not provide this as of now, so my question is directed whether there are any obstacles to do so. There’s the point mkx made about the restricted environment that needs to be used for ROS so that it fits the relatively small size of memories used on MT devices. But I wonder if there’s anything else in play.
Sure, certain things can be omitted, then it’s probably about documenting these restrictions (if this is not the right channel, then I may try to direct a question directly to MT support to confirm).
Actually, I’m a happy ROS user for over 13 years and I’m very satisfied with both MT products and their software. Recently, I’ve noticed there’s the ROSE package that is relatively new so I gave it a try and found that existing preformatted devices can’t be used, so I was just trying to discuss this here.
The thing about GPL licensing of the kernel code and associated userspace tooling and associated supply of the source code with changes, as the license itself demands for such code to be reused and/or modified further, is also just my curiosity. It’s the most straightforward way for me to see the changes and spot the possible restrictions. It’s because I’m working as developer in open source environment and this theme is very close to me. But I guess those sources are simply on demand, if I asked MT support. I didn’t mean to be rude. Sorry if it felt that way.
@mkx is correct, RouterOS basically uses the Linux kernel, but not the Linux tools. So AFAIK there is only the hardware encryption with OPAL that is support.
You can file a feature request at https://help.mikrotik.com & see what Mikrotik says. They have been adding things to ROSE, and some software encryption support is not a unreasonable request.
Hmm, are you sure? Because the documentation mentions https://help.mikrotik.com/docs/spaces/ROS/pages/259031065/ROSE-storage#ROSEstorage-Crypted which is exactly the software-based encryption. When I tried that, it passed just fine (unless the command “add crypted-backend=usb1 encryption-key=strong_key slot=crypted-usb1 type=crypted” was a no-operation one - truth is that I haven’t unpluged it from MT and tried on a Linux machine to double check it actually did something, will do…).
The SED is documeted separately at the very end of the documentation: https://help.mikrotik.com/docs/spaces/ROS/pages/259031065/ROSE-storage#ROSEstorage-Self-EncryptionDrives
Also the video tutorial https://www.youtube.com/watch?v=ONjpJ6rZE48 mentions that you can choose between dm-crypt (the software-based encryption) and OPAL/SED and both are supported. But yeah, I’ll check details with MT support.
Nope. 
But they don’t say version etc, so yeah compatibility might be tricky but dunno obviously… So worth a ticket.