Hi Mikrotik Forum,
is it possible to run ceph (OSD) as Container on a RDS an “share/map” the NVMe Drives to it?
Best Regards!
You’d be better off running ceph on a separate Linux box and using iscsi or nvme-over-tcp to export the individual drives to your ceph host(s). MikroTik’s container implementation doesn’t give the user raw access to drives.
But then you’d stand little to gain if the RDS2216 went away (crashes, reboots, software updates) since all the drives would also go away.
Thanks for your message!
Do you have any idea how to make two or three RDS’s as a Cluster, if one device fails or install firmware, the storage is still up?
At this stage, that is up to the capabilities of your hosts. Since what he’s talking about is tied with the block storage bullet points he’s just mentioned (iSCSI and NVMe), it seems to me that exporting the raw drives to hypervisors or directly to the VM’s themselves would be the implied way to create the type of redundancy he’s mentioning. Nothing that ROSE offers at the filesystem level is going to be real-time enough for an HA cluster, and they don’t currently offer any kind of built-in HA synchronization at the drive level.
One way to accomplish this would be to have two or three RDS2216’s export their drives via NVMe to three Ceph hosts, which the Ceph hosts would use as OSD’s. This is not a whole lot different than exporting iSCSI LUNs from a NetApp or similar storage box. This creates three (or four) layers: hardware host, storage protocol host, virtualization host, and application host (with virtualization and application often being the same hardware).
I’ll chime in on this. With the changes to containers (device passthrough, KVM) coming up in 7.20, we’ve been able to run OSDs under QEMU-KVM.
On a slightly related note; running QEMU virtual machines with an RBD root works, too.