mikrotik-rds-csi
I built a Container Storage Interface Driver (CSI) for dynamically allocating file based PVC's on an RDS from OpenShift!
![]()
Check it out! Feedback welcome.
I'd love
for Mikrotik to take this and sponsor its further development.
mikrotik-rds-csi
I built a Container Storage Interface Driver (CSI) for dynamically allocating file based PVC's on an RDS from OpenShift!
![]()
Check it out! Feedback welcome.
I'd love
for Mikrotik to take this and sponsor its further development.
Really nice project indeed!
One thing I’d be interested in is how you plan to handle fencing/recovery if a node disappears while an NVMe/TCP connection is still alive and the same RWO volume gets scheduled on another node. Making sure there can never be two writers seems important once this moves beyond the MVP stage.
It might also be worth looking at the RDS NVMe/TCP host authentication/access-control options.
And since these are file-backed volumes, some real-world VM benchmarks would be interesting too, especially latency and random/sync I/O.
Keep up the good work!
@Larsa Thank you for the kind words! ![]()
I have this serving storage to an AMD EPYC 9554 based bare metal cluster, with dual port Nvidia ConnectX-6 Dx 100 gbps NICs (LACP) through a Mikrotik CRS520-4XS-16XQ (Also with 2x downlinks to the RDS in LACP). The RDS has 8x Samsung 990 NVMe drives in a RAID10.
I can get 2.1 GBps Read / 1.9 GBps Write sequential with 1 MB block from a pod with a mounted PVC, and I don't remember the IOPS count with smaller block sizes. By default the CSI contains helpful build sripts for an XFS RAID to serve the files. Mikrotik is explicit that the performance shouldn't be expected to be near what serving raw block devices would be when you serve files. I haven't tried to see if EXT4 would be better or not.
If I have time I'll do a full spread of benchmarks (They are built and included in the Git Repo with a test Pod deployment)
That’s actually a pretty impressive result considering the whole stack involved, especially with file-backed volumes on the RDS. Around 2 GB/s from a pod through CSI, NVMe/TCP, XFS and RAID is definitely respectable.
Looking forward to the full spread of benchmarks!