Ceph, an open source distributed storage system, has been ported to run natively on Windows Server to provide improved performance over the iSCSI gateway and better integration into the Windows ecosystem. Key components like librbd and librados have been ported to Windows and a new WNBD kernel driver implements fast communication with Linux-based Ceph OSDs. This allows access to RBD block devices and CephFS file systems from Windows with comparable performance to Linux.

1. Ceph on Windows
Alessandro Pilotti | CEO @ Cloudbase Solutions

2. Why porting Ceph on Windows?
+ Ceph is the most popular distributed OSS storage solution
+ Windows Server has a large market share, esp. in the enterprise
+ Ceph iSCSI Gateway performance and bottleneck complains

3. How did it happen?
+

4. Architectural goals
+ UX on Windows as close to the Linux as possible
+ Must integrate well in the Windows ecosystem
+ Outperform the iSCSI gateway and get close to Linux native
performance
+ Secure
+ OS: Windows Server 2016, 2019 (and Windows 10 for dev)
+ Non goal (yet): porting OSD to Windows

5. RBD on Windows - Architecture
WNBD.sys
rbd-wnbd.exe
WNBD.dll
Librbd
librados
OSD
Linux Windows
Mapped
Disk
OSD

6. Librbd, librados and the CLI
+ Ported librbd and librados and relevant Ceph CLI to Windows
+ RBD mappings are managed by rbd-wnbd (can be invoked via rbd):
• rdb device map
• rbd device unmap
• rbd device list
+ Mappings managed at boot by a Windows service
+ rbd and many CLI tools work like on Linux

7. WNBD
+ Windows Kernel driver (Virtual Storport Miniport)
+ Implements two communication options
• NBD via TCP/IP, compatible with any Linux NBD server, or
• local userspace/kernel channel (much faster!)
+ LGPL 2.1
+ https://github.com/cloudbase/wnbd

8. Windows Ceph configuration files location
+ Default location (equivalent to /etc/ceph):
• %ProgramData%Ceph
• E.g.: C:ProgramDataCeph
+ ceph.conf and keyring files can be copied from a Linux node

9. CephFS
+ Windows lacks natively FUSE or a similar userspace filesystem driver
+ CephFS on Windows leverages Dokany
+ Usage example: ceph-dokan.exe -l x
+ Evolution of previous ceph-dokan community work

10. Windows installer
+ Seamless integration in the Windows
ecosystem
+ Can be automated
+ Open source
+ Continuous builds available:
+ https://cloudbase.it/ceph-for-windows/

11. Hyper-V and Ceph
+ Hyper-V can now access RBD images / disks
+ This includes also booting a VM from a RBD image:
• New-VM VM1
• # Retrieve disk number with: rbd device list
• Add-VMHardDiskDrive -VMName VM1 -DiskNumber 1 –Passthru
• Start-VM VM1

12. Performance
+ Huge improvement over the iSCSI gateway
+ Outperforms even both rbd-nbd and krbd on Linux in our tests!
• WNBD protocol implemented with DeviceIOControl
• Improved IO worker threads allocation
+ Details:
• https://cloudbase.it/ceph-on-windows-performance/

13. Demo

14. https://docs.ceph.com/en/latest/install/windows-install/
Get in touch
@cloudbaseit
https://cloudbase.it/ceph-for-windows/
http://ask.cloudbase.it