Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Using Rook to Manage Kubernetes Storage with Ceph


Published on

Moh Ahmed and Raymond Maika presented 'Using Rook to Manage Kubernetes Storage with Ceph' at Montreal's first Cloud Native Day, which took place on June 11 in Montreal.

Published in: Technology
  • Be the first to comment

Using Rook to Manage Kubernetes Storage with Ceph

  2. 2. Agenda • What is Rook? • Rook Timeline • Operator Pattern • Rook Design • Rook Architecture with Ceph • Container Storage Interface • Monitoring Ceph on Kubernetes • Demo of a Ceph Cluster Upgrade • Upcoming Rook Features
  3. 3. What is Rook? • Reliance on external storage – Not portable – Requires these services to be accessible – Deployment burden • Reliance on cloud provider managed services – Vendor lock-in • Day 2 operations - who is managing the storage?
  4. 4. What is Rook? A Storage Orchestrator for Kubernetes
  5. 5. What is Rook? A Storage Orchestrator for Kubernetes Also, a framework for many other storage providers and solutions
  6. 6. Rook Timeline v0.7 • Focus on Ceph storage v0.8 • CockroachDB • Minio v0.9 • Cassandra • EdgeFS • NFS • Ceph independence v1.0 • Backend maturity and stability • Ease of upgrade of operator and storage • Experimental CSI- Ceph driver 2018/02 2018/07 2018/12 2019/05 Incubating
  7. 7. Storage Backends Overview EdgeFS
  8. 8. Operator Pattern • A method of packaging, deploying, and managing an application • Extends Kubernetes API through Custom Resource Definitions (CRDs) • Reconciliation loops to enforce state of the CRDs • The Operator will: – Observe the objects – Analyze current vs. desired state – Act on changes Observe AnalyzeAct
  9. 9. etcdetcd Rook Design Kubernetes API kubectl etcd Rook Operator Rook Agent Kubelet Rook Flex Driver Management & Health API New Objects: Volume Attachments New Objects: Storage Clusters Storage Pools Object Store File Store Objects: Deployments DaemonSets Pods Services StorageClass / PV / PVC ClusterRole Namespace Config Maps Daemons CSI Driver
  10. 10. Rook Architecture - Ceph Container Container Container Container Volume Claim Volume Claim Rook-agent Rook-agent Rook-agent Rook-agent Rook-agentOperator
  11. 11. Container Storage Interface • A specification to establish a standard for block and file storage system • Allows the freedom to develop volume plug-ins externally from the orchestrator • Similar to how Container Network Interface (CNI) became a standard • Orchestrator agnostic ensuring compatibility across different platforms
  12. 12. Ceph CSI Driver • Implements an interface between a CSI Orchestrator (e.g. Kubernetes) and the Ceph cluster Ceph CSI Driver Version CSI Spec Version v0.3.0 v0.3 v1.0.0 v1.0.0 Kubernetes CSI Spec Compatibility Status v1.9 v0.1.0 Alpha v1.10 v0.2.0 Beta v1.11 v0.3.0 Beta v1.13 v0.3.0, v1.0.0 GA
  13. 13. Ceph CSI Driver Feature CephFS Plugin RADOS Block Device (RBD) Plugin Dynamically provision, de-provision File mode RWO volume >=v0.3.0 >=v0.3.0 Dynamically provision, de-provision File mode RWX volume >=v0.3.0 >=v0.3.0 Creating and deleting snapshot - v1.0.0 Provision volume from snapshot - >=v0.3.0 Provision volume from another volume - v1.0.0 Resize volume - -
  14. 14. Monitoring Ceph on K8s • Ceph Manager Daemon (ceph-mgr) – Became required since 12.x (luminous) release – Used to provide monitoring interfaces – Has a Prometheus plugin – Built in dashboard exposed with Rook 0.8 release
  15. 15. Upgrade Workflow • Since Rook v0.9: – The operator and its storage can be upgraded independently – Two separate images for the Rook operator and the Ceph cluster • Rook 1.0 supports: – Ceph Luminous (v12) – Ceph Mimic (v13) – Ceph Nautilus (v14) • Upgrading the Ceph cluster is as simple as editing the image in the CephCluster object
  16. 16. Demo • Show the CRDs and the pod deployment • Show monitoring tools • Upgrade the Ceph cluster to a new version
  17. 17. Future Plans • Rook v1.1 – Increased stability for other storage backends – Stable release for Ceph-CSI plugin – Improved upgrade workflows
  18. 18. Get Involved • Contribute to Rook – – • Slack - • Twitter - @rook_io • Forums -!forum/rook-dev • Community Meetings
  19. 19. References • Thanks to Jared Watts, founder of Upbound and Senior maintainer of the Rook project for his help and usage of some of his slides materials • • • 2017 • • proposals/storage/ • storage/ •