MicroCeph on MicroK8s — an exciting fusion of powerful storage capabilities and lightweight container orchestration. MicroCeph, the compact and scalable storage solution, finds its perfect match in the MicroK8s ecosystem.

1. Ceph storage on MicroK8s!
Angelos Kolaitis
Senior Engineer @MicroK8s
1.29 Release Lead Shadow
MicroK8s
Friday 1st of September, 1:30 pm GMT
https://meet.google.com/hjy-uogt-tax
Philip Williams
Product Manager
Ceph
&

2. Agenda
● What is MicroCeph?
● MicroCeph advantages
● Deployment of MicroCeph
● MicroCeph on MicroK8s

3. What is MicroCeph?
Consistent, Isolated and Upgrade Friendly
● Streamlined and effortless deployment
● Minimal setup and maintenance overheads
● MicroCeph is containerised snap with all of its
dependencies and runs fully isolated from the
underlying host
● Using channels for the distribution, going from
latest/stable to latest/edge is easy

4. MicroCeph advantages
Storage for non-experts
● Easiest way to deploy Ceph, just a few commands
● Designed for non-experts
● Highly scalable
○ dev workstation, to edge, to datacenter
● Reliable and resilient
● Open-source distributed storage system
● Flexibility for object, block, and file-level storage
● Supports encryption at rest too!

5. snap install microceph
microceph cluster bootstrap
microceph cluster add node[x]
microceph cluster join pasted-output-from-node1
microceph.ceph status
microceph disk add /dev/sd[x] --wipe
microceph.ceph status
microceph.ceph osd status
MicroCeph deployment
microceph.com
QR Code?

6. MicroCeph demo

8. Ceph Storage in MicroK8s
MicroK8s +

9. - Kubernetes solves container orchestration issues at scale
- I can run my massive distributed workloads at scale (machine learning, video
transcoding, my SaaS business)
- Where can I store the end results?
- We need persistent storage!
- CSI (Container Storage Interface)
- Kubernetes does not solve the problem of distributed storage, since
performance, stability requirements might differ
- Instead, it defines a standard interface to consume any storage
- Which CSI driver? Depends!
- On a public cloud? aws-ebs-csi-driver, azurefile-csi-driver, …
- On a private cloud? cinder-csi-driver, …
- On bare-metal? BYOS!
- Ceph
Distributed Storage In Kubernetes

10. - Ceph CSI!
- Supports CephRBD (ideal
for RWO volumes)
- Supports CephFS (supports
RWX volumes)
Ceph storage in Kubernetes

11. sudo microk8s enable rook-ceph
Deploy rook-operator on the
MicroK8s cluster
Rook can:
- deploy a Ceph cluster inside
the MicroK8s cluster, using
disks from the hosts
- integrate with external
Ceph clusters and manage
them as citizens in
Kubernetes
MicroK8s Rook Ceph addon (1/2)

12. sudo microk8s enable rook-ceph
Why Rook?
- Can detect the right CSI
version depending on running
K8s and Ceph
- (Optionally) Can be used to
manage Ceph from
Kubernetes
- (Optionally) Simplifies ceph
auth clients and keyrings
- (Optionally) Can deploy a
Ceph cluster on top of the
Kubernetes nodes
MicroK8s Rook Ceph addon (2/2)

13. Install MicroCeph
$ sudo snap install microceph --channel quincy/stable
$ sudo microceph cluster bootstrap
$ sudo microceph disk add $disks --wipe --encrypt
MicroK8s / MicroCeph (1/4)

14. Install MicroK8s
$ sudo snap install microk8s --classic --channel 1.28/stable
MicroK8s / MicroCeph (2/4)

15. Enable Ceph addon
$ sudo microk8s enable rook-ceph
MicroK8s / MicroCeph (3/4)

16. Consume MicroCeph
$ sudo microk8s connect-external-ceph
MicroK8s / MicroCeph (4/4)

17. microk8s connect-external-ceph
- Connect to Ceph cluster and initialize RBD pool
- Detect appropriate Ceph CSI version to deploy
- Generate Ceph client keyrings for ceph-csi
- Create Kubernetes resources for connecting to Ceph (ceph mons, keyrings)
- Create StorageClasses for consuming Ceph storage
MicroK8s / MicroCeph

18. Deploy microceph and microk8s on an edge environment and use Ceph for
storage
$ sudo snap install microceph --channel quincy/stable
$ sudo snap install microk8s --classic --channel 1.28/stable
$ sudo microk8s enable rook-ceph
$ sudo microk8s connect-external-ceph
Scenarios (1/4)

19. More complex Ceph configuration, use an RBD pool backed by specific OSDs
(e.g. tenant separation, use only SSD disks, …)
$ ceph osd crush rule create-replicated replicated_ssd default host ssd
$ rbd pool create microk8s-ssd-pool
$ ceph osd pool set microk8s-ssd-pool crush_rule replicated_ssd
$ microk8s connect-external-ceph --rbd-pool microk8s-ssd-pool
Scenarios (2/4)

20. CephFS for RWX support
$ ceph fs volume create microk8s-cephfs0
$ microk8s connect-external-ceph
Scenarios (3/4)

21. Integrate with any Ceph cluster
$ microk8s connect-external-ceph
--ceph-conf ./ceph.conf
--ceph-keyring ./keyring.conf
Scenarios (4/4)

22. Can I Try This Out?
Sure!
1.28 cluster in 5 min
$ sudo snap install microk8s --classic --channel 1.28/stable
MicroK8s releases on 1.28/edge within 24 hours after upstream release
MicroK8s

23. Thank you
Interaction
• Websites
https://microk8s.io/
https://github.com/canonical/microk8s
• Discussion on
https://discuss.kubernetes.io/tag/microk8s
or in Slack on the #microk8s channel
• Report Issues
https://github.com/canonical/microk8s/issues
• MicroK8s Community Sync
Every Friday, 1:30pm GMT
Video call link: https://meet.google.com/hjy-uogt-tax
Upcoming sessions from MicroK8s 1.28
- 1.28 Upstream Updates
- MicroK8s CIS conformance
- MicroK8s Dual Stack
- MicroK8s Ceph integration (feat MicroCeph)
Upcoming sessions calendar
https://www.meetup.com/microk8s-meetup-group/
MicroK8s

24. Thank you. Questions?