OpenEBS Technical Workshop at KubeCon San Diego
•Download as PPTX, PDF•
0 likes•68 views
The document summarizes an OpenEBS technical workshop that will take place at KubeCon on November 18th in San Diego. It introduces Container Attached Storage (CAS) and OpenEBS as solutions for running stateful applications on Kubernetes. It discusses how OpenEBS addresses challenges with managing stateful applications on Kubernetes and keeping storage agile. It also provides overviews of the OpenEBS architecture, including pluggable storage engines like cStor and Jiva, and components like the Node Device Manager.
Recommended
Recommended
More Related Content
What's hot
What's hot (18)
Similar to OpenEBS Technical Workshop at KubeCon San Diego
Similar to OpenEBS Technical Workshop at KubeCon San Diego (20)
More from MayaData Inc
More from MayaData Inc (12)
Recently uploaded
Recently uploaded (20)
OpenEBS Technical Workshop at KubeCon San Diego
- 1. OpenEBS Technical Workshop @ KubeCon November 18th, San Diego Presented by MayaData & D2iQ
- 2. CAS - Container Attached Storage + OpenEBS
- 3. The only Cloud Native Storage solution architected for Cloud Native Workloads Built on Kubernetes for Kubernetes
- 4. OpenEBS Enterprise fixes 2 big problems Deploying and managing stateful applications on Kubernetes is difficult Stateful applications on Kubernetes are not agile because data has gravity
- 5. The problem with Stateful microservice microservice microservice microservice microservicemicroservice INSIDE K8s Very Agile OUTSIDE K8s Locked in place
- 6. Let’s keep it in microservice microservice microservice microservice microservicemicroservice INSIDE K8s Very Agile
- 7. Applications have changed and someone forgot to tell storage
- 8. CAS = Container Attached Storage
- 9. Not agile Lock-in Incompatible Snapshots Storage Server Vol Vol Vol No HA Lock-in No Snapshots Simple, familiar App granularity K8S native 100% user space NAS/EBS DAS/Local PV OpenEBS CAS K8s Node K8s Node K8s Node App PV PV App PV App App AppApp PV PV App PV App App PV CAS Pod(s) PV PV CAS Pod(s) CAS Pod(s) CAS Pod(s) CAS Pod(s) Architecture Comparison
- 10. + Simple + Teams are autonomous + Additive to underlying systems or cloud volumes or JBODs + Target Users: ○ SRE ○ App Developer ○ Storage Admin OpenEBS Architecture
- 11. OpenEBS architecture ● Kubernetes Concepts : Namespaces, RBAC,CRD, Taints & Tolerations, Pod Affinity and Anti Affinity rules, etc., ● Kubernetes Storage Concepts: Running Stateful Workloads via PV, PVC, Storage Classes and Dynamic Provisioner ● Kubernetes and CNCF Management Tools: Kube Dashboard, Prometheus, Grafana, Opentracing, Jaeger ● Kubernetes Incubator Projects: Node Exporter, Node Problem Detector K8s Node App PVC PV PV App PVC PV App PVC Vol Vol Vol BlockDev (Raw/w FS) Dev DevDev
- 12. OpenEBS architecture Cluster Components OpenEBS Operator NDM Operator Storage Manager(s) (CSI Controller) NDM Storage (CSI) Agent Others ( Velero, MayaOnline..) Node Components Node n)Node 1) NDM Storage (CSI) Agent Others ( Velero, MayaOnline..) Others(Velero, MayaOnline, … ) Data Engines Data Engines
- 13. Pluggable Storage Engines The developer experience remains the same. Provisioned via StorageClass! Features Comparison cStor Jiva Local PV Max Capacity Expandable - upto total capacity of devices attached to a Node. Max 50G. Matches capacity of attached Device Protect against Node Failures Supported (Replication) Supported (Replication) Not Supported Volume Snapshots Supported (Reference based, COW) Supported (Reference Based) Not Supported Volume Clones Supported (Optimized) Not Supported Not Supported Volume Backup Supported (Optimized) Supported (Full Backup) Supported (Full Backup) External HDD, SSD on Nodes Preferred. Optional Optional
- 14. Pluggable Storage Engines Disks management differs based on the Storage Engine. Capability/Demands cStor Jiva Local PV Device Management Yes Not Supported Yes Replication Native n-way Native n-way No Volume Resize Yes Yes Yes* Recover data from failed disk to new disk Yes (Optimized) Yes Not Supported Read Write Many (RWM) Support Yes (via NFS) Yes (via NFS) Yes* Performance Container/Network Overhead Container/Network Overhead No Overhead Scale n(1) Pod per volume excluding shared pool pods. n(1 + r) Target Pod and Replica pods per Volume. No Overhead
- 15. cStor Monitoring, Alerting, Logging, Tracking, . . . (Example: Prometheus Fluentd, Jaegar . . . ) cStor Volume Target kubectl CSI Agent ( iSCSI Initiator) Stateless Target and its Service Infra Management Components Application Node Metrics Exporter Metrics Exporter PV and PVC Storage Nodes (1 . . . m) cStor Pool Stateful Application Pod ServiceKubelet cStor Volume Mgmt cStor Pool Mgmt Node Disk
- 16. cStor cStor Volume Target Pod Application Node Storage Nodes (1 . . . m) cStor Pool Pod Stateful Application Pod Node Disk cStor Pool Pod Node Disk cStor Pool Pod Node Disk cStor Volume Target does Synchronous Replication, i.e writes copies of the data to each of the available Replica Pools. cStor Volume Target attaches an unique sequence number to each of the block - before sending the copies to Replica Pools.
- 17. Jiva Monitoring, Alerting, Logging, Tracking, . . . (Example: Prometheus Fluentd, Jaegar . . . ) Jiva Volume Target kubectl CSI Agent ( iSCSI Initiator) Stateless Target and its Service Infra Management Components Application Node Metrics Exporter PV and PVC Storage Nodes (1 . . . m) Jiva Replica Stateful Application Pod ServiceKubelet Host Path
- 18. LocalPV kube-apiserver kube-scheduler . . . etcd kubectl Monitoring, Alerting, Logging, Tracking, . . . (Example: Prometheus Fluentd, Jaegar . . . ) Node Components (1 . . . n) Kubernetes Master Components Infra Management Components Cluster Level OpenEBS Components Provisioner(s) CSI Controller PVC Node Device Manager (NDM), Velero NDM, Restic Node Device Stateful Application Pod PV Application Operators SC Application Layer Dynamic Provision Layer Disk/Device Management Layer
- 19. MayaStor ● Low latency, high throughput data engine based on NVMe-oF technology ○ micro-VM ready (secure containers) ● Lockless, shared nothing design with scale per CPU core approach, written in RUST for additional safety guarantees ● In flight data integrity leveraging (DIF/DIX) crucial for multi- cloud and data mobility ● 100% in user space, and integration with VPP-VCL and DPDK for even higher performance demanding workloads ● 100% CSI based, creating a declarative dataplane where the user interacts with the concepts of (now well understood) PV and PVCS
- 20. MayaStor key features ● Composable: consume existing storage subsystems and unify them as a single PV ○ RBD (ceph), iSCSI, local disks (PV), existing NVMe arrays ● Strict data placement capabilities leveraging existing k8s concepts in terms of labels, taints and tollerations (data routing) with pluggable data scheduling algorithms ● Cross-cloud data at rest encryption, no more data pumps when moving between different cloud vendors ● Hybrid PVs, make use of SCM, NVMe and rotational or object ○ Use SCM for write journals, NVMe for caching and object for data storage or whatever the developer wants ● Inspire to be what for storage what k8s is for compute: make your data as agile as the applications that they serve
- 21. Node Device Manager (NDM) ● NDM runs as a daemonset and maintains the block devices as CRs ● NDM operator links north bound and sound bound interfaces Application Developer PVC OpenEBS LocalPV Provisioner OpenEBS cStor/Jiva volume Provisioner cStor Pool Provisioner NDM Operator NDM Sound bound provisioner CSI Drivers OpenStack VSAN OpenSDS Legacy (NetApp/Pure) Block devices in etcd MayaOnline/ MayaOnprem (DataOps) Auto provisioning of disks VSANEBS/GPD/ Azure Disks Any CSI driver ● Complete disk management including auto provisioning to smoothen the data ops ● Data mobility becomes easier with auto provisioning on remote clouds
- 22. What is OpenEBS What is OpenEBS
- 23. Your complete solution for building, testing, running, managing and monitoring stateful applications on Kubernetes with data migration, high availability, backup and disaster recovery.
- 24. A complete Data Agility Solution STORE AND PROTECT VALIDATE AND HARDEN UNDERSTAND AND CONTROL CONTAINER ATTACHED STORAGE CHAOS ENGINEERING DATA MIGRATION, HA, DR
- 25. All the features provided by a Storage Admin Delivered as Kubernetes-Native Services
- 27. is now Delivering an Enterprise-Grade Cloud-Native Experience
- 28. Educate + Adopt Design + Build Deploy + Manage Scale Our Approach to the Rapidly Maturing Cloud-Native Market Cloud-Native Journey Complexity+Scale Early Late LowHigh Konvoy Get Started with Kubernetes Cloud-Native Advisory + Services & Support + Learning Hub Support Organizational Transformation to Cloud-Native KUDO SDK (OSS + Dev + Data) Automate the Entire Application Kommander DC/OS + MKE + DS Engine
- 29. Cloud Provider Container Platform & Resource Manager Training Monitoring Log View and Search Debugging Service Discovery, Mesh and Networking K8’s Installer Container Runtime Storage Manage ApplicationInstall Application Packages https://landscape.cncf.io/
- 30. Keeping up and Securing a Fast Moving Ecosystem is a Real Challenge
- 31. ECOSYSTEM & IAAS Hardware Release MgmtCloud Mgmt OS Storage DNS Service (CoreDNS) Disaster Recovery (Velero) Ingress (Traefik) Logging (Fluentbit, Elastic, Kibana) ServiceMesh (Istio) ADD-ONS HypervisorPURE KUBERNETES Control Plane | etcd Containerd | Ansible Calico | Terraform Monitoring (Telegraf, Prometheus, Grafana) Load Balancer (Metal-LB) Konvoy: Getting Cloud-Native K8s into Production • Best of breed cloud-native components in a single distribution • Swappable standards, partner tech • CSI • CNI • Logging / monitoring • Simplified operations (from zero to one) • “Make K8s + CNCF dead simple” • Tested and pre-integrated solution Why Konvoy? © D2iQ. All rights reserved.
- 32. Konvoy: Cloud-Native Kubernetes Distribution Infra. Automation Kubernetes Core Add-Ons Deploy, Monitor and Log Validated & Tested Reference Architecture Kubeadm
- 34. Konvoy Command Workflow [ init ] → up → [ check ] → down
- 35. Create Konvoy Cluster $ konvoy up …
- 38. Delete Konvoy Cluster $ konvoy down …
- 40. Update Konvoy Cluster $ konvoy up edit cluster.yaml then
- 41. Upgrade Konvoy Cluster $ konvoy upgrade edit cluster.yaml then
- 42. Konvoy Provisioner AWS/GCP/Azure/... $ konvoy init --provisioner=aws edit cluster.yaml $ konvoy up $ konvoy down
- 43. Konvoy Provisioner Bare Metal $ konvoy init --provisioner=none create inventory.yaml edit cluster.yaml $ konvoy up $ konvoy down
- 46. Demo
- 47. Demo ● How Konvoy with OpenEBS installation works? ● Konvoy Demo ● Use case ○ Persistent Storage for Konvoy Components ■ Prometheus, Elastic (EFK), Velero ○ Persistent Storage for Workloads on Konvoy ■ Kafka, PostgreSQL, MySQL, Redis, Cassandra, MinIO... ○ Migrating workloads into Konvoy ● OpenEBS Director Demo ○ DMaaS MinIO workload
- 48. Resources 1. Konvoy Documentation: https://docs.d2iq.com/ksphere/konvoy/ 2. OpenEBS Documentation: https://docs.d2iq.com/ksphere/konvoy/ 3. Konvoy and OpenEBS Solution instructions: https://docs.d2iq.com/ksphere/konvoy/partner-solutions/openebs/ 4. OpenEBS Enterprise Platform: https://portal.mayadata.io/home 5. OpenEBS and Konvoy Workshop: https://github.com/openebs/community/tree/master/workshop
- 49. Konvoy with OpenEBS installation Installing Konvoy with OpenEBS done in 3 phases: 1) konvoy up with addons that don’t require persistent storage 2) Install OpenEBS 3) konvoy up with stateful addons on OpenEBS https://docs.d2iq.com/ksphere/konvoy/partner-solutions/openebs/
- 50. Get Started
- 51. Book demo and Free eval
- 52. Ask for our Konvoy Solution guide
- 53. Why MayaData OpenEBS Enterprise Platform OpenEBS Community OpenEBS Enterprise Platform Support Self Up to 24x7 Performance enhancement X Yes Bug / patches / updates X Priority and personalized OpenEBS Director Basic - 7 days logs / 3 nodes SaaS Advanced - Up to 13 months SaaS and On-Premises DMaaS, RBAC, Backup & Migration X Yes You support the development of OpenEBS X Yes
- 54. Starting at $ 63/Month ➔ $ 63 per month ➔ Up to 10 hours of professional assistance to set up OpenEBS Enterprise + Director in your environment with 3 subs. ➔ Expert advice on how to deal with Stateful Applications on Kubernetes ➔ Enterprise class support ➔ Yaml for most popular use cases ➔ Intro training ➔ Personalized ROI document
- 55. Q & A