Containerized Storage for Containers- Kubernetes LA Meetup , July 2017
•Download as PPTX, PDF•
1 like•415 views
The Containerized Storage for Containers(OpenEBS) slides were presented in the Kubernetes Meetup , LA on July 18th , 2017. OpenEBS delivers storage and storage services to containerized environments. OpenEBS allows stateful workloads to be managed more like stateless containers. OpenEBS storage services include: per container (or pod) QoS SLAs, tiering and replica policies across AZs and environments, and predictable and scalable performance. To know more about OpenEBS visit our website www.openebs.io
More Related Content
What's hot
What's hot (20)
Similar to Containerized Storage for Containers- Kubernetes LA Meetup , July 2017
Similar to Containerized Storage for Containers- Kubernetes LA Meetup , July 2017 (20)
More from OpenEBS
More from OpenEBS (19)
Recently uploaded
Recently uploaded (20)
Containerized Storage for Containers- Kubernetes LA Meetup , July 2017
- 1. Containerized Storage for Containers Kubernetes LA Meetup July 18, 2017 @OpenEBS Evan Powell - @epowell101
- 2. Who? Team of storage and DevOps engineers with a 5 year track record of delivering containerized storage (built on jails) to NTT, eSilicon, TCS and other service providers. CloudByte is the pre-existing SDS brand. Evan Powell, CEO Director of Ops & BD who started founding companies back in 2000 and is still at it. Also an investor and advisor and friend to enterprise focused entrepreneurs. Joined as CloudByte / Open EBS CEO 7/1/17
- 3. What if storage for container native applications was itself container native?
- 4. ✓ Intent inherited ✓ Scale w/o bottlenecks ✓ No special skills needed ✓ Granularity ✓ Resiliency ✓ Inate multitenancy ✓ Multi-cloud portability ✓ Smaller “blast radius”
- 5. Manifests express intent K8S used with special care for apps requiring persistence because they require brittle tight coupling. Hard wired connections via plug-in “IO Blender” Container 1 (Node1) Container 2 (Node 2) Container 3 (Node 3) stateless stateful Stateful still too difficult Container 1 Container 2 Container 3 Container 1 Container 2 Container 3 NAS SAN S3 NAS Cloud Any Server, Any Cloud
- 6. Manifests express intent stateless ✓ Intent inherited Container 1 Container 2 Container 3 Container 1 Container 2 Container 3 Container 1 Container 2 Container 3 stateful No changes to DevOps workflow even for containers requiring persistence. Users manifest their intent and the storage and storage controllers adjust automatically as needed. Data Container Data Container Data Container Containerized storage for containers Any Server, Any Cloud Any Server, Any Cloud
- 7. ✓ Same workflow, same tools, same skills Volumes Provisioners PVC/PV StorageClasses Attach/Detach Controllers CNCF/CSI Container 1 Container 2 Container 3 Data Container Data Container Data Container stateful Containerized storage for containers Dynamic Provisioning High Availability Data Protection Storage Silos, Kubernetes Managed Vendor Lock-in, Cross cloud Slow Upgrades, Seamless Upgrades Monolithic!, Microservices Any Server, Any Cloud
- 8. Storage Orchestration - K8s Vs OpenEBS Functionality K8s OpenEBS Volume Life-cycle - provisioning, attaching/detach, de-provisioning X Monitoring the volume data X Scheduling the volume provisioning intelligently X Smart space management X Availability of volume X Application aware backup/snapshots X QoS guarantee (latency guarantee) X Smart movement of the volumes across racks and regions X Cataloging of storage requirements for different applications X K8s takes care of connecting applications to stateful storage. OpenEBS answers the other persistent storage questions
- 9. Storage just fades away as a concern
- 10. How is OpenEBS built ? Storage containerization Storage orchestration Deep storage operations Core Storage features is built using is integrated with is built into (OpenEBS “Maya”) is built into (OpenEBS) (Control plane) SmartCapacityManagement Volume LatencyMonitoringandAuto-healing Adaptto varyingworkloads- ApplicationAwareStorage(Pods) AWS EBSLikeAPIs Storage Analytics (DataPlane) Containerizedvolumes Thinand scalableblockstorage EnterpriseHA IntelligentQoS SmartTieringfor higherperformance
- 11. Architecture: Kubernetes K8S Master Minion POD Container Container ContainerKubelet POD Container Container ContainerKubelet POD Container Container ContainerKubelet Minion POD Container Container ContainerKubelet POD Container Container ContainerKubelet POD Container Container ContainerKubelet Minion POD Container Container ContainerKubelet POD Container Container ContainerKubelet POD Container Container ContainerKubelet etcd APIs Cntrl Schld Minions run on physical nodes PODs group containers, share an IP address, and each include a Kubelet agent K8S Master services include: etcd, APIs, the scheduler, the control manager & others
- 12. Architecture: Kubernetes + OpenEBS K8S Master Minion POD Container Container ContainerKubelet POD Container Container ContainerKubelet POD Container Container ContainerKubelet Minion POD Container Container ContainerKubelet POD Container Container ContainerKubelet POD Container Container ContainerKubelet Minion POD Container Container ContainerKubelet POD Container Container ContainerKubelet POD Container Container ContainerKubelet etcd APIs Cntrl Schld OpenEBS MayaAPIs Schld Data Containers run in PODs on physical machines Data Containers group physical disks local or remote, manage replicas, provide QoS control and more MAYA runs on the Master; delivers services such as: APIs, the storage scheduler, analytics & others
- 13. Why move the data & configs next to the app? Today’s scale out storage systems are almost infinitely configurable. Matching their configurations to static workloads is difficult. In a world of dynamic, ever evolving workloads due to micro-services and multi-cloud deployments, it is not a tractable problem for humans. Let’s say you want to benchmark for your workload: Source: Alex Aizman’s blog: https://storagetarget.com/author/alexaizman/
- 14. Why move the data & configs next to the app? Today’s scale out storage systems are almost infinitely configurable. Matching their configurations to static workloads is difficult. In a world of dynamic, ever evolving workloads due to micro services and multi-cloud deployments, it is not a tractable problem for humans. Let’s say you want to benchmark for your workload: Duration of run ~6,700 yearsSource: Alex Aizman’s blog: https://storagetarget.com/author/alexaizman/
- 15. More benefits of Cloud Native approach OpenEBS always available multi-cloud block services are delivered via containerized microservices Granularity: Each pod has its own block protocol stack, tiering engine, QoS engine and more. SLAs per POD. Performance & efficiency: access patterns maintained per-volume. Hot-data on NVMe-Flash or on 3DX-Memory. Cold- data on slower disks / SAN/ Cloud-Storage/S3. Scalability: Metadata based on “size of the volume” & not on “number of volumes”. Can reduce metadata traversal 100x. Blast radius: Eliminates risk of scale out storage system corruption. And much, much more such as cost savings due to resource efficiency, innate multitenancy, and cross cloud portability The Conway’s Law benefits might be the most important - innovation unlocked ✓ ✓ ✓ ✓
- 16. ✓ Scalbility - no metadata bottleneck Global Metadata at a system level Logical volumes Lun Application Metadata is managed at block level IO Blocks IO Blocks 100TB Raw 8 TB Meta 100TB Raw XFS Meta data XFS Meta data2 TB Meta 2 TB Meta Logical Volumes in XFS files Volume meta Lun Application IO Blocks IO Chunks Metadata of the volume is managed at chunk level 100 GB Meta The volume IO processing has to deal with the global metadata of 8TB The volume IO processing has to deal with the volume metadata of 100GB
- 18. Cluster Admin - Enabling OpenEBS on K8s K8s master K8s minions Step 1: Use can existing K8s Cluster or Launch a new Cluster (locally or in your fav cloud provider or use stackpoint.io to create a cluster.) Step 2: Configure OpenEBS on K8s. kubectl apply -f openebs-operator.yaml Step 3: Configure or Customize OpenEBS kubectl apply -f openebs-config.yaml kubectl apply -f openebs-storageclasses.yaml https://github.com/openebs/openebs/tree/master/k8s
- 19. Developer - Using OpenEBS PVs Step 1: Checkout the StorageClasses available kubectl get sc Step 2: Modify the Application YAML to use the openebs storage class. kubectl apply -f demo-jupyter-openebs.yaml https://github.com/openebs/openebs/blob/master/k8s/demo/jupyter/demo-jupyter-openebs.yaml K8s master K8s minions Step 3: Launch Jupyter UI http://<external-ip>:<port>/
- 20. Admin’s interaction with OpenEBS K8s master K8s minions Storage classes Central Repo Install OpenEBS kubectl apply -f openebs-operator.yaml kubectl apply -f openebs-storageclasses.yaml kubectl apply -f openebs-config.yaml 1 2 3
- 21. Developer’s interaction with OpenEBS K8s master K8s minions - Data science developer - Wants to set up a jupyter notebook (not an expert in storage) 1 kubectl apply -f demo-jupyter-openebs.yaml 2 - Configure the jupyter yaml file for all the jupyter functionality - Read the available OpenEBS storage classes and choose one - Launch the jupyter application kubectl get sc
- 22. In short…. Pod 1 OR Pod 1VS Cloud Native Cloud Washing
- 24. What do you think? https://github.com/openebs Join the community #slack Slack.openebs.io blog.openebs.io @openebs
- 25. https://www.openebs.io/careers/ Join Us and contribute to Kubernetes - Prioritized Replication Networking/Storage - Federated Clusters - Analytics and ML Services - CNCF/CSI
Editor's Notes
- As you need a special storage engineer. A really special one.
- Join the growing community. Do follow us on twitter and our blog.
- Join the growing community. Do follow us on twitter and our blog.
- Join the growing community. Do follow us on twitter and our blog.