stackconf 2022: Data Management in Kubernetes – Backup, DR, HA
•
0 likes•60 views
Kubernetes is everywhere now, right? You see how companies are embracing this technology more and more, it’s like the Kardashians or Rosalía or TikTok; MAINSTREAM! BUT, as companies evolve and grow their container environments, they realize that stateful apps require more than a robust Kubernetes distribution, data is the key of their apps. Applications are as important as the data that they use. Managing 1M bank transactions is not the same as using bridge images, so you have to be very careful on how you control and use that data. Portworx is here to help with those problems. In this session, we will see how Portworx is able to manage volumes in the cloud with Kubernetes in seconds AND migrate that data from cluster to cluster in minutes, even a Disaster Recovery environment with ZERO data loss and NO downtime. Do you want to see it live? join this session!
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to stackconf 2022: Data Management in Kubernetes – Backup, DR, HA
Similar to stackconf 2022: Data Management in Kubernetes – Backup, DR, HA (20)
Recently uploaded
Recently uploaded (20)
stackconf 2022: Data Management in Kubernetes – Backup, DR, HA
- 1. Miguel Rojas Cloud Architect EMEA Portworx by Pure Storage mrojas@purestorage.com @portworx @purestorage Delivering a Kubernetes data platform with Portworx
- 2. © 2022 Pure Storage 2 Kubernetes, Containers and Companies What’s your story? Founded 2014 ‘15 ‘20 ‘17 A little history
- 3. 3 © 2022 Pure Storage, Inc. Cloud Native is driven by Kubernetes adoption OF NEW APPS are developed in containers. 95% OF ENTERPRISES 81% OF CNCF MEMBERS 78% use Kubernetes in production. work with 2+ public cloud providers. OF GLOBAL BUSINESSES 85% will be running containers in production by 2025 (up from 35% in 2019). Source: CNCF Survey 2021
- 4. 4 © 2022 Pure Storage, Inc. The Kubernetes stack has room to mature Which areas of the Kubernetes tech stack need to mature the most to make it easier to deploy cloud-native apps? Source: Kubernetes Adoption Survey 2021 - Portworx Ranked top three Application development frameworks Storage and data management Monitoring and observability Networking Orchestration and management 75% 68% 57% 54% 46%
- 5. 5 © 2022 Pure Storage, Inc. Connector Approach to Container-Native Two approaches common in the industry today… Container-Native Storage Approach ”Connector” Approach Storage Array 1 Storage Array 2 CSI • 1:1 container <> volume mapping • Container capabilities vary depending on the capabilities of the underlying array • Storage arrays often are challenged by the object count (# volumes, connections) and change rate (10,000s changes/day) • Innovation limited to the features in the open CSI specification • Storage arrays managed independently from K8s Storage Array Local SSDs/HDDs Storage Pool Cloud Storage • Many (1,000s):1 container <> volume mapping • Designed to scale to 10,000s operations/day • Ensures consistent storage services across ANY infrastructure • Storage services container-granular and application consistent • Storage “encapsulated” in a container-native virtual volume that is free to move with the container • Managed as a native part of K8s Pure PSO NetApp Dell
- 6. 6 © 2022 Pure Storage, Inc. Kubernetes key attributes MASSIVE-SCALE KUBERNETES-NATIVE PLUGGABLE ARCHITECTURE OPTIMIZED FOR MULTI-CLOUD WORLD APPLICATION CENTRIC Scales to 1,000s of nodes, 1,000s of concurrent operations per minute All platform operations should be through a consistent control plane, the Kubernetes API Able to offer a consistent control pane on heterogeneous infrastructure environments Easily swap in and out core ”infrastructure” services – networking, storage, monitoring Platform capabilities should be focused on the needs of an application first Compute Platform
- 7. 7 © 2022 Pure Storage, Inc. Today, Portworx Is the Leading Kubernetes Data Services Platform Any App Any Distribution GKE DATABASE ANALYTICS STREAMING SEARCH/LOG AI/ML 5G/IoT Any Cloud Any Infrastructure Bare Metal Pilot Go Live Global CaaS Test/ Dev Every Stage The Kubernetes Data Services Platform PX-Store PX-Secure PX-Backup PX-DR PX-Migrate PX-Autopilot Portworx Data Services
- 8. 8 © 2022 Pure Storage, Inc. What would you like to solve? The Kubernetes Data Services Platform PX-Store PX-Secure PX-Backup PX-DR PX-Migrate PX-Autopilot Storage, Share, Scale, Monitor Anywhere in K8s Backup and restore from, and to, anywhere DR: RTO of minutes, Minutes RPO Migrate or duplicate environments fast Auto-pilot Intelligent, performant, hyper converged self-service Storage. The Kubernetes Data Services Platform PX-Store PX-Secure PX-Backup PX-DR PX-Migrate PX-Autopilot Portworx Data Services Secure and grant storage access with RBAC and Integrations DBaaS anywhere fully managed by Portworx.
- 9. 9 © 2022 Pure Storage, Inc. Portworx sets the gold standard for Kubernetes Storage Portworx named a Cloud-Native Data Storage & Enterprise Kubernetes Storage leader in 2021 GigaOm Radars Cloud-Native Data Storage GigaOm Radar for Cloud-Native Data Storage for Kubernetes, 2021 GigaOm Radar for Enterprise Kubernetes Storage, 2021 Enterprise Kubernetes Storage
- 10. 10 © 2022 Pure Storage, Inc. Use case: App and Data Migrations Data centre 1 / Cloud 1 Data centre 2 / Cloud 2 Kubernetes 1 Kubernetes 2 OBJECT STORAGE ► Async data & object replication ► Snap restore based ► Scheduled or on-demand ► RPO 5 minutes ► RTO 5 minutes
- 11. 11 © 2022 Pure Storage, Inc. Use case: Container backup & restore Application granular backup & restore App 1 App 2 1. k8s objects 2. Config 3. Data On-premises ANY Object Storage NFS – coming soon! PX-Backup App 1 App 2 App 1 App 2 Second site Cloud environment RESTORE RESTORE
- 12. 12 © 2022 Pure Storage, Inc. Portworx Backup Namespace A AWS mysql-0 Namespace B On-Prem mongo-0 Backup Location Schedule Rules Backup Location Schedule AWS S3-A AWS S3-B Daily Hourly FB S3-A FB S3-B Weekly Monthly Developer User Type App Owner Admin ● Backup/restore their apps ● App + namespace granularity Permissions + ● Set schedules ● Set rules ● Use existing cloud accounts + ● Add users ● Create and share cloud accounts & backup locations ● Create custom roles
- 14. 14 © 2022 Pure Storage, Inc. Operator Kubernetes Installing Portworx Installing Portworx deploys an Operator to run worker nodes, not on the control plane nodes Control plane Worker Worker Worker Spec based installation HELM Package manager
- 15. 15 © 2022 Pure Storage, Inc. Installing Portworx Local storage or SAN based storage will be used to create a virtual storage pool SSD SSD LUN SSD SSD LUN SSD SSD LUN Storage Pool Worker Worker Worker
- 16. 16 © 2022 Pure Storage, Inc. Using Portworx Application can create Persistent Volumes from the storage pool using Persistent Volume Claims SSD SSD LUN SSD SSD LUN SSD SSD LUN Storage Pool Worker Worker Worker Replication factor 1 Replication factor 2 Replication factor 3 50G 50G 50G For high availability you can mirror volumes across nodes using a replication factor, which can be applied to the volume or the stroage class
- 17. 17 © 2022 Pure Storage, Inc. Respond to a node failure Application can create Persistent Volumes from the storage pool using Persistent Volume Claims SSD SSD LUN SSD SSD LUN SSD SSD LUN Storage Pool Worker Worker Worker 50G 50G 50G Portworx integrates with the Kubernetes scheduler to reschedule the Pod on a worker with a local replica
- 18. Designing a Portworx deployment Deployment models
- 19. 19 © 2022 Pure Storage, Inc. Standard deployment • Single Portworx and Kubernetes cluster • Minimum of three worker nodes • A block storage device Worker Worker Worker
- 20. 20 © 2022 Pure Storage, Inc. AZ 2 AZ 3 AZ 1 Worker Worker Worker Multi-zone high availability (HA) VOL VOL VOL Volume replica placement Follows K8s failure domains Multi-zone Kubernetes cluster ETCD ETCD ETCD Worker Worker Worker
- 21. 21 © 2022 Pure Storage, Inc. Storage Cluster Scale and manage storage Compute Cluster Scale and manage compute PWX PWX PWX PWX Portworx cluster KVDB VOL VOL VOL VOL Disaggregated compute and storage PVC PVC PVC
- 22. 22 © 2022 Pure Storage, Inc. Case Study: Consumer scale for Enterprise PaaS CHALLENGE ► Scale back-office apps responsible for T-Mobile store experience during Apple launches SOLUTION ► Portworx Enterprise running on-prem to provide complete data lifecycle management- provisioning, encryption, backup, and more. RESULTS ► Regularly withstand the launch of new Apple products results in dramatic spike in usage Data services used:
- 23. 23 © 2022 Pure Storage, Inc. © 2022 Pure Storage, Inc.