Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Storage for containers and cloud-native deployments - Rancher Online Meetup - March 2017

1,580 views

Published on

In our March 2017 Rancher Online Meetup we dove into running persistent storage for containers in Rancher. The presentation included an overview of persistent storage in Rancher, an introduction to Quobyte and an overview of Portworx.

Published in: Technology
  • Be the first to comment

Storage for containers and cloud-native deployments - Rancher Online Meetup - March 2017

  1. 1. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Storage for Containers and Cloud-Native Deployments March 28, 2017 #ranchermeetup
  2. 2. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Shannon Williams Co-Founder/VP Sales @smw355 1 Will Chan Co-Founder/VP Engineering @willchan00 #ranchermeetup
  3. 3. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Bjorn Kolbeck Co-Founder Quobyte @quobyte 2 #ranchermeetup Gou Rao Co-Founder and CTO Portworx @raogou
  4. 4. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . First things first… This is a not a ! 3 #ranchermeetup
  5. 5. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .4 There are rules for a meetup! • We won’t be done on time • Questions are always welcome • Demo, then demo some more • Things will break, be patient #ranchermeetup
  6. 6. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .5 Join the conversation on Twitter #ranchermeetup
  7. 7. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Agenda • Rancher Overview – Shannon • Storage Concepts and Data Persistence in Rancher – Will • Quobyte Overview – Bjorn • Portworx Overview – Gou • Latest Rancher Release Details – Will 6 #ranchermeetup
  8. 8. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Rancher Labs 7 #ranchermeetup An open-source software platform for managing containers A minimalist OS built explicitly to run Docker
  9. 9. © 2017 Rancher Labs, Inc. A complete container management platform that makes it easy to… 8 INNOVATE WITH CONTAINERS without compromising flexibility by empowering developers with fast access to the latest tools MANAGE APPLICATIONS by simplifying day to day application lifecycle management RUN CONTAINERS with the most complete set of container and infrastructure management capabilities Production ready ✔ 20 million+ downloads ✔ Open platform for innovating ✔ Easy to use interface ✔ Multi-tenant ✔ Role based access ✔ 24X7 support ✔ And more….
  10. 10. © 2017 Rancher Labs, Inc. Complete Container Management Platform 9 Application Catalog Container Orchestration and SchedulingUser Mgmt RBAC AD/LDAP SAML Ops Mgmt CI/CD Registries Monitoring Networking Multi-tenant Environments Environment 1 Environment N Infrastructure Services Storage ……. ..Environment 2 Security DNS/LB
  11. 11. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .#ranchermeetup Storage for Containers and Cloud- Native Deployments
  12. 12. © 2017 Rancher Labs, Inc. Docker Volume Plugins • Released as part of Docker 1.8 in Aug 2015. • Since then, there has been many volume plugins created including storage plugins to: • cloud providers (AWS, GCE, Azure, DO) • storage vendors (NetApp, EMC, Quobyte, Portworx) • more frameworks (rex-ray, openstorage) • While is this great, you still have to download it, install it, and manage it…
  13. 13. © 2017 Rancher Labs, Inc. Volume Plugin Manager • Rancher is your Docker volume plugin manager and provides the following: • Distribution – automatically deploys volume plugins to desired hosts managed by Rancher so they are made available immediately • Version Management and Upgrades – provides version management and allows users to automatically update plugins on all hosts with a single click (or API call) • Lifecycle Management – Provides health checks and automatic redeployment of plugins if things go wrong • Plugin-aware Container Scheduling – provides container scheduling based on deployment of plugins so you never schedule containers on hosts that have no access to storage
  14. 14. © 2017 Rancher Labs, Inc. How is this done? • Leverage Rancher’s catalog to for version management, upgrades, and distribution. • Leverage Rancher for lifecycle management of the volume plugin which needs to be packaged as a Docker container. • Leverage Rancher for container scheduling.
  15. 15. © 2017 Rancher Labs, Inc. Supported Plugins
  16. 16. © 2017 Rancher Labs, Inc. Supported Plugins • Rancher already supports NFS with EBS and EFS made GA in 1.6 (end of April) • Open source contributions include plugins for Rex-ray • Vendor contributions include Netapp, Portworx, and Quobyte
  17. 17. © 2017 Rancher Labs, Inc. Docker Plugins Storage Vendors Rancher Supported Cloud Providers
  18. 18. QUOBYTE - THE DATA CENTER FILE SYSTEM Fast and Reliable SoftwareStorage
  19. 19. Introduction • Björn Kolbeck Co-Founder atQuobyte 2006 2010-2013 2013 1 8
  20. 20. Google-Style Infrastructure Data Center “Warehouse-scale” Container Infrastructure Linux + Containers +Borg Google File System Replication + ErasureCoding Applications 1 9
  21. 21. GIFEE … but I’m not Google?! 2 0 • Scalability • Agility /Flexibility • Fault-tolerance • Commodityhardware • Containers (Kubernetes) • Storage? – Google has customstack – Everyone else: POSIX & legacy apps
  22. 22. Quobyte Data Center File System Scale-out shared POSIX filesystem Linux Linux Linux Applications MySQL, LAMP, Cassandra, ElasticSearch… 2 1 Commodity Servers no RAID, NVRAM, journalingdevice…
  23. 23. Data Center File System Reliability in Software: Storage must be cattle! • Split-brain safereplication • Erasure Coding • End-to-end checksums • Never trust thehardware! • Ignore: Broken/slow drives, servers, network cards,packet loss, network splits, brokenswitches… 2 2
  24. 24. Data Center File SystemLinear Scalabiliy without Bottlenecks • 2x4x6xServers = 2x4x6xIOPS, throughput, capacity • No bottleneck: Big data, video, webserver, hpc… • Grow compute &storage together 2 3
  25. 25. Data Center File System High Performance for All Workloads • High performance for all workloadtypes – Throughput – IOPS – Metadata intensive (small fileworkloads) – Consistent low latency <1ms • Share data viainterfaces – File system (Linux, Mac,Win) – S3 – Hadoop stat read(4k) write(4k) 420µs 2 4 150µs100µs
  26. 26. Data Center File System Smart Data Placement: Optimize your Apps ➡ Tiering ➡ Isolation ➡Application-aware data placement • Filename • Extension • Application • Age /Access 2 5 HDDSSDDatabase
  27. 27. Data Center File System Smart Data Placement: Cassandra fast and cheap Commit log(*.log) • Appendworkload • Replication:x3 • NVMe • Transactions <500usec SSTables(*.db) • Read-only •EC: x1.375 • Read optimizedSSDs • Lowest $/IOPS 10
  28. 28. Quobyte + Containers Integrated • Persistent Volume Plugin (>=1.4) – Global, replicas,container… – Automatic volume provisioning • UserId mapping (Quobyteblog) • Volume plugin • https://github.com/quobyte/docker-volume 27
  29. 29. Get Quobyte Today! • www.quobyte.com/get-it • Soon: RancherCatalog • hello@quobyte.com • Meet usat 28
  30. 30. © 2017 Rancher Labs, Inc. Data Services for Cloud Native Applications
  31. 31. © 2017 Rancher Labs, Inc. Portworx + Rancher: The Transformation to Cloud Native DevOps Infrastructure X86 Commodity Servers or Cloud Monitoring Orchestration/Scheduling Minimal OS Portworx Cloud Native Infrastructure Fixed Infrastructure Security Kubernetes w/ Rancher Containers HP EMC NetApp IBM Cisco VMware Dell Oracle RedHat Microsoft
  32. 32. © 2017 Rancher Labs, Inc. The Problem: Static, Out of Band Storage Provisioning SSD HDD nginx python mysql SAN html5 rest sql Apps hit a wall going to production Traditional Storage forced into a container, Out-of-Band Applications App Unaware Storage Cloud Connectors like RexRay SSD EBS
  33. 33. © 2017 Rancher Labs, Inc. Persistence IOPs Connectors SSD HDD SAN EBS S3 Admins Realize that Apps Don’t Want Volumes 1. Don’t make data protection a science project 2. Stop trying to connect legacy SAN/NAS/SDS to containers – They are not container or application aware 3. Your applications want on-demand services delivered programmatically via the scheduler 4. Storage services need to be “app-aware”, for example Cassandra-aware – Traditional storage is not.
  34. 34. © 2017 Rancher Labs, Inc. The Solution: Data Services built for DevOps for ANY Infrastructure nginx python mysql html5 rest sql SSD HDD SAN Cloud Portworx SSD EBS • App-aware • Any infrastructure • Programmatic No more volumes or storage per application to manage Rancher with Kubernetes
  35. 35. © 2017 Rancher Labs, Inc. Programmatic Data Services for DevOps WordPress WordPress PX MySQL MySQL persistence encryption CoS namespace Auth/ACLs HA Operational Experience Delivered as a lightweight container with support for volumes and graph. Runs on any server, any cloud. Converged Performance Data collocated with container compute Container Granular Controls Encryption, CoS, replication, snapshots are done at container granularity Hybrid Cloud Ops Cloud agnostic deployment and data accessibility across multiple clouds S3 Object
  36. 36. © 2017 Rancher Labs, Inc. Portworx is Topology and Application Aware Application … x86 Servers x86 Servers rack x86 Servers x86 Servers rack x86 Servers x86 Servers rack vol volvol vol vol Scheduler Portworx places an application’s data ON THE NODE that it executes on Container granular volumes on ANY infrastructure
  37. 37. © 2017 Rancher Labs, Inc. Portworx vs ScaleIO or CEPH x86 Servers x86 Servers rack … x86 Servers x86 Servers rack Portworx Places an Application’ss Data Local to a Few Topology Constrained Servers. Cassandra runs converged. x86 Servers x86 Servers rack … x86 Servers x86 Servers rack x86 Servers ScaleIO and CEPH shard an application’s data across many servers to get capacity. This breaks convergence. You also don’t have application granular settings on the volumes.
  38. 38. © 2017 Rancher Labs, Inc. Portworx is Enterprise Grade Software
  39. 39. © 2017 Rancher Labs, Inc. Demo: Portworx in the Rancher Catalog Demo
  40. 40. © 2017 Rancher Labs, Inc. Why Rancher: from the Portworx Perspective ▪ Start to finish • Launch Containers and Infrastructure services in seconds • Rich Community + Enterprise Catalog of services ▪ Deploy anywhere • Support for top cloud providers like Amazon, Digital Ocean, Packet … • Broad community of users moving to cloud-native microservices
  41. 41. © 2017 Rancher Labs, Inc. Portworx is Open Core OpenStorage.org Scheduler Data Layer: OpenStorage is a massively scalable data layer for Containers and Schedulers like Kubernetes and Mesosphere LCFS Data Path: LCFS provides core data path functionality for Docker images. Check it out on https://github.com/portworx/lcfs PORX Bundled Data Services: Try out the Portworx community edition at https://github.com/portworx/px-dev
  42. 42. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .41 Demo #ranchermeetup
  43. 43. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Latest Release 42 Rancher 1.5 – March 3, 2017 Key Features: - Templating support for catalog items - API Interceptor - Enhanced Network Policies - Webhooks – Host Scaling, DockerHub upgrade - Metadata Improvements - Portainer.io for Swarm UI
  44. 44. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Next Releases 43 Rancher 1.6 – Late April Key features: - Catalog Enhancements – Ability to add catalogs per environment - HAProxy Enhancements - - Certifying Rancher-EBS - - Scheduling Enhancements - schedule containers evenly across pools of hosts - K8S 1.6 Support with Docker 1.13/Docker 17.03 and etcdv3
  45. 45. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Getting Started Rancher and RancherOS are in GitHub – Get Involved! 44 #ranchermeetup http://github.com/rancher
  46. 46. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Even better - try.rancher.com… 45
  47. 47. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Then join a free training class… 46 http://rancher.com/training
  48. 48. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Thank you rancher.com #ranchermeetup

×