Downloaded 92 times
Uploaded by{code} by Dell EMC
Deep Dive on REX-Ray, libStorage and the Container Storage Interface - Clinton Kitson - Dell EMC World 2017
REX-Ray provides container storage orchestration and works with various container platforms and storage backends. It runs as a Docker volume plugin and uses libStorage to integrate with different storage drivers. The Container Storage Interface (CSI) aims to standardize storage integration across container orchestrators. REX-Ray plans to implement CSI to provide a universal storage interface and improve the user experience of container storage. Maintaining storage plugins is challenging, so {code} builds managed plugins to simplify integration with platforms like ScaleIO.
More Related Content
Similar to Deep Dive on REX-Ray, libStorage and the Container Storage Interface - Clinton Kitson - Dell EMC World 2017
Deep Dive on REX-Ray, libStorage and the Container Storage Interface - Clinton Kitson - Dell EMC World 2017
- 1. Deep Dive onREX-Ray, libStorage and the Container Storage Interface Clint Kitson Technical Director @clintkitson {code} by Dell EMC
- 2. © Copyright 2017Dell Inc.2 Open source at Dell EMC – Contributing to and maintaining open source projects – Community engagement – Technical solution leadership {code} by Dell EMC is a group of passionate open source engineers and advocates building community around software-based infrastructure. Platinum Sponsor
- 3. There is nosuch thing as a stateless architecture. Applications need data.
- 4. © Copyright 2017Dell Inc.4 Yes, applications need data Lots of different types of persistent services to consider Files Blocks Documents Logstreams Time Series Media and Streaming Modern or Traditional Applications Storage Services Other Persistence Services Objects
- 5. © Copyright 2017Dell Inc.5 Use services from any cloud True portability of applications requires interoperability My Cloud Public Cloud Universal Network and Storage Interfaces Container Orchestrators ..and interoperating with cloud services ..with your preferred CO Running Docker Container Images ..with or without virtualization DEW2017 Sessions No teleportation here.
- 6. © Copyright 2017Dell Inc.6 Storage with persistent applications Cloud Native thinking… Use available storage features Plan for failures and transience Interoperate and orchestrate Storage is critical in a cloud operating model supporting your applications.
- 7. How does thiswork today? An example: REX-Ray and Docker Managed Plugins
- 8. © Copyright 2017Dell Inc.8 Storage orchestration with containers How do the basics work? Storage is attached and detached from host instances where containers are targeted to run. Container Host Container Runtime/Orchestrator REX-Ray, the container storage orchestration engine
- 9. © Copyright 2017Dell Inc.9 Storage Services Digging a little deeper REX-Ray runs as Docker Managed Plugins. It provides storage services from backend platforms. Docker Engine > docker run > docker plugin Storage, Networking, and Authorization Plugins runC Containers • Any block/file/object platform • Amazon AWS/EFS/S3 • Dell EMC ScaleIO/Isilon/ECS • Digital Ocean Block Storage • FittedCloud EBS • Google Compute Engine PD/CSB • OpenStack • Red hat Ceph RBD/RGW • S3-Compatible Object Storage • VMware vSphere • .. Contribute!!
- 10. © Copyright 2017Dell Inc.10 REX-Ray Deployment Models • Standalone – Deployed to every node and communicates directly with storage platform, one or many storage drivers included • Agent and Controller (via libStorage API) – Agent › Deployed to every node and communicates only with a controller, no storage drivers included – Controller › Deployed centrally and communicates directly with storage platform, one or many storage drivers included • Plugins/Containers – Standalone/Agent can be packaged as containers or managed plugins – Controller can be packaged as a container
- 11. © Copyright 2017Dell Inc.11 Value in features • Centralized or standalone architectures • Encrypted communication and per-client authentication • Trusted and simple storage operations for any use case • Expose different or more granular operations • CLI operations Container storage orchestrators provide useful storage features not natively available from cloud or storage platforms. For example, Enterprises may care about..
- 12. © Copyright 2017Dell Inc.12 REX-Ray in the Docker Store
- 13. © Copyright 2017Dell Inc.13 REX-Ray with ScaleIO $ docker plugin install store/rexray/scaleio Installed plugin store/rexray/scaleio:latest $ docker plugin ls ID NAME DESCRIPTION ENABLED 450420731dc3 store/rexray/scaleio:latest REX-Ray for Dell EMC ScaleIO true $ docker run -ti --volume- driver=store/rexray/scaleio -v test:/test busybox $ df -h /test
- 14. © Copyright 2017Dell Inc.14 Docker Engines Standalone REX-Ray deployment with ScaleIO SDC SDS SDS SDS MDMs GW Pools Volumes Data-Plane Control-Plane docker run -ti --volume- driver=rexray/scaleio -v test:/test busybox
- 15. © Copyright 2017Dell Inc.15 How does that actually work? Docker Engine JSON/RPC over HTTP/Volume.Mount {“name”: “test”} • Introspect my instance ID • Attach volume “test” to me • Wait for volume to be discovered locally • Get device path • Check for filesystem – Create if does not exist • Mount file system • Return mounted path to container Container Host {“/var/lib/libstorage/ volumes/test”} Docker Volume Driver interface DVDI Cloud Native Storage
- 16. DEMO Try it yourselfin the HOL!!
- 17. Container orchestrators and runtimeshave different implementations and levels of storage support.
- 18. © Copyright 2017Dell Inc.18 Storage Plugins Interoperability today Docker Volume Driver Interface DVDI DVDCLI Flex Interface DVDI JSON over RPC JSON over Proc JSON/RPC over HTTP Storage Platform JSON/RPC over HTTP In-Tree
- 19. © Copyright 2017Dell Inc.19 Container Orchestrators Yesterday (2017-04-10) Container Orchestrator Storage Interface Centraliz ation Storage Roles Storage Resource Model Resource Details Storage Services Operator Automation Docker Swarm DVD (RPC/JSON) Plugin responsibi lity AppOwners Volumes N/A Plugin in container N/A Mesos DVDI proc (RPC/JSON) Plugin responsibi lity AppOwners N/A N/A Plugin on host Frameworks Kubernetes Flex proc, In- tree kubectl and kubelet plugin Ops, AppOwners Volumes Some Kube node plugin, Host process Custom Controllers Cloud Foundry N/A, In-tree N/A AppOwners Volumes N/A N/A N/A Container Storage Interface** gRPC Native Ops, AppOwners Volumes, Snapshots, Pool, Platform Material parameters Plugin in container or on Storage platform CO Choice
- 20. © Copyright 2017Dell Inc.20 Progress is made on collaboration and solid foundations Container Storage Interface (CSI) CSI Implementations Operator Automation Cloud Native Storage Different Interfaces • Fragmented storage integrations hurts customer experience and limits cloud native use cases for persistent applications • Value in how applications make use of storage resources and not how volumes are created, removed, and attached
- 21. © Copyright 2017Dell Inc.21 Container Storage Interface (CSI) Benefits • Ensure excellent user experience for consumers of storage with cloud native platforms • Simplify storage platform responsibilities through a single integration method • Provide common storage orchestration and common storage operations to container orchestrators (CO) and runtimes • Enable container orchestrators to differentiate on valuable features
- 22. © Copyright 2017Dell Inc.22 CSI Operations and Model Container Storage Interface Volumes/Devices Snapshots? Pools? Storage Platform? Create Remove List Attach Detach Probe Publish Unpublish Capabilities Create Remove List Update Inspect List Inspect Services? Master Agent Controller Objects
- 23. © Copyright 2017Dell Inc.23 CSI – A universal storage interface Any Storage or Cloud Platform Docker Mesos Kubernetes Cloud FoundryREX-Ray or others Storage Drivers Kernel Driver FC SAN IP SAN NAS Object Docker Volume Plugins Kubernetes FlexVolume and Provisioner Today Container Storage Orchestration Engines implement CSI Tomorrow Container Orchestration Engines implement CSI directly Cloud Other Frameworks CSI
- 24. © Copyright 2017Dell Inc.24 Our Future Vision for CSI Container Storage Interface client Conformance Library Implementations Specification server tested Container Orchestrator/Run time Storage Plugin tested gRPC libStorage and any storage driver + REX-Ray =
- 25. © Copyright 2017Dell Inc.25 REX-Ray as a CSI implementation Storage Service Container Storage Service Container Storage Plugin Container Container Orchestrato r Cloud Native Storage mount.special REX-Ray Agent Storage Controller Container REX-Ray Controller libStorage Executor Attach or Mount Device or Volume Download Executor Cluster node discovery and introspective operations Attach/Detach/Create /Remove Volume Runs anywhere Container Runtime CSI CSI
- 26. © Copyright 2017Dell Inc.26 Maintaining a plugin isn’t easy Plugins..plugins..plugins! It is a big commitment to maintain all aspects of a quality plugin. Integration Tests Artifact delivery Storage API bindings Storage Operations CLI/Service Filesystem operations Documentation End-to-end Tests Unit Tests Interface API bindings Instance Introspection Device Discovery Obsolescence Container packaging
- 27. © Copyright 2017Dell Inc.27 Building a managed plugin $ git clone https://github.com/codedellemc/rexray $ cd rexray $ DRIVERS=scaleio make build-docker-plugin $ docker plugin ls ID NAME DESCRIPTION ENABLED 725046842d0e rexray/scaleio:0.8.2-5 REX-Ray for Dell EMC ScaleIO false $ ls .docker/plugins/scaleio/Dockerfile .docker/plugins/scaleio/Dockerfile $ ls .docker/plugins/scaleio/config.json .docker/plugins/scaleio/config.json
- 28. © Copyright 2017Dell Inc.28 Summary • CSI is an industry collaboration among container schedulers and storage vendors and is critical to operating smarter applications • libStorage is a client/server library that makes it easier to create storage drivers now that work today and will work with CSI implementations • REX-Ray storage plugins provides interoperability with container runtimes today and tomorrow
- 29. © Copyright 2017Dell Inc.29 codedellemc.com community.codedellemc.com @codeDellEMC blog.codedellemc.com {code} by Dell EMC is a group of passionate open source engineers and advocates working to build a community around software-based infrastructure. rexray.codedellemc.com github.com/codedellemc/labs Clint Kitson @clintonskitson github.com/clintonskitson HOL01 Use REX-Ray & ScaleIO w/ Docker, Mesos and Kubernetes
Editor's Notes
- #3 Add talking points to each point OSS projects: Docker, Mesos, Kubernetes, Cloud Foundry
- #30 First, a few things about the team that has made this possible. The Dell EMC {code} team is a team made up of open source software engineers and developer advocates, focused on making EMC a well-known name within the open source community. We will focus on one of their projects, REX-Ray, in this presentation.