Recommended
More Related Content
What's hot
What's hot (20)
Similar to Managing ScaleIO as Software on Mesos
Similar to Managing ScaleIO as Software on Mesos (20)
Recently uploaded
Recently uploaded (20)
Managing ScaleIO as Software on Mesos
- 1. Managing ScaleIO as Software on Mesos David vonThenen {code} by Dell EMC http://dvonthenen.com @dvonthenen
- 2. © Copyright 2017 Dell Inc.2 Open source at Dell EMC – Contribute to meaningful OSS projects – Create new thought leading OSS applications – Drive awareness of OSS opportunities with Dell EMC product teams – Participate in relevant community engagement projects – Act in the interest of building a community {code} by Dell EMC is a group of passionate open source engineers and advocates working to build a community around software-based infrastructure. Platinum Sponsor
- 3. © Copyright 2017 Dell Inc.3 Agenda • What is Mesos? • What is ScaleIO? • Using ScaleIO on Mesos • Easier way? Yes, ScaleIO Framework! • Demo
- 5. © Copyright 2017 Dell Inc.5 What is Mesos? • From a resource perspective, it’s a cluster manager • From an application perspective, it’s a scheduler • Is a Container Scheduler • Mesos Universal Containerizer – Docker – rkt (CoreOS) – Mesos Containerizer • Fair and efficient workload placement • Adheres to a set of constraints
- 6. © Copyright 2017 Dell Inc.6 Components of Mesos • Zookeeper – Provides leader election through quorum – Enable distributed coordination (cluster state) • Master – Control plane – Enables fine-grained resource sharing (CPU, memory, disk) – Manages Mesos Framework • Agent – Compute. Where the task will run.
- 7. © Copyright 2017 Dell Inc.7 Where can I run Mesos?
- 8. © Copyright 2017 Dell Inc.8 Mesos will run on…
- 9. © Copyright 2017 Dell Inc.9 Mesos Framework • Ability to schedule tasks based on Application needs • Framework implements two components: – Scheduler – Accepts/Denies Resources – Executor – Container + Application • Multiple Frameworks run within the Mesos cluster • Dogfooding – Marathon is a Framework – Marathon is a basic “default” Framework
- 10. © Copyright 2017 Dell Inc.10 Frameworks / Offer-Accept
- 11. © Copyright 2017 Dell Inc.11 Generic Application + More
- 12. © Copyright 2017 Dell Inc.12 Specialized Frameworks
- 13. © Copyright 2017 Dell Inc.13 Framework = Custom Scheduling • Ability to customize the scheduling of your particular application – Customization in the form of: – Run-time? – Availability? – Fault Tolerance? – Hardware Acceleration? – Location?
- 15. © Copyright 2017 Dell Inc.15 What is ScaleIO? Three ways to consume – Build your own - Software-only – Pre-validated - ScaleIO Ready Node – Turnkey with Networking – VxRack FLEX • Platform Agnostic • Media agnostic Software-Defined; Distributed; Data Center Scale; Shared Block Storage SSDHDD NVMePCIe
- 16. © Copyright 2017 Dell Inc.16 Why ScaleIO? Lightweight • Extremely lightweight • Leaves plenty of resources for services • StorageReview’s ScaleIO Performance Testing “We saw exemplary performance in both configurations. The most impressive feat though was how well ScaleIO in HCI did, demonstrating how little impact the storage overhead has on the platform.”
- 17. © Copyright 2017 Dell Inc.17 Why ScaleIO? Grow as you need • Add (or remove) nodes and storage devices on the fly – granular • Cost savings without need to overprovision
- 18. © Copyright 2017 Dell Inc.18 Why ScaleIO? Infrastructure Agnostic • Hardware agnostic • Supports any major OS or a combination within the same system • Community contributed drivers, features, and additional functionality SSDHDD NVMePCIe
- 19. © Copyright 2017 Dell Inc.19 Why ScaleIO? Scalability & Performance • Managed as a single service rather than silos • ScaleIO automatically balances resources for performance and utilization • Result = Predictable performance at scale
- 20. © Copyright 2017 Dell Inc.20 Why ScaleIO? Automation • Easily automatable • Robust CLI and complete REST API allows easy scripting via configuration management or orchestration tools
- 21. © Copyright 2017 Dell Inc.21 Why ScaleIO? Manageability • Minutes to discover and deploy • Standard x86 servers and Ethernet makes operations a breeze • Virtual IPs and auto- failover for MDM cluster
- 22. © Copyright 2017 Dell Inc.22 Why ScaleIO? Availability & Resiliency • Built-in data protection • Distributed Mesh Mirror • Extremely fast rebuild times for resiliency
- 23. © Copyright 2017 Dell Inc.23 Go for a test drive! • Try ScaleIO as a free download! • https://www.emc.com/products-solutions/trial-software-download/scaleio.htm
- 24. ScaleIO on Mesos Let’s deploy it!
- 25. © Copyright 2017 Dell Inc.25 Install Components • Hefty installation manual… but the components are: • MDM cluster – MDM rpm/deb – LIA rpm/deb – Gateway rpm/deb • ScaleIO Data Server – SDS rpm/deb • ScaleIO Data Client – SDC rpm/deb • What about doing this on 100s or 1000s of nodes?
- 26. © Copyright 2017 Dell Inc.26 Example Deployment • 3 Rack Hyper-converged Deployment • MDM Cluster – 3 MDM nodes – Install packages: 3 x MDM, 3 x LIA, and 3 x Gateway • 41 x 3 racks = 123 servers – Install packages: 123 x SDS, 123 x SDC • 255 packages to install!!
- 27. © Copyright 2017 Dell Inc.27 Well… DevOps • DevOps isn’t the magic bullet… • Resources on-premise and in the cloud • Different operating systems • Same operating systems, different versions • Understand operation details of applications • Interfaces change with software versions • Hardware isn’t the same
- 28. © Copyright 2017 Dell Inc.28 Monitoring? • What about maintenance? • Who is going to monitor this thing? • Still need to know when things go south!
- 29. Easier way? Yes, the ScaleIO Framework
- 30. © Copyright 2017 Dell Inc.30 Easier way?
- 31. © Copyright 2017 Dell Inc.31 ScaleIO Framework • Idea! Let’s create a Software-based Storage Framework • ScaleIO + Mesos Framework = Awesome Sauce! • First release in Sept 2016. Now on version 0.3.1 • https://github.com/codedellemc/scaleio-framework
- 32. © Copyright 2017 Dell Inc.32 What the Framework provides… • Framework installs and configures ScaleIO on all Mesos Agent (compute) nodes • Stand up ScaleIO without knowing operational details • Monitor the storage platform • Persistent storage native to the container scheduling platform • Globally accessible storage! • What storage array? Reduce complexity • Deploy Anywhere!
- 33. © Copyright 2017 Dell Inc.33 Why do we care? • Many container workloads are long running • Most applications have state: user data, configuration, and etc • Top 7 of 12 Apps in Docker Hub are persistent applications!
- 34. © Copyright 2017 Dell Inc.34 Death of a Container • Where does my data go? • Turned to the compute node’s local disk to store data • What happens on a node failure? • Production applications require high availability • External Storage /etc /var /bin /opt /data
- 35. © Copyright 2017 Dell Inc.35 What this means for your Apps… • No data loss on infrastructure failures • Highly Available containers and Apps! • Insulates changes with: – Container scheduler (APIs, etc) – Storage platform (workflows, APIs, etc) • Production ready!
- 36. © Copyright 2017 Dell Inc.36 Schedulers + SDS = ???
- 37. © Copyright 2017 Dell Inc.37 Manages the Storage Enablement • ScaleIO Framework installs and configures software that enables Mesos containers to provision and use ScaleIO volumes! • REX-Ray – Vendor agnostic storage orchestration engine – AWS, Azure, Ceph, DigitalOcean, GCE, ScaleIO, VirtualBox, and more – Home: https://rexray.codedellemc.com – GitHub: https://github.com/codedellemc/rexray • mesos-module-dvdi – Provides hooks for Mesos Agent nodes to manage external storage – Contributed back to Apache Mesos – GitHub: https://github.com/codedellemc/mesos-module-dvdi
- 39. © Copyright 2017 Dell Inc.39 Manage the Storage Enablement • Mesos Configuration – 3 Mesos Master nodes (Management) – 3 Mesos Agent nodes (Compute) • ScaleIO Cluster (Scale-out storage) – Will install on top of 3 Mesos Agent nodes – 180 GB local disks on each node to make up this Storage Pool • ScaleIO Framework – GitHub: https://github.com/codedellemc/scaleio-framework • Persistent External Storage – Using REX-Ray › GitHub: https://github.com/emccode/rexray – Using mesos-module-dvdi › GitHub: https://github.com/emccode/mesos-module-dvdi
- 40. © Copyright 2017 Dell Inc.40 The Moving Parts Scheduler Mesos Cluster Mesos Agent App Mesos Agent App Offer Offer
- 41. Take our projects for a spin at the {code} Labs http://github.com/codedellemc/labs
- 42. © Copyright 2017 Dell Inc.42 codedellemc.com community.codedellemc.com @codeDellEMC blog.codedellemc.com {code} by Dell EMC is a group of passionate open source engineers and advocates working to build a community around software-based infrastructure. rexray.codedellemc.com github.com/codedellemc/labs HOL01 Use REX-Ray & ScaleIO w/ Docker, Mesos and Kubernetes David vonThenen @dvonthenen http://dvonthenen.com
Editor's Notes
- I work on the {code} team which is a group within Dell EMC that develops and contributes to various open source projects in the container space. Our group is most known for working on projects enabling different container schedulers to consume different storage platforms easily. Past Life: Backup and Recovery Solution working on Vmware virtualized environments.
- Add talking points to each point OSS projects: Docker (REX-Ray), Mesos (mesos-module-dvdi), Kubernetes, Cloud Foundry Recently the {code} team has been engaged with the Cloud Native Computing Foundation (CNCF) working on the Container Storage Interface (CSI)
- How do I divide the resources between my apps When an app goes down, you want to bring the app back up Even more basic than that… Mesos is a Container Scheduler Spread your apps amongst compute Create 3 instances of my app
- - How do I efficiently utilize my CPU, memory, and disk? - It makes sense to put a CPU intensive app with an App that requires a lot of memory. Density!
- Dive a little deeper… The one really cool thing about Mesos is this concept of Frameworks. Allows you to override the behavior of how your application is placed in your cluster by implementing a customized scheduler. Maybe users out there may only interact with Marathon in their day to day activities.
- 3 instances running on different nodes, 3 instances on different racks Special fault tolerance requirements Maybe your application is compute intensive such that you need utilize GPUs Spread your application across different availability zones
- Software only Complete flexibility – No HCL Not to say that some infrastructure won’t work better than others. But it’s designed to run on anything. End user supplies server, switch, rack, etc. Worried about a hidden cost of HW integration: Dell PowerEdge servers tuned, optimized and validated for ScaleIO Builds for Hyper-converged or Storage-only ScaleIO can be installed on almost any Operating System; there are plugins or integrations that allow almost any platform to either provide or consume storage. You can combine spinning disks, SSDs, PCI express drives and NVMe in a single system – allowing users to create different performance pools
- Some SDS products eat a lot of resources to provide the storage services. Ideal for Hyper-converged Infrastructure because the storage system doesn’t get in the way of the other compute loads. Storage review was so impressed it created an Editors Choice award so it could give it to ScaleIO.
- Is built on an elastic architecture, meaning… if you need more capacity, you could add more disk to nodes If you need more throughput, you could add more nodes to your configuration Grow as you need
- Bring whatever network and server gear you like, and fill it with the sort of drives you want. Run it on the OS or hypervisor of your choice, and even have the option to combine these in a single system. Open source projects and collaborations continually enable new integrations … and we’ll be hearing about some of them today!
- What happens too often is that storage, ends up getting silo’ed leading to poor utilization and/or performance. An example of that is under utilized resources on a storage array because you need to anticipate growth. By pooling resources across all stacks and allowing for consumption back to any stack, ScaleIO allows for better utilization of your hardware. ScaleIO does all of the work to optimize placement of data and balancing resources. If you lose nodes or add more, ScaleIO automatically redistributes and rebalances the data. What this means is that we get predictable performance at large scale even at near full capacity.
- Since ScaleIO is a Software-based Storage Platform its can mutliple automation vectors from a wide variety of DevOps tools.
- But Operations are simplified just in virtue of relying only on standard x86 servers and Ethernet. Things that all well-known and understood ScaleIO takes the headaches out of certain problems that would be left to Operations teams to solve. A lot this is addressed by the Metadata Management nodes by virtual of doing leader leader election amongst themselves. The elastic architecture handles the data striping and rebalancing when nodes are added or removed.
- ScaleIO writes data in a distributed mesh mirroring scheme. There are always two copies, on separate fault units, to ensure that data is always protected. Because of the highly parallelized architecture of ScaleIO, if there is a problem with a disk or a storage-contributing node, it can drop that device and rebuild the data elsewhere within seconds or minutes, rather than hours or days.
- The Deployment Guide is 302 pages in its current state.
- In terms of automating the installation this is probably the easy scenario which is: - select 3 nodes for the MDM nodes and set them aside for Management/Metadata - then install everything else (SDS and SDC) on the other nodes.
- What about if you want multiple protection domains? Or systems earmarked as storage providers (SDSs)? Or storage consumers (SDCs) at scale? How to identify them in your DevOps tool?
- That just talks about installation. That is arguably the easy part… what happens when it comes to monitoring and Day 2 type operations.
- Operational details – Has been baked into the Framework via the customer scheduler Monitor the storage platform – That’s a principle feature that Mesos Frameworks provide. Application specialization. Native – in the sense that if you bring new Mesos nodes online, the Framework will imprint those new nodes immediately to consume ScaleIO Deploy Anywhere - ScaleIO is infrastructure agnostic
- First, a few things about the team that has made this possible. The Dell EMC {code} team is a team made up of open source software engineers and developer advocates, focused on making EMC a well-known name within the open source community. We will focus on one of their projects, REX-Ray, in this presentation.