SlideShare a Scribd company logo
1© Copyright 2016 EMC Corporation. All rights reserved. 1© Copyright 2016 EMC Corporation. All rights reserved.
DEEP DIVE WITH MESOS & PERSISTENT
STORAGE FOR APPLICATIONS
DAVID VONTHENEN, EMC {code}
2© Copyright 2016 EMC Corporation. All rights reserved.
• David vonThenen, Developer Advocate - EMC {code}
– Twitter: @dvonthenen
– Blog: http://dvonthenen.com
– Github: https://github.com/dvonthenen
PRESENTER
3© Copyright 2016 EMC Corporation. All rights reserved.
• Quick Review of Mesos
• Frameworks and Two Layer Scheduling
• Production Applications
• Demo
• Q&A
AGENDA
4© Copyright 2016 EMC Corporation. All rights reserved. 4© Copyright 2016 EMC Corporation. All rights reserved.
QUICK REVIEW OF MESOS
5© Copyright 2016 EMC Corporation. All rights reserved.
• From a resource perspective, it’s a cluster manager:
– Pools resources to be centrally managed as a single unit
• From an application perspective, it’s a scheduler:
– Dispatches workloads to consume pooled resources
• Often described as a Data Center Operating System
(DCOS)
– Static versus Dynamic Partition of Resource
– Efficient utilization of CPU, Memory, Disk
• Proven resource scheduler for production workloads
QUICK REVIEW OF MESOS
6© Copyright 2016 EMC Corporation. All rights reserved.
SCHEDULERS ARE DIFFICULT TO BUILD
7© Copyright 2016 EMC Corporation. All rights reserved.
• Master
– Manages Mesos Agents
– Enables fine-grained sharing of resources (CPU, RAM, …)
across frameworks in the form of resource offers.
• Zookeeper
– Enables highly reliable distributed coordination (cluster
state)
• Agent (aka Slave)
– The entity that will be running the task
MESOS COMPONENTS
8© Copyright 2016 EMC Corporation. All rights reserved.
MESOS ARCHITECTURE
9© Copyright 2016 EMC Corporation. All rights reserved.
• Generic
– Anything that can be run on a standard Linux shell
– Provides a mechanism for deploying applications packages
– Supports Docker workloads
• Framework
– A plug-in to Mesos that allows you to implement a sub-
scheduler
– specialize a workload by providing custom placement and
resource allocation
APPLICATION SUPPORT
10© Copyright 2016 EMC Corporation. All rights reserved.
GENERIC APPLICATIONS
11© Copyright 2016 EMC Corporation. All rights reserved. 11© Copyright 2016 EMC Corporation. All rights reserved.
FRAMEWORKS AND
TWO LAYER SCHEDULING
12© Copyright 2016 EMC Corporation. All rights reserved.
• On the surface…
• Allows specialization of
workload by providing
custom placement and
resource allocation
• Provides monitoring of
these workloads to protect
and react when things fail
MESOS FRAMEWORK
13© Copyright 2016 EMC Corporation. All rights reserved.
• Scheduler
– a scheduler that registers with the master to be offered
resources
• Executor
– an executor process that is launched on slave nodes to run
the framework’s tasks
FRAMEWORK COMPONENTS
14© Copyright 2016 EMC Corporation. All rights reserved.
FRAMEWORK ARCHITECTURE
15© Copyright 2016 EMC Corporation. All rights reserved.
FRAMEWORK ARCHITECTURE
16© Copyright 2016 EMC Corporation. All rights reserved.
• Odds are you have been using at least 1 framework
without realizing it…
AS IT TURNS OUT…
• Marathon is a framework
– A framework for performing container
orchestration for long running tasks
• Chronos is a framework
– A framework for performing batch or
scheduled tasks
17© Copyright 2016 EMC Corporation. All rights reserved.
• Zooming into the “Marathon” box…
• Marathon is a Scheduler
• Launching a Marathon task, the “generic”
applications are created inside a “generic” (or
default) executor
• Dogfooding!
MARATHON DISSECTED
18© Copyright 2016 EMC Corporation. All rights reserved.
MESOS FRAMEWORKS
19© Copyright 2016 EMC Corporation. All rights reserved.
• 2 layer scheduling or sub-
scheduling of resources
• Manages the lifecycle of your
workload through application
specific monitoring
• What does this mean
specifically?
DIGGING DEEPER
20© Copyright 2016 EMC Corporation. All rights reserved.
BUILDING HADOOP CLUSTERS
21© Copyright 2016 EMC Corporation. All rights reserved.
MANAGING HADOOP CLUSTERS
22© Copyright 2016 EMC Corporation. All rights reserved.
APPLICATIONS USING HADOOP
23© Copyright 2016 EMC Corporation. All rights reserved.
THIS SEEMS SILLY
24© Copyright 2016 EMC Corporation. All rights reserved.
• Allows you to subsume the default scheduler such
that:
• You can install and configure your application
• Resources can be allocated in a way that optimizes
your applications
• Allows operation real-time automated improvements
for your applications
– Scaling up for load balancing peak times
– Scaling back to free up resources
2 LAYER SCHEDULING
25© Copyright 2016 EMC Corporation. All rights reserved.
• The details of your applications are hidden from the
end user!
– Deployment
– Operations
– Growth
– Disaster recovery
– Performance
• Just a consumer of the application as a service!
THE NON-TECHNICAL VERSION
26© Copyright 2016 EMC Corporation. All rights reserved.
• CouchBase Framework (doesn’t exist, let’s imagine it
does)
– CouchBase is a NoSQL DB
• Monitor the CouchBase instances, when nearing full
– Dynamically add an node using “server-add”
– Then call “rebalance”
• Reactive and Proactive but fully automated
• I think we had all seen this movie…
FOR EXAMPLE
27© Copyright 2016 EMC Corporation. All rights reserved.
SKYNET
28© Copyright 2016 EMC Corporation. All rights reserved. 28© Copyright 2016 EMC Corporation. All rights reserved.
PRODUCTION APPLICATIONS
29© Copyright 2016 EMC Corporation. All rights reserved.
• Containers are ephemeral therefore stateless
• Quickly found the need for state
– Configuration
– Data
– Logging / Auditing
• Containerizers like Mesos and Docker introduced
exporting to local storage
• Failure of a node? hardware? A better way?
EVOLUTION OF STATEFUL
30© Copyright 2016 EMC Corporation. All rights reserved.
• Persistent
– Containers can come and go,
but continue where they left off
– Application Data, Logs, Audit
Trail, etc
• External
– Storage lives outside the Mesos
Agent node
– High Availability
– Tolerates node failure
PERSISTENT EXTERNAL STORAGE
31© Copyright 2016 EMC Corporation. All rights reserved.
• REX-Ray
– Provides a vendor agnostic storage orchestration engine
– AWS, GCE, ScaleIO, XtremIO, many many more
– Github: https://github.com/emccode/rexray
• mesos-module-dvdi
– Provides a hook to Mesos agent nodes to provision, attach,
and detach external storage
– Github: https://github.com/emccode/mesos-module-dvdi
HOW WE ACHIEVE THIS?
32© Copyright 2016 EMC Corporation. All rights reserved.
• We can run “real world” applications
• Persistence – DBs
• Highly Available, Fault Tolerant
• Audit trail – Transaction Logs
• Not worry about the “2am call”
STATELESS TO STATEFUL
33© Copyright 2016 EMC Corporation. All rights reserved.
LET’S FACE IT…
34© Copyright 2016 EMC Corporation. All rights reserved.
• Trouble is… this still represents the “old way” of
thinking
• PREVENT THE CRASH OR FAILURE!
• Problem is you can’t
• What is the alternative?
STATEFUL 2.0?
35© Copyright 2016 EMC Corporation. All rights reserved.
• Mesos frameworks can proactively monitor for thing
going sideways
• Leveraging external persistent storage and
frameworks fail fast and recover quicker!
• Skynet!
FRAMEWORKS + STATEFUL 2.0
36© Copyright 2016 EMC Corporation. All rights reserved. 36© Copyright 2016 EMC Corporation. All rights reserved.
DEMO
37© Copyright 2016 EMC Corporation. All rights reserved.
• Running on Amazon EC2
– Mesos
– Marathon
– Mesos DNS
• Production HA Cluster
– 3 x Master + Zookeeper Nodes (t2.micros)
– 5 x Slave Nodes with Docker Support (t2.medium)
DEMO CONFIGURATION
38© Copyright 2016 EMC Corporation. All rights reserved.
• Elastic Search Framework
– Create and mount external volume
– Pour some data into Elastic Search
– Github: https://github.com/mesos/elasticsearch
• Elastic Search Instances backed by Persistent
External Storage
– Using REX-Ray
– Using mesos-module-dvdi
DEMO: FRAMEWORK + EXTERNAL STORAGE
39© Copyright 2016 EMC Corporation. All rights reserved.
• Deploy Elastic Search Framework using REST API
– Observe Scheduler creation
– Using browser, log into the Elastic Search Framework UI
– Observe Executor nodes get created and attaching to EBS
volumes
• Simulate Failure
– Kill EC2 Instance
– Watch executor recreation and reattach to EBS volumes
DEMO: FRAMEWORK + EXTERNAL STORAGE
40© Copyright 2016 EMC Corporation. All rights reserved.
DEMO: AUTO DEPLOY ELASTIC SEARCH
41© Copyright 2016 EMC Corporation. All rights reserved.
Data Persistence in the New Container World
Wednesday 3PM
Joshua Bernstein
VP of Technology for ETD
Tobi Knaup
CEO & Co-Founder of Mesosphere
Guru Session
42© Copyright 2016 EMC Corporation. All rights reserved.
@EMCcode
@dvonthenen
emccode.com
community.emccode.com
Come visit us at
Booth #1044
or in the vLab
Questions?
43© Copyright 2016 EMC Corporation. All rights reserved. 43© Copyright 2016 EMC Corporation. All rights reserved.
EMC World 2016 - code.14 Deep Dive with Mesos and Persistent Storage for Applications

More Related Content

What's hot

EMC World 2016 - code.10 Jumpstart your Open Source Presence through new Coll...
EMC World 2016 - code.10 Jumpstart your Open Source Presence through new Coll...EMC World 2016 - code.10 Jumpstart your Open Source Presence through new Coll...
EMC World 2016 - code.10 Jumpstart your Open Source Presence through new Coll...
{code}
 
EMC World 2016 - mioaITL.08 Infrastructure as Code: Not Your Parent's Data Ce...
EMC World 2016 - mioaITL.08 Infrastructure as Code: Not Your Parent's Data Ce...EMC World 2016 - mioaITL.08 Infrastructure as Code: Not Your Parent's Data Ce...
EMC World 2016 - mioaITL.08 Infrastructure as Code: Not Your Parent's Data Ce...
{code}
 
Docker Training - June 2015
Docker Training - June 2015Docker Training - June 2015
Docker Training - June 2015
{code}
 
EMC World 2016 - code.04 Extending Mesos for Storage and External Resources
EMC World 2016 - code.04 Extending Mesos for Storage and External ResourcesEMC World 2016 - code.04 Extending Mesos for Storage and External Resources
EMC World 2016 - code.04 Extending Mesos for Storage and External Resources
{code}
 
EMC World 2016 - code.07 Resiliency and Availability of a Cloud Native Infras...
EMC World 2016 - code.07 Resiliency and Availability of a Cloud Native Infras...EMC World 2016 - code.07 Resiliency and Availability of a Cloud Native Infras...
EMC World 2016 - code.07 Resiliency and Availability of a Cloud Native Infras...
{code}
 
Highly Available And Distributed Containers - ContainerCon NA 2016
Highly Available And Distributed Containers - ContainerCon NA 2016Highly Available And Distributed Containers - ContainerCon NA 2016
Highly Available And Distributed Containers - ContainerCon NA 2016
{code}
 
EMC World 2016 - code.03 Introduction to Containers
EMC World 2016 - code.03 Introduction to ContainersEMC World 2016 - code.03 Introduction to Containers
EMC World 2016 - code.03 Introduction to Containers
{code}
 
EMC World 2016 - code.15 Better Together: Scale-Out Databases on Scale-Out St...
EMC World 2016 - code.15 Better Together: Scale-Out Databases on Scale-Out St...EMC World 2016 - code.15 Better Together: Scale-Out Databases on Scale-Out St...
EMC World 2016 - code.15 Better Together: Scale-Out Databases on Scale-Out St...
{code}
 
Automating Your Data Center with RackHD - EMC World 2016
Automating Your Data Center with RackHD - EMC World 2016Automating Your Data Center with RackHD - EMC World 2016
Automating Your Data Center with RackHD - EMC World 2016
Kendrick Coleman
 
Highly Available Persistent Applications in Containers - DockerCon16
Highly Available Persistent Applications in Containers - DockerCon16Highly Available Persistent Applications in Containers - DockerCon16
Highly Available Persistent Applications in Containers - DockerCon16
{code}
 
EMC World 2016 - code.02 Introduction to Immutable Infrastructure
EMC World 2016 - code.02 Introduction to Immutable InfrastructureEMC World 2016 - code.02 Introduction to Immutable Infrastructure
EMC World 2016 - code.02 Introduction to Immutable Infrastructure
{code}
 
EMC World 2016 - cnaITL.01 Adopting An Open Source Strategy
EMC World 2016 - cnaITL.01 Adopting An Open Source StrategyEMC World 2016 - cnaITL.01 Adopting An Open Source Strategy
EMC World 2016 - cnaITL.01 Adopting An Open Source Strategy
{code}
 
EMC World 2016 - Deep Dive with Mesos and Persistent Storage for Applications
EMC World 2016 - Deep Dive with Mesos and Persistent Storage for ApplicationsEMC World 2016 - Deep Dive with Mesos and Persistent Storage for Applications
EMC World 2016 - Deep Dive with Mesos and Persistent Storage for Applications
David vonThenen
 
EMC {code} 2.0 - Charter, Principles, and Execution
EMC {code} 2.0 - Charter, Principles, and ExecutionEMC {code} 2.0 - Charter, Principles, and Execution
EMC {code} 2.0 - Charter, Principles, and Execution
{code}
 
EMC World 2016 - code.11 Intimidate me not - How to Contribute to Large Open ...
EMC World 2016 - code.11 Intimidate me not - How to Contribute to Large Open ...EMC World 2016 - code.11 Intimidate me not - How to Contribute to Large Open ...
EMC World 2016 - code.11 Intimidate me not - How to Contribute to Large Open ...
{code}
 
EMC World 2016 - code.09 Introduction to the Docker Platform
EMC World 2016 - code.09 Introduction to the Docker PlatformEMC World 2016 - code.09 Introduction to the Docker Platform
EMC World 2016 - code.09 Introduction to the Docker Platform
{code}
 
BrightTalk session-The right SDS for your OpenStack Cloud
BrightTalk session-The right SDS for your OpenStack CloudBrightTalk session-The right SDS for your OpenStack Cloud
BrightTalk session-The right SDS for your OpenStack Cloud
Eitan Segal
 
Re-Platforming All the Things
Re-Platforming All the ThingsRe-Platforming All the Things
Re-Platforming All the Things
Mesosphere Inc.
 
EMC World 2016 - code.12 Managing a Large Open Source community at EMC and Do...
EMC World 2016 - code.12 Managing a Large Open Source community at EMC and Do...EMC World 2016 - code.12 Managing a Large Open Source community at EMC and Do...
EMC World 2016 - code.12 Managing a Large Open Source community at EMC and Do...
{code}
 
CoreOS 101 - EMC World 2015
CoreOS 101 - EMC World 2015CoreOS 101 - EMC World 2015
CoreOS 101 - EMC World 2015
Jonas Rosland
 

What's hot (20)

EMC World 2016 - code.10 Jumpstart your Open Source Presence through new Coll...
EMC World 2016 - code.10 Jumpstart your Open Source Presence through new Coll...EMC World 2016 - code.10 Jumpstart your Open Source Presence through new Coll...
EMC World 2016 - code.10 Jumpstart your Open Source Presence through new Coll...
 
EMC World 2016 - mioaITL.08 Infrastructure as Code: Not Your Parent's Data Ce...
EMC World 2016 - mioaITL.08 Infrastructure as Code: Not Your Parent's Data Ce...EMC World 2016 - mioaITL.08 Infrastructure as Code: Not Your Parent's Data Ce...
EMC World 2016 - mioaITL.08 Infrastructure as Code: Not Your Parent's Data Ce...
 
Docker Training - June 2015
Docker Training - June 2015Docker Training - June 2015
Docker Training - June 2015
 
EMC World 2016 - code.04 Extending Mesos for Storage and External Resources
EMC World 2016 - code.04 Extending Mesos for Storage and External ResourcesEMC World 2016 - code.04 Extending Mesos for Storage and External Resources
EMC World 2016 - code.04 Extending Mesos for Storage and External Resources
 
EMC World 2016 - code.07 Resiliency and Availability of a Cloud Native Infras...
EMC World 2016 - code.07 Resiliency and Availability of a Cloud Native Infras...EMC World 2016 - code.07 Resiliency and Availability of a Cloud Native Infras...
EMC World 2016 - code.07 Resiliency and Availability of a Cloud Native Infras...
 
Highly Available And Distributed Containers - ContainerCon NA 2016
Highly Available And Distributed Containers - ContainerCon NA 2016Highly Available And Distributed Containers - ContainerCon NA 2016
Highly Available And Distributed Containers - ContainerCon NA 2016
 
EMC World 2016 - code.03 Introduction to Containers
EMC World 2016 - code.03 Introduction to ContainersEMC World 2016 - code.03 Introduction to Containers
EMC World 2016 - code.03 Introduction to Containers
 
EMC World 2016 - code.15 Better Together: Scale-Out Databases on Scale-Out St...
EMC World 2016 - code.15 Better Together: Scale-Out Databases on Scale-Out St...EMC World 2016 - code.15 Better Together: Scale-Out Databases on Scale-Out St...
EMC World 2016 - code.15 Better Together: Scale-Out Databases on Scale-Out St...
 
Automating Your Data Center with RackHD - EMC World 2016
Automating Your Data Center with RackHD - EMC World 2016Automating Your Data Center with RackHD - EMC World 2016
Automating Your Data Center with RackHD - EMC World 2016
 
Highly Available Persistent Applications in Containers - DockerCon16
Highly Available Persistent Applications in Containers - DockerCon16Highly Available Persistent Applications in Containers - DockerCon16
Highly Available Persistent Applications in Containers - DockerCon16
 
EMC World 2016 - code.02 Introduction to Immutable Infrastructure
EMC World 2016 - code.02 Introduction to Immutable InfrastructureEMC World 2016 - code.02 Introduction to Immutable Infrastructure
EMC World 2016 - code.02 Introduction to Immutable Infrastructure
 
EMC World 2016 - cnaITL.01 Adopting An Open Source Strategy
EMC World 2016 - cnaITL.01 Adopting An Open Source StrategyEMC World 2016 - cnaITL.01 Adopting An Open Source Strategy
EMC World 2016 - cnaITL.01 Adopting An Open Source Strategy
 
EMC World 2016 - Deep Dive with Mesos and Persistent Storage for Applications
EMC World 2016 - Deep Dive with Mesos and Persistent Storage for ApplicationsEMC World 2016 - Deep Dive with Mesos and Persistent Storage for Applications
EMC World 2016 - Deep Dive with Mesos and Persistent Storage for Applications
 
EMC {code} 2.0 - Charter, Principles, and Execution
EMC {code} 2.0 - Charter, Principles, and ExecutionEMC {code} 2.0 - Charter, Principles, and Execution
EMC {code} 2.0 - Charter, Principles, and Execution
 
EMC World 2016 - code.11 Intimidate me not - How to Contribute to Large Open ...
EMC World 2016 - code.11 Intimidate me not - How to Contribute to Large Open ...EMC World 2016 - code.11 Intimidate me not - How to Contribute to Large Open ...
EMC World 2016 - code.11 Intimidate me not - How to Contribute to Large Open ...
 
EMC World 2016 - code.09 Introduction to the Docker Platform
EMC World 2016 - code.09 Introduction to the Docker PlatformEMC World 2016 - code.09 Introduction to the Docker Platform
EMC World 2016 - code.09 Introduction to the Docker Platform
 
BrightTalk session-The right SDS for your OpenStack Cloud
BrightTalk session-The right SDS for your OpenStack CloudBrightTalk session-The right SDS for your OpenStack Cloud
BrightTalk session-The right SDS for your OpenStack Cloud
 
Re-Platforming All the Things
Re-Platforming All the ThingsRe-Platforming All the Things
Re-Platforming All the Things
 
EMC World 2016 - code.12 Managing a Large Open Source community at EMC and Do...
EMC World 2016 - code.12 Managing a Large Open Source community at EMC and Do...EMC World 2016 - code.12 Managing a Large Open Source community at EMC and Do...
EMC World 2016 - code.12 Managing a Large Open Source community at EMC and Do...
 
CoreOS 101 - EMC World 2015
CoreOS 101 - EMC World 2015CoreOS 101 - EMC World 2015
CoreOS 101 - EMC World 2015
 

Viewers also liked

Managing a large open source community - OSCON 2016
Managing a large open source community - OSCON 2016Managing a large open source community - OSCON 2016
Managing a large open source community - OSCON 2016
{code}
 
oVirt and OpenStack integration
oVirt and OpenStack integrationoVirt and OpenStack integration
oVirt and OpenStack integration
Nimbus Concept
 
Puppet – Make stateful apps easier than stateless
Puppet – Make stateful apps easier than statelessPuppet – Make stateful apps easier than stateless
Puppet – Make stateful apps easier than stateless
Starcounter
 
Pets versus Cattle: servers evolved
Pets versus Cattle: servers evolvedPets versus Cattle: servers evolved
Pets versus Cattle: servers evolved
Phil Cryer
 
Servers as Pets v. Servers as Cattle
Servers as Pets v. Servers as CattleServers as Pets v. Servers as Cattle
Servers as Pets v. Servers as Cattle
Radamanthus Batnag
 
Why should i care about stateful containers?
Why should i care about stateful containers?Why should i care about stateful containers?
Why should i care about stateful containers?
ClusterHQ
 
Enterprise wikis: an introduction
Enterprise wikis: an introductionEnterprise wikis: an introduction
Enterprise wikis: an introduction
XWiki
 
Reasons to be cheerful - Fronteers 2010
Reasons to be cheerful - Fronteers 2010Reasons to be cheerful - Fronteers 2010
Reasons to be cheerful - Fronteers 2010
Christian Heilmann
 
Born in the Cloud; Build it Like a Startup
Born in the Cloud; Build it Like a StartupBorn in the Cloud; Build it Like a Startup
Born in the Cloud; Build it Like a Startup
Amazon Web Services
 
DCOS Presentation
DCOS PresentationDCOS Presentation
DCOS Presentation
Jan Repnak
 
Open Source is Good for Both Business and Humanity - DockerCon 2016
Open Source is Good for Both Business and Humanity - DockerCon 2016 Open Source is Good for Both Business and Humanity - DockerCon 2016
Open Source is Good for Both Business and Humanity - DockerCon 2016
{code}
 
Source Code Management Basics
Source Code Management BasicsSource Code Management Basics
Source Code Management Basics
Chris Tankersley
 
Pets vs. Cattle: The Elastic Cloud Story
Pets vs. Cattle: The Elastic Cloud StoryPets vs. Cattle: The Elastic Cloud Story
Pets vs. Cattle: The Elastic Cloud Story
Randy Bias
 
EMC World 2016 - code.05 Automating your Physical Data Center with RackHD
EMC World 2016 - code.05 Automating your Physical Data Center with RackHDEMC World 2016 - code.05 Automating your Physical Data Center with RackHD
EMC World 2016 - code.05 Automating your Physical Data Center with RackHD
{code}
 
Containers: Life Beyond Microservices? by Sushil Kumar, Robin Systems
Containers: Life Beyond Microservices? by Sushil Kumar, Robin SystemsContainers: Life Beyond Microservices? by Sushil Kumar, Robin Systems
Containers: Life Beyond Microservices? by Sushil Kumar, Robin Systems
Docker, Inc.
 

Viewers also liked (15)

Managing a large open source community - OSCON 2016
Managing a large open source community - OSCON 2016Managing a large open source community - OSCON 2016
Managing a large open source community - OSCON 2016
 
oVirt and OpenStack integration
oVirt and OpenStack integrationoVirt and OpenStack integration
oVirt and OpenStack integration
 
Puppet – Make stateful apps easier than stateless
Puppet – Make stateful apps easier than statelessPuppet – Make stateful apps easier than stateless
Puppet – Make stateful apps easier than stateless
 
Pets versus Cattle: servers evolved
Pets versus Cattle: servers evolvedPets versus Cattle: servers evolved
Pets versus Cattle: servers evolved
 
Servers as Pets v. Servers as Cattle
Servers as Pets v. Servers as CattleServers as Pets v. Servers as Cattle
Servers as Pets v. Servers as Cattle
 
Why should i care about stateful containers?
Why should i care about stateful containers?Why should i care about stateful containers?
Why should i care about stateful containers?
 
Enterprise wikis: an introduction
Enterprise wikis: an introductionEnterprise wikis: an introduction
Enterprise wikis: an introduction
 
Reasons to be cheerful - Fronteers 2010
Reasons to be cheerful - Fronteers 2010Reasons to be cheerful - Fronteers 2010
Reasons to be cheerful - Fronteers 2010
 
Born in the Cloud; Build it Like a Startup
Born in the Cloud; Build it Like a StartupBorn in the Cloud; Build it Like a Startup
Born in the Cloud; Build it Like a Startup
 
DCOS Presentation
DCOS PresentationDCOS Presentation
DCOS Presentation
 
Open Source is Good for Both Business and Humanity - DockerCon 2016
Open Source is Good for Both Business and Humanity - DockerCon 2016 Open Source is Good for Both Business and Humanity - DockerCon 2016
Open Source is Good for Both Business and Humanity - DockerCon 2016
 
Source Code Management Basics
Source Code Management BasicsSource Code Management Basics
Source Code Management Basics
 
Pets vs. Cattle: The Elastic Cloud Story
Pets vs. Cattle: The Elastic Cloud StoryPets vs. Cattle: The Elastic Cloud Story
Pets vs. Cattle: The Elastic Cloud Story
 
EMC World 2016 - code.05 Automating your Physical Data Center with RackHD
EMC World 2016 - code.05 Automating your Physical Data Center with RackHDEMC World 2016 - code.05 Automating your Physical Data Center with RackHD
EMC World 2016 - code.05 Automating your Physical Data Center with RackHD
 
Containers: Life Beyond Microservices? by Sushil Kumar, Robin Systems
Containers: Life Beyond Microservices? by Sushil Kumar, Robin SystemsContainers: Life Beyond Microservices? by Sushil Kumar, Robin Systems
Containers: Life Beyond Microservices? by Sushil Kumar, Robin Systems
 

Similar to EMC World 2016 - code.14 Deep Dive with Mesos and Persistent Storage for Applications

EMC World 2016 - Introduction to Mesos and Mesosphere
EMC World 2016 - Introduction to Mesos and MesosphereEMC World 2016 - Introduction to Mesos and Mesosphere
EMC World 2016 - Introduction to Mesos and Mesosphere
David vonThenen
 
OSDC 2016 - Mesos and the Architecture of the New Datacenter by Jörg Schad
OSDC 2016 - Mesos and the Architecture of the New Datacenter by Jörg SchadOSDC 2016 - Mesos and the Architecture of the New Datacenter by Jörg Schad
OSDC 2016 - Mesos and the Architecture of the New Datacenter by Jörg Schad
NETWAYS
 
Cloud Native Applications - DevOps, EMC and Cloud Foundry
Cloud Native Applications - DevOps, EMC and Cloud FoundryCloud Native Applications - DevOps, EMC and Cloud Foundry
Cloud Native Applications - DevOps, EMC and Cloud Foundry
Bob Sokol
 
Webinar: Nightmares of a Container Orchestration System - Jorg Schad
Webinar: Nightmares of a Container Orchestration System - Jorg SchadWebinar: Nightmares of a Container Orchestration System - Jorg Schad
Webinar: Nightmares of a Container Orchestration System - Jorg Schad
Codemotion
 
Webinar - Nightmares of a Container Orchestration System - Jorg Schad
Webinar - Nightmares of a Container Orchestration System - Jorg SchadWebinar - Nightmares of a Container Orchestration System - Jorg Schad
Webinar - Nightmares of a Container Orchestration System - Jorg Schad
Codemotion
 
Deploying Containers in Production and at Scale
Deploying Containers in Production and at ScaleDeploying Containers in Production and at Scale
Deploying Containers in Production and at Scale
Mesosphere Inc.
 
20191201 kubernetes managed weblogic revival - part 1
20191201 kubernetes managed weblogic revival - part 120191201 kubernetes managed weblogic revival - part 1
20191201 kubernetes managed weblogic revival - part 1
makker_nl
 
The Kubernetes WebLogic revival (part 1)
The Kubernetes WebLogic revival (part 1)The Kubernetes WebLogic revival (part 1)
The Kubernetes WebLogic revival (part 1)
Simon Haslam
 
How Container Schedulers and Software-based Storage will Change the Cloud
How Container Schedulers and Software-based Storage will Change the CloudHow Container Schedulers and Software-based Storage will Change the Cloud
How Container Schedulers and Software-based Storage will Change the Cloud
David vonThenen
 
SCaLE 15x - How Container Schedulers and Software-Defined Storage will Change...
SCaLE 15x - How Container Schedulers and Software-Defined Storage will Change...SCaLE 15x - How Container Schedulers and Software-Defined Storage will Change...
SCaLE 15x - How Container Schedulers and Software-Defined Storage will Change...
David vonThenen
 
Using ScaleIO in an OpenStack Environment
Using ScaleIO in an OpenStack EnvironmentUsing ScaleIO in an OpenStack Environment
Using ScaleIO in an OpenStack Environment
Jason Sturgeon
 
Kubernetes One-Click Deployment: Hands-on Workshop (Munich)
Kubernetes One-Click Deployment: Hands-on Workshop (Munich)Kubernetes One-Click Deployment: Hands-on Workshop (Munich)
Kubernetes One-Click Deployment: Hands-on Workshop (Munich)
QAware GmbH
 
Introduction to Apache Mesos and DC/OS
Introduction to Apache Mesos and DC/OSIntroduction to Apache Mesos and DC/OS
Introduction to Apache Mesos and DC/OS
Steve Wong
 
Better Practices when Using Terraform to Manage Oracle Cloud Infrastructure
Better Practices when Using Terraform to Manage Oracle Cloud InfrastructureBetter Practices when Using Terraform to Manage Oracle Cloud Infrastructure
Better Practices when Using Terraform to Manage Oracle Cloud Infrastructure
Simon Haslam
 
Platform as a Service with Kubernetes and Mesos
Platform as a Service with Kubernetes and Mesos Platform as a Service with Kubernetes and Mesos
Platform as a Service with Kubernetes and Mesos
Miguel Zuniga
 
DevOps in Age of Kubernetes
DevOps in Age of KubernetesDevOps in Age of Kubernetes
DevOps in Age of Kubernetes
Mesosphere Inc.
 
DevOps vs. Site Reliability Engineering (SRE) in Age of Kubernetes
DevOps vs. Site Reliability Engineering (SRE) in Age of KubernetesDevOps vs. Site Reliability Engineering (SRE) in Age of Kubernetes
DevOps vs. Site Reliability Engineering (SRE) in Age of Kubernetes
DevOps.com
 
Microservices with Terraform, Docker and the Cloud. JavaOne 2017 2017-10-02
Microservices with Terraform, Docker and the Cloud. JavaOne 2017 2017-10-02Microservices with Terraform, Docker and the Cloud. JavaOne 2017 2017-10-02
Microservices with Terraform, Docker and the Cloud. JavaOne 2017 2017-10-02
Derek Ashmore
 
Highly available nodejs
Highly available nodejsHighly available nodejs
Highly available nodejs
pfremm
 
CI/CD with Docker, DC/OS, and Jenkins
CI/CD with Docker, DC/OS, and JenkinsCI/CD with Docker, DC/OS, and Jenkins
CI/CD with Docker, DC/OS, and Jenkins
Karl Isenberg
 

Similar to EMC World 2016 - code.14 Deep Dive with Mesos and Persistent Storage for Applications (20)

EMC World 2016 - Introduction to Mesos and Mesosphere
EMC World 2016 - Introduction to Mesos and MesosphereEMC World 2016 - Introduction to Mesos and Mesosphere
EMC World 2016 - Introduction to Mesos and Mesosphere
 
OSDC 2016 - Mesos and the Architecture of the New Datacenter by Jörg Schad
OSDC 2016 - Mesos and the Architecture of the New Datacenter by Jörg SchadOSDC 2016 - Mesos and the Architecture of the New Datacenter by Jörg Schad
OSDC 2016 - Mesos and the Architecture of the New Datacenter by Jörg Schad
 
Cloud Native Applications - DevOps, EMC and Cloud Foundry
Cloud Native Applications - DevOps, EMC and Cloud FoundryCloud Native Applications - DevOps, EMC and Cloud Foundry
Cloud Native Applications - DevOps, EMC and Cloud Foundry
 
Webinar: Nightmares of a Container Orchestration System - Jorg Schad
Webinar: Nightmares of a Container Orchestration System - Jorg SchadWebinar: Nightmares of a Container Orchestration System - Jorg Schad
Webinar: Nightmares of a Container Orchestration System - Jorg Schad
 
Webinar - Nightmares of a Container Orchestration System - Jorg Schad
Webinar - Nightmares of a Container Orchestration System - Jorg SchadWebinar - Nightmares of a Container Orchestration System - Jorg Schad
Webinar - Nightmares of a Container Orchestration System - Jorg Schad
 
Deploying Containers in Production and at Scale
Deploying Containers in Production and at ScaleDeploying Containers in Production and at Scale
Deploying Containers in Production and at Scale
 
20191201 kubernetes managed weblogic revival - part 1
20191201 kubernetes managed weblogic revival - part 120191201 kubernetes managed weblogic revival - part 1
20191201 kubernetes managed weblogic revival - part 1
 
The Kubernetes WebLogic revival (part 1)
The Kubernetes WebLogic revival (part 1)The Kubernetes WebLogic revival (part 1)
The Kubernetes WebLogic revival (part 1)
 
How Container Schedulers and Software-based Storage will Change the Cloud
How Container Schedulers and Software-based Storage will Change the CloudHow Container Schedulers and Software-based Storage will Change the Cloud
How Container Schedulers and Software-based Storage will Change the Cloud
 
SCaLE 15x - How Container Schedulers and Software-Defined Storage will Change...
SCaLE 15x - How Container Schedulers and Software-Defined Storage will Change...SCaLE 15x - How Container Schedulers and Software-Defined Storage will Change...
SCaLE 15x - How Container Schedulers and Software-Defined Storage will Change...
 
Using ScaleIO in an OpenStack Environment
Using ScaleIO in an OpenStack EnvironmentUsing ScaleIO in an OpenStack Environment
Using ScaleIO in an OpenStack Environment
 
Kubernetes One-Click Deployment: Hands-on Workshop (Munich)
Kubernetes One-Click Deployment: Hands-on Workshop (Munich)Kubernetes One-Click Deployment: Hands-on Workshop (Munich)
Kubernetes One-Click Deployment: Hands-on Workshop (Munich)
 
Introduction to Apache Mesos and DC/OS
Introduction to Apache Mesos and DC/OSIntroduction to Apache Mesos and DC/OS
Introduction to Apache Mesos and DC/OS
 
Better Practices when Using Terraform to Manage Oracle Cloud Infrastructure
Better Practices when Using Terraform to Manage Oracle Cloud InfrastructureBetter Practices when Using Terraform to Manage Oracle Cloud Infrastructure
Better Practices when Using Terraform to Manage Oracle Cloud Infrastructure
 
Platform as a Service with Kubernetes and Mesos
Platform as a Service with Kubernetes and Mesos Platform as a Service with Kubernetes and Mesos
Platform as a Service with Kubernetes and Mesos
 
DevOps in Age of Kubernetes
DevOps in Age of KubernetesDevOps in Age of Kubernetes
DevOps in Age of Kubernetes
 
DevOps vs. Site Reliability Engineering (SRE) in Age of Kubernetes
DevOps vs. Site Reliability Engineering (SRE) in Age of KubernetesDevOps vs. Site Reliability Engineering (SRE) in Age of Kubernetes
DevOps vs. Site Reliability Engineering (SRE) in Age of Kubernetes
 
Microservices with Terraform, Docker and the Cloud. JavaOne 2017 2017-10-02
Microservices with Terraform, Docker and the Cloud. JavaOne 2017 2017-10-02Microservices with Terraform, Docker and the Cloud. JavaOne 2017 2017-10-02
Microservices with Terraform, Docker and the Cloud. JavaOne 2017 2017-10-02
 
Highly available nodejs
Highly available nodejsHighly available nodejs
Highly available nodejs
 
CI/CD with Docker, DC/OS, and Jenkins
CI/CD with Docker, DC/OS, and JenkinsCI/CD with Docker, DC/OS, and Jenkins
CI/CD with Docker, DC/OS, and Jenkins
 

Recently uploaded

FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdfFIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
Ana-Maria Mihalceanu
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance
 
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
James Anderson
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
BookNet Canada
 
Elevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object CalisthenicsElevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object Calisthenics
Dorra BARTAGUIZ
 
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Nexer Digital
 
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
Neo4j
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
Alan Dix
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
DianaGray10
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Albert Hoitingh
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
Uni Systems S.M.S.A.
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Paige Cruz
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
ControlCase
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 

Recently uploaded (20)

FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdfFIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
 
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
 
Elevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object CalisthenicsElevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object Calisthenics
 
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?
 
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 

EMC World 2016 - code.14 Deep Dive with Mesos and Persistent Storage for Applications

  • 1. 1© Copyright 2016 EMC Corporation. All rights reserved. 1© Copyright 2016 EMC Corporation. All rights reserved. DEEP DIVE WITH MESOS & PERSISTENT STORAGE FOR APPLICATIONS DAVID VONTHENEN, EMC {code}
  • 2. 2© Copyright 2016 EMC Corporation. All rights reserved. • David vonThenen, Developer Advocate - EMC {code} – Twitter: @dvonthenen – Blog: http://dvonthenen.com – Github: https://github.com/dvonthenen PRESENTER
  • 3. 3© Copyright 2016 EMC Corporation. All rights reserved. • Quick Review of Mesos • Frameworks and Two Layer Scheduling • Production Applications • Demo • Q&A AGENDA
  • 4. 4© Copyright 2016 EMC Corporation. All rights reserved. 4© Copyright 2016 EMC Corporation. All rights reserved. QUICK REVIEW OF MESOS
  • 5. 5© Copyright 2016 EMC Corporation. All rights reserved. • From a resource perspective, it’s a cluster manager: – Pools resources to be centrally managed as a single unit • From an application perspective, it’s a scheduler: – Dispatches workloads to consume pooled resources • Often described as a Data Center Operating System (DCOS) – Static versus Dynamic Partition of Resource – Efficient utilization of CPU, Memory, Disk • Proven resource scheduler for production workloads QUICK REVIEW OF MESOS
  • 6. 6© Copyright 2016 EMC Corporation. All rights reserved. SCHEDULERS ARE DIFFICULT TO BUILD
  • 7. 7© Copyright 2016 EMC Corporation. All rights reserved. • Master – Manages Mesos Agents – Enables fine-grained sharing of resources (CPU, RAM, …) across frameworks in the form of resource offers. • Zookeeper – Enables highly reliable distributed coordination (cluster state) • Agent (aka Slave) – The entity that will be running the task MESOS COMPONENTS
  • 8. 8© Copyright 2016 EMC Corporation. All rights reserved. MESOS ARCHITECTURE
  • 9. 9© Copyright 2016 EMC Corporation. All rights reserved. • Generic – Anything that can be run on a standard Linux shell – Provides a mechanism for deploying applications packages – Supports Docker workloads • Framework – A plug-in to Mesos that allows you to implement a sub- scheduler – specialize a workload by providing custom placement and resource allocation APPLICATION SUPPORT
  • 10. 10© Copyright 2016 EMC Corporation. All rights reserved. GENERIC APPLICATIONS
  • 11. 11© Copyright 2016 EMC Corporation. All rights reserved. 11© Copyright 2016 EMC Corporation. All rights reserved. FRAMEWORKS AND TWO LAYER SCHEDULING
  • 12. 12© Copyright 2016 EMC Corporation. All rights reserved. • On the surface… • Allows specialization of workload by providing custom placement and resource allocation • Provides monitoring of these workloads to protect and react when things fail MESOS FRAMEWORK
  • 13. 13© Copyright 2016 EMC Corporation. All rights reserved. • Scheduler – a scheduler that registers with the master to be offered resources • Executor – an executor process that is launched on slave nodes to run the framework’s tasks FRAMEWORK COMPONENTS
  • 14. 14© Copyright 2016 EMC Corporation. All rights reserved. FRAMEWORK ARCHITECTURE
  • 15. 15© Copyright 2016 EMC Corporation. All rights reserved. FRAMEWORK ARCHITECTURE
  • 16. 16© Copyright 2016 EMC Corporation. All rights reserved. • Odds are you have been using at least 1 framework without realizing it… AS IT TURNS OUT… • Marathon is a framework – A framework for performing container orchestration for long running tasks • Chronos is a framework – A framework for performing batch or scheduled tasks
  • 17. 17© Copyright 2016 EMC Corporation. All rights reserved. • Zooming into the “Marathon” box… • Marathon is a Scheduler • Launching a Marathon task, the “generic” applications are created inside a “generic” (or default) executor • Dogfooding! MARATHON DISSECTED
  • 18. 18© Copyright 2016 EMC Corporation. All rights reserved. MESOS FRAMEWORKS
  • 19. 19© Copyright 2016 EMC Corporation. All rights reserved. • 2 layer scheduling or sub- scheduling of resources • Manages the lifecycle of your workload through application specific monitoring • What does this mean specifically? DIGGING DEEPER
  • 20. 20© Copyright 2016 EMC Corporation. All rights reserved. BUILDING HADOOP CLUSTERS
  • 21. 21© Copyright 2016 EMC Corporation. All rights reserved. MANAGING HADOOP CLUSTERS
  • 22. 22© Copyright 2016 EMC Corporation. All rights reserved. APPLICATIONS USING HADOOP
  • 23. 23© Copyright 2016 EMC Corporation. All rights reserved. THIS SEEMS SILLY
  • 24. 24© Copyright 2016 EMC Corporation. All rights reserved. • Allows you to subsume the default scheduler such that: • You can install and configure your application • Resources can be allocated in a way that optimizes your applications • Allows operation real-time automated improvements for your applications – Scaling up for load balancing peak times – Scaling back to free up resources 2 LAYER SCHEDULING
  • 25. 25© Copyright 2016 EMC Corporation. All rights reserved. • The details of your applications are hidden from the end user! – Deployment – Operations – Growth – Disaster recovery – Performance • Just a consumer of the application as a service! THE NON-TECHNICAL VERSION
  • 26. 26© Copyright 2016 EMC Corporation. All rights reserved. • CouchBase Framework (doesn’t exist, let’s imagine it does) – CouchBase is a NoSQL DB • Monitor the CouchBase instances, when nearing full – Dynamically add an node using “server-add” – Then call “rebalance” • Reactive and Proactive but fully automated • I think we had all seen this movie… FOR EXAMPLE
  • 27. 27© Copyright 2016 EMC Corporation. All rights reserved. SKYNET
  • 28. 28© Copyright 2016 EMC Corporation. All rights reserved. 28© Copyright 2016 EMC Corporation. All rights reserved. PRODUCTION APPLICATIONS
  • 29. 29© Copyright 2016 EMC Corporation. All rights reserved. • Containers are ephemeral therefore stateless • Quickly found the need for state – Configuration – Data – Logging / Auditing • Containerizers like Mesos and Docker introduced exporting to local storage • Failure of a node? hardware? A better way? EVOLUTION OF STATEFUL
  • 30. 30© Copyright 2016 EMC Corporation. All rights reserved. • Persistent – Containers can come and go, but continue where they left off – Application Data, Logs, Audit Trail, etc • External – Storage lives outside the Mesos Agent node – High Availability – Tolerates node failure PERSISTENT EXTERNAL STORAGE
  • 31. 31© Copyright 2016 EMC Corporation. All rights reserved. • REX-Ray – Provides a vendor agnostic storage orchestration engine – AWS, GCE, ScaleIO, XtremIO, many many more – Github: https://github.com/emccode/rexray • mesos-module-dvdi – Provides a hook to Mesos agent nodes to provision, attach, and detach external storage – Github: https://github.com/emccode/mesos-module-dvdi HOW WE ACHIEVE THIS?
  • 32. 32© Copyright 2016 EMC Corporation. All rights reserved. • We can run “real world” applications • Persistence – DBs • Highly Available, Fault Tolerant • Audit trail – Transaction Logs • Not worry about the “2am call” STATELESS TO STATEFUL
  • 33. 33© Copyright 2016 EMC Corporation. All rights reserved. LET’S FACE IT…
  • 34. 34© Copyright 2016 EMC Corporation. All rights reserved. • Trouble is… this still represents the “old way” of thinking • PREVENT THE CRASH OR FAILURE! • Problem is you can’t • What is the alternative? STATEFUL 2.0?
  • 35. 35© Copyright 2016 EMC Corporation. All rights reserved. • Mesos frameworks can proactively monitor for thing going sideways • Leveraging external persistent storage and frameworks fail fast and recover quicker! • Skynet! FRAMEWORKS + STATEFUL 2.0
  • 36. 36© Copyright 2016 EMC Corporation. All rights reserved. 36© Copyright 2016 EMC Corporation. All rights reserved. DEMO
  • 37. 37© Copyright 2016 EMC Corporation. All rights reserved. • Running on Amazon EC2 – Mesos – Marathon – Mesos DNS • Production HA Cluster – 3 x Master + Zookeeper Nodes (t2.micros) – 5 x Slave Nodes with Docker Support (t2.medium) DEMO CONFIGURATION
  • 38. 38© Copyright 2016 EMC Corporation. All rights reserved. • Elastic Search Framework – Create and mount external volume – Pour some data into Elastic Search – Github: https://github.com/mesos/elasticsearch • Elastic Search Instances backed by Persistent External Storage – Using REX-Ray – Using mesos-module-dvdi DEMO: FRAMEWORK + EXTERNAL STORAGE
  • 39. 39© Copyright 2016 EMC Corporation. All rights reserved. • Deploy Elastic Search Framework using REST API – Observe Scheduler creation – Using browser, log into the Elastic Search Framework UI – Observe Executor nodes get created and attaching to EBS volumes • Simulate Failure – Kill EC2 Instance – Watch executor recreation and reattach to EBS volumes DEMO: FRAMEWORK + EXTERNAL STORAGE
  • 40. 40© Copyright 2016 EMC Corporation. All rights reserved. DEMO: AUTO DEPLOY ELASTIC SEARCH
  • 41. 41© Copyright 2016 EMC Corporation. All rights reserved. Data Persistence in the New Container World Wednesday 3PM Joshua Bernstein VP of Technology for ETD Tobi Knaup CEO & Co-Founder of Mesosphere Guru Session
  • 42. 42© Copyright 2016 EMC Corporation. All rights reserved. @EMCcode @dvonthenen emccode.com community.emccode.com Come visit us at Booth #1044 or in the vLab Questions?
  • 43. 43© Copyright 2016 EMC Corporation. All rights reserved. 43© Copyright 2016 EMC Corporation. All rights reserved.