SlideShare a Scribd company logo
Software for the hyperscale datacenter
Florian Leibert, CEO & Founder
flo@mesosphere.io
Imagine if…
All your servers in
your datacenter
and cloud
were pooled together
So they behave like one
big computer
as easy as
…and
building
new
datacenter
apps is as
easy as
building an
app for one
machine PowerBook G4
Fantasy?
Mesos Users Today
Sigmoid Analytics
Today’s Legacy Datacenter
Provision VMs in the cloud or on physical servers
Today’s Legacy Datacenter
Install Hadoop on a static set of machines
Installing an Application with Static Partitioning
Install Web Server on a static set of machines
Installing an Application with Static Partitioning
Scale up Hadoop manually
Resizing an Application with Static Partitioning
What if your Laptop was operated like your Data
Issues with Statically Partitioned Data Centers
Complex

Machine sprawl, manual resize/scale
Limited

No software failure handling, “black box”
Inefficient

Static partitioning, overhead
Not Developer-Friendly

Long time to roll out software, development starts at the
machine level
Aggregation
Mesosphere aggregates resources, makes a data center look like one big
computer
Mesosphere runs on top of a VM or on bare metal
Applications in the Cloud Era
Cloud Era:

Big apps, small servers
Client-Server Era:

Small apps, big servers
Server
Virtualization
App App App App
App
Aggregation
Serv Serv Serv Serv
From Static Partitioning to Elastic Sharing
Static Partitioning
Elastic Sharing
WEB HADOOPCACHE
WASTED
FREEFREE
HADOOP
WEB
CACHE
WASTED WASTED
100% —
100% —
Applications are Changing
Deployments
Mesos Facts
Scales to 10,000s of nodes
Top-level Apache project
Twitter and Airbnb are major users and contributors
APIs for C++, Python, JVM, Go
Pluggable CPU, memory, IO isolation
Packages and commercial support through
Mesosphere
Highly available, scalable, elastic
Apache Mesos Features
Multi-tenancy	

Improved resource utilization	

Resource Isolation	

Fault-tolerance, HA	

Scalability + Elasticity
26
Mesos Slave
Hadoop task-tracker Mesos Executor
Task #1 Task #2 ./ruby XYZ
Mesos Slave
Docker Executor Docker Executor
java -jar XYZ.jar ./xyz
Mesos Master Mesos Master Mesos Master
Hadoop
scheduler
Marathon
scheduler
Zookeeper
quorum
Mesos Slave
Hadoop task-tracker Mesos Executor
Task #1 Task #2 ./ruby XYZ
Mesos Slave
Docker Executor Docker Executor
java -jar XYZ.jar ./xyz
Mesos Master Mesos Master Mesos Master
Hadoop
scheduler
Marathon
scheduler
Zookeeper
quorum
28
Mesos Slave
Hadoop task-tracker Mesos Executor
Task #1 Task #2 ./ruby XYZ
Mesos Slave
Docker Executor Docker Executor
java -jar XYZ.jar ./xyz
Mesos Master Mesos Master Mesos Master
Hadoop
scheduler
Marathon
scheduler
Zookeeper
quorum
29
Mesos Slave
Hadoop task-tracker Mesos Executor
Task #1 Task #2 ./ruby XYZ
Mesos Slave
Docker Executor Docker Executor
java -jar XYZ.jar ./xyz
Mesos Master Mesos Master Mesos Master
Hadoop
scheduler
Marathon
scheduler
Zookeeper
quorum
30
Resource Offers and Launching a Task
1. resourceOffers()
2. launchTasks()
3. launchTask() 4. statusUpdate()
5. statusUpdate()
Mesos Slave
Executor
Mesos Master
Scheduler
Task Task
Response times and overhead are significantly
Timetoprovision(seconds)
1
100
10000
Bare metal VM Container
Inspired by Tomas Barton’s Mesos talk at InstallFest in Prague
Mesos provides fine-grained resource isolation
Compute Node
Mesos Slave Process
Hadoop task-tracker Mesos Executor
Task #1 Task #2 ruby XYZ
Container
(Cgroups)
Executor
Mesos provides fine-grained resource isolation
Compute Node
Mesos Slave Process
Hadoop task-tracker
Task #1 Task #2
Container
(Cgroups)
Task #3
Mesos provides componentized resource
Mesos Slave Process
Mesos Containerizer
CGroups CPU isolator
CGroups Memory isolator
Launcher
Container foo
Task baz
Containerizer API
Executor bar
Mesos provides pluggable resource isolation
External Containerizer
External Containerizer API
Mesos Slave Process
External Containerizer Program
Container foo
MySQL
Containerizer API
Ubuntu 13.10
Container bar
Ruby
Centos 6.4
github.com/mesosphere/deimos
From Static Partitioning to Elastic Sharing
Static Partitioning
Elastic Sharing
WEB HADOOPCACHE
WASTED
FREEFREE
HADOOP
WEB
CACHE
WASTED WASTED
100% —
100% —
Mesos has no single point of failure
Tasks keep running!
Framework
Masters
Master node can fail-over
Tasks keep running!
Framework
Masters
Slave processes can fail-over
Tasks keep running!
Compute Node
Mesos Slave Process
Mesos Executor Mesos Executor
The UNIX Operating System Stack
SSHd
Linux, BSD
MySQLApache
Kernel
ApplicationsMemcached
Init, Upstart, Systemd Init System
The Mesos Stack
Memcached
Mesos
RedisRails
Kernel
ApplicationsElasticsearch
Marathon Init System
Mesosphere Demo
Marathon
What is Marathon?
“Init Daemon” for the data center
•Runs any Linux binary without modification (e.g. Rails,
Tomcat, …)
•Cluster-wide process supervisor
!
Private PaaS
•Service discovery
•Automated software and hardware failure handling
•Deployment and scaling
Marathon Design Goals
Simplify

Fewer things to manage = fewer sources of error

All machines have the same configuration
Automate

Nobody likes to get paged at night

Automatically respond to hardware & software failures
Improve efficiency

Humans are bad at estimating resource requirements

Use software-controlled elastic resource sharing
Self-serve API

Developers want to have control over their apps

Give them direct access to cluster resources
Marathon Key Features
REST/JSON API
Easy to use web interface
Authentication & SSL
Highly available - no single point of failure
Placement constraints (nodes, racks, etc.)
Service discovery & load balancing via HAProxy
Event system for integration with 3rd party components, like load
balancers
!
Marathon Workflow
3.foo.com1.foo.com 2.foo.com
Rails Rails
Mesos
Marathon
POST /v2/apps
Marathon Workflow
1.foo.com 3.foo.com2.foo.com
Rails Rails
Mesos
Marathon
POST /v2/apps
Play
Play
Marathon API - Launching Self-Contained Apps
•Command to start the app
•URL(s) to the app archive/configuration
•Environment variables

!
POST /v2/apps
{
“id”: “Play”,
“uris”: [“http://downloads.mesosphere.io/tutorials/
PlayHello.zip”]
“cmd”: “./Hello-*/bin/hello -Dhttp.port=$PORT”,
“env”: {“SECRET”: “password123”}
}
Marathon API - Launching Dockers
•Starting with Mesos 0.19 containers are 1st class citizens
•Deimos is the Docker containerizer
!
POST /v2/apps
{
“id”: “Cassandra”,
“container”: {
“image”: “docker:///mesosphere/cassandra:2.0.6”,
“options”: [“-v”, “/mnt:/mnt:rw”, “-e”,
“CLUSTER_NAME=prod”]
}
}
Marathon API - Integration with Deimos
Mesos Master
Marathon
Launch 3x Redis
Mesos Slave
Deimos
Task 1
Task 2
docker run ...
Deimos
docker run ...
Mesos Slave
Deimos
docker run ...
Task 3
Marathon API - Scaling Apps
•Just tell Marathon how many you want!
!
!
PATCH /v2/apps/Play
{
“instances”: 4
}
Marathon Service Discovery with HAProxy
Apps available on localhost & known
port
HAProxy updates via Marathon REST
API
HAProxy runs on every cluster node
Configurable policies
Rails
HAProxy
Rails
HAProxy
SQL
HAProxy
Search Search
HAProxy
Cache
Marathon
Mesosphere Products & Services
Infrastructure
• Try out Mesos on Amazon Web Services

https://elastic.mesosphere.io
• On premise and cloud provisioning tools 
• Linux Packages
• Mesosphere Plugins and Tools
• Professional Services
• Training, Installation, Support Contracts
Thank you.

More Related Content

More from Mesosphere Inc.

Java EE Modernization with Mesosphere DCOS
Java EE Modernization with Mesosphere DCOSJava EE Modernization with Mesosphere DCOS
Java EE Modernization with Mesosphere DCOS
Mesosphere Inc.
 
Operating Kubernetes at Scale (Australia Presentation)
Operating Kubernetes at Scale (Australia Presentation)Operating Kubernetes at Scale (Australia Presentation)
Operating Kubernetes at Scale (Australia Presentation)
Mesosphere Inc.
 
Episode 4: Operating Kubernetes at Scale with DC/OS
Episode 4: Operating Kubernetes at Scale with DC/OSEpisode 4: Operating Kubernetes at Scale with DC/OS
Episode 4: Operating Kubernetes at Scale with DC/OS
Mesosphere Inc.
 
Episode 3: Kubernetes and Big Data Services
Episode 3: Kubernetes and Big Data ServicesEpisode 3: Kubernetes and Big Data Services
Episode 3: Kubernetes and Big Data Services
Mesosphere Inc.
 
Episode 2: Deploying Kubernetes at Scale
Episode 2: Deploying Kubernetes at ScaleEpisode 2: Deploying Kubernetes at Scale
Episode 2: Deploying Kubernetes at Scale
Mesosphere Inc.
 
Episode 1: Building Kubernetes-as-a-Service
Episode 1: Building Kubernetes-as-a-ServiceEpisode 1: Building Kubernetes-as-a-Service
Episode 1: Building Kubernetes-as-a-Service
Mesosphere Inc.
 
Best Practices for Managing Kubernetes and Stateful Services: Mesosphere & Sy...
Best Practices for Managing Kubernetes and Stateful Services: Mesosphere & Sy...Best Practices for Managing Kubernetes and Stateful Services: Mesosphere & Sy...
Best Practices for Managing Kubernetes and Stateful Services: Mesosphere & Sy...
Mesosphere Inc.
 
Webinar: What's New in DC/OS 1.11
Webinar: What's New in DC/OS 1.11Webinar: What's New in DC/OS 1.11
Webinar: What's New in DC/OS 1.11
Mesosphere Inc.
 
Webinar: End-to-End CI/CD with GitLab and DC/OS
Webinar: End-to-End CI/CD with GitLab and DC/OSWebinar: End-to-End CI/CD with GitLab and DC/OS
Webinar: End-to-End CI/CD with GitLab and DC/OS
Mesosphere Inc.
 
Webinar: Operating Kubernetes at Scale
Webinar: Operating Kubernetes at ScaleWebinar: Operating Kubernetes at Scale
Webinar: Operating Kubernetes at Scale
Mesosphere Inc.
 
Webinar: Déployez facilement Kubernetes & vos containers
Webinar: Déployez facilement Kubernetes & vos containersWebinar: Déployez facilement Kubernetes & vos containers
Webinar: Déployez facilement Kubernetes & vos containers
Mesosphere Inc.
 
Webinar: Deep Learning Pipelines Beyond the Learning
Webinar: Deep Learning Pipelines Beyond the LearningWebinar: Deep Learning Pipelines Beyond the Learning
Webinar: Deep Learning Pipelines Beyond the Learning
Mesosphere Inc.
 
Running Distributed TensorFlow with GPUs on Mesos with DC/OS
Running Distributed TensorFlow with GPUs on Mesos with DC/OS Running Distributed TensorFlow with GPUs on Mesos with DC/OS
Running Distributed TensorFlow with GPUs on Mesos with DC/OS
Mesosphere Inc.
 
Manage Microservices & Fast Data Systems on One Platform w/ DC/OS
Manage Microservices & Fast Data Systems on One Platform w/ DC/OSManage Microservices & Fast Data Systems on One Platform w/ DC/OS
Manage Microservices & Fast Data Systems on One Platform w/ DC/OS
Mesosphere Inc.
 
Jolt: Distributed, fault-tolerant test running at scale using Mesos
Jolt: Distributed, fault-tolerant test running at scale using MesosJolt: Distributed, fault-tolerant test running at scale using Mesos
Jolt: Distributed, fault-tolerant test running at scale using Mesos
Mesosphere Inc.
 
Deploying Kong with Mesosphere DC/OS
Deploying Kong with Mesosphere DC/OSDeploying Kong with Mesosphere DC/OS
Deploying Kong with Mesosphere DC/OS
Mesosphere Inc.
 
Tech Preview: Kubernetes on Mesosphere DC/OS 1.10
Tech Preview: Kubernetes on Mesosphere DC/OS 1.10Tech Preview: Kubernetes on Mesosphere DC/OS 1.10
Tech Preview: Kubernetes on Mesosphere DC/OS 1.10
Mesosphere Inc.
 
Discover the all new Mesosphere DC/OS 1.10
Discover the all new Mesosphere DC/OS 1.10Discover the all new Mesosphere DC/OS 1.10
Discover the all new Mesosphere DC/OS 1.10
Mesosphere Inc.
 
Mesosphere & Magnetic: Take the pain out of running complex and critical serv...
Mesosphere & Magnetic: Take the pain out of running complex and critical serv...Mesosphere & Magnetic: Take the pain out of running complex and critical serv...
Mesosphere & Magnetic: Take the pain out of running complex and critical serv...
Mesosphere Inc.
 
Easy Docker Deployments with Mesosphere DCOS on Azure
Easy Docker Deployments with Mesosphere DCOS on AzureEasy Docker Deployments with Mesosphere DCOS on Azure
Easy Docker Deployments with Mesosphere DCOS on Azure
Mesosphere Inc.
 

More from Mesosphere Inc. (20)

Java EE Modernization with Mesosphere DCOS
Java EE Modernization with Mesosphere DCOSJava EE Modernization with Mesosphere DCOS
Java EE Modernization with Mesosphere DCOS
 
Operating Kubernetes at Scale (Australia Presentation)
Operating Kubernetes at Scale (Australia Presentation)Operating Kubernetes at Scale (Australia Presentation)
Operating Kubernetes at Scale (Australia Presentation)
 
Episode 4: Operating Kubernetes at Scale with DC/OS
Episode 4: Operating Kubernetes at Scale with DC/OSEpisode 4: Operating Kubernetes at Scale with DC/OS
Episode 4: Operating Kubernetes at Scale with DC/OS
 
Episode 3: Kubernetes and Big Data Services
Episode 3: Kubernetes and Big Data ServicesEpisode 3: Kubernetes and Big Data Services
Episode 3: Kubernetes and Big Data Services
 
Episode 2: Deploying Kubernetes at Scale
Episode 2: Deploying Kubernetes at ScaleEpisode 2: Deploying Kubernetes at Scale
Episode 2: Deploying Kubernetes at Scale
 
Episode 1: Building Kubernetes-as-a-Service
Episode 1: Building Kubernetes-as-a-ServiceEpisode 1: Building Kubernetes-as-a-Service
Episode 1: Building Kubernetes-as-a-Service
 
Best Practices for Managing Kubernetes and Stateful Services: Mesosphere & Sy...
Best Practices for Managing Kubernetes and Stateful Services: Mesosphere & Sy...Best Practices for Managing Kubernetes and Stateful Services: Mesosphere & Sy...
Best Practices for Managing Kubernetes and Stateful Services: Mesosphere & Sy...
 
Webinar: What's New in DC/OS 1.11
Webinar: What's New in DC/OS 1.11Webinar: What's New in DC/OS 1.11
Webinar: What's New in DC/OS 1.11
 
Webinar: End-to-End CI/CD with GitLab and DC/OS
Webinar: End-to-End CI/CD with GitLab and DC/OSWebinar: End-to-End CI/CD with GitLab and DC/OS
Webinar: End-to-End CI/CD with GitLab and DC/OS
 
Webinar: Operating Kubernetes at Scale
Webinar: Operating Kubernetes at ScaleWebinar: Operating Kubernetes at Scale
Webinar: Operating Kubernetes at Scale
 
Webinar: Déployez facilement Kubernetes & vos containers
Webinar: Déployez facilement Kubernetes & vos containersWebinar: Déployez facilement Kubernetes & vos containers
Webinar: Déployez facilement Kubernetes & vos containers
 
Webinar: Deep Learning Pipelines Beyond the Learning
Webinar: Deep Learning Pipelines Beyond the LearningWebinar: Deep Learning Pipelines Beyond the Learning
Webinar: Deep Learning Pipelines Beyond the Learning
 
Running Distributed TensorFlow with GPUs on Mesos with DC/OS
Running Distributed TensorFlow with GPUs on Mesos with DC/OS Running Distributed TensorFlow with GPUs on Mesos with DC/OS
Running Distributed TensorFlow with GPUs on Mesos with DC/OS
 
Manage Microservices & Fast Data Systems on One Platform w/ DC/OS
Manage Microservices & Fast Data Systems on One Platform w/ DC/OSManage Microservices & Fast Data Systems on One Platform w/ DC/OS
Manage Microservices & Fast Data Systems on One Platform w/ DC/OS
 
Jolt: Distributed, fault-tolerant test running at scale using Mesos
Jolt: Distributed, fault-tolerant test running at scale using MesosJolt: Distributed, fault-tolerant test running at scale using Mesos
Jolt: Distributed, fault-tolerant test running at scale using Mesos
 
Deploying Kong with Mesosphere DC/OS
Deploying Kong with Mesosphere DC/OSDeploying Kong with Mesosphere DC/OS
Deploying Kong with Mesosphere DC/OS
 
Tech Preview: Kubernetes on Mesosphere DC/OS 1.10
Tech Preview: Kubernetes on Mesosphere DC/OS 1.10Tech Preview: Kubernetes on Mesosphere DC/OS 1.10
Tech Preview: Kubernetes on Mesosphere DC/OS 1.10
 
Discover the all new Mesosphere DC/OS 1.10
Discover the all new Mesosphere DC/OS 1.10Discover the all new Mesosphere DC/OS 1.10
Discover the all new Mesosphere DC/OS 1.10
 
Mesosphere & Magnetic: Take the pain out of running complex and critical serv...
Mesosphere & Magnetic: Take the pain out of running complex and critical serv...Mesosphere & Magnetic: Take the pain out of running complex and critical serv...
Mesosphere & Magnetic: Take the pain out of running complex and critical serv...
 
Easy Docker Deployments with Mesosphere DCOS on Azure
Easy Docker Deployments with Mesosphere DCOS on AzureEasy Docker Deployments with Mesosphere DCOS on Azure
Easy Docker Deployments with Mesosphere DCOS on Azure
 

Recently uploaded

Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
Drona Infotech
 
Malibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed RoundMalibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed Round
sjcobrien
 
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, FactsALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
Green Software Development
 
14 th Edition of International conference on computer vision
14 th Edition of International conference on computer vision14 th Edition of International conference on computer vision
14 th Edition of International conference on computer vision
ShulagnaSarkar2
 
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdfTop Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
VALiNTRY360
 
Webinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for EmbeddedWebinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for Embedded
ICS
 
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
kalichargn70th171
 
Hand Rolled Applicative User Validation Code Kata
Hand Rolled Applicative User ValidationCode KataHand Rolled Applicative User ValidationCode Kata
Hand Rolled Applicative User Validation Code Kata
Philip Schwarz
 
Unveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdfUnveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdf
brainerhub1
 
J-Spring 2024 - Going serverless with Quarkus, GraalVM native images and AWS ...
J-Spring 2024 - Going serverless with Quarkus, GraalVM native images and AWS ...J-Spring 2024 - Going serverless with Quarkus, GraalVM native images and AWS ...
J-Spring 2024 - Going serverless with Quarkus, GraalVM native images and AWS ...
Bert Jan Schrijver
 
Lecture 2 - software testing SE 412.pptx
Lecture 2 - software testing SE 412.pptxLecture 2 - software testing SE 412.pptx
Lecture 2 - software testing SE 412.pptx
TaghreedAltamimi
 
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
mz5nrf0n
 
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
XfilesPro
 
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
mz5nrf0n
 
SQL Accounting Software Brochure Malaysia
SQL Accounting Software Brochure MalaysiaSQL Accounting Software Brochure Malaysia
SQL Accounting Software Brochure Malaysia
GohKiangHock
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
Hornet Dynamics
 
What next after learning python programming basics
What next after learning python programming basicsWhat next after learning python programming basics
What next after learning python programming basics
Rakesh Kumar R
 
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
dakas1
 
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CDKuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
rodomar2
 
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
gapen1
 

Recently uploaded (20)

Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
 
Malibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed RoundMalibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed Round
 
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, FactsALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
 
14 th Edition of International conference on computer vision
14 th Edition of International conference on computer vision14 th Edition of International conference on computer vision
14 th Edition of International conference on computer vision
 
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdfTop Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
 
Webinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for EmbeddedWebinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for Embedded
 
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
 
Hand Rolled Applicative User Validation Code Kata
Hand Rolled Applicative User ValidationCode KataHand Rolled Applicative User ValidationCode Kata
Hand Rolled Applicative User Validation Code Kata
 
Unveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdfUnveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdf
 
J-Spring 2024 - Going serverless with Quarkus, GraalVM native images and AWS ...
J-Spring 2024 - Going serverless with Quarkus, GraalVM native images and AWS ...J-Spring 2024 - Going serverless with Quarkus, GraalVM native images and AWS ...
J-Spring 2024 - Going serverless with Quarkus, GraalVM native images and AWS ...
 
Lecture 2 - software testing SE 412.pptx
Lecture 2 - software testing SE 412.pptxLecture 2 - software testing SE 412.pptx
Lecture 2 - software testing SE 412.pptx
 
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
 
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
 
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
 
SQL Accounting Software Brochure Malaysia
SQL Accounting Software Brochure MalaysiaSQL Accounting Software Brochure Malaysia
SQL Accounting Software Brochure Malaysia
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
 
What next after learning python programming basics
What next after learning python programming basicsWhat next after learning python programming basics
What next after learning python programming basics
 
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
 
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CDKuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
 
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
 

Apache Mesos and Mesosphere: Live webcast by CEO and Co-Founder Florian Leibert

  • 1. Software for the hyperscale datacenter Florian Leibert, CEO & Founder flo@mesosphere.io
  • 3. All your servers in your datacenter and cloud
  • 5. So they behave like one big computer
  • 7. …and building new datacenter apps is as easy as building an app for one machine PowerBook G4
  • 9.
  • 10.
  • 12.
  • 14. Provision VMs in the cloud or on physical servers Today’s Legacy Datacenter
  • 15. Install Hadoop on a static set of machines Installing an Application with Static Partitioning
  • 16. Install Web Server on a static set of machines Installing an Application with Static Partitioning
  • 17. Scale up Hadoop manually Resizing an Application with Static Partitioning
  • 18. What if your Laptop was operated like your Data
  • 19. Issues with Statically Partitioned Data Centers Complex
 Machine sprawl, manual resize/scale Limited
 No software failure handling, “black box” Inefficient
 Static partitioning, overhead Not Developer-Friendly
 Long time to roll out software, development starts at the machine level
  • 20. Aggregation Mesosphere aggregates resources, makes a data center look like one big computer Mesosphere runs on top of a VM or on bare metal
  • 21. Applications in the Cloud Era Cloud Era:
 Big apps, small servers Client-Server Era:
 Small apps, big servers Server Virtualization App App App App App Aggregation Serv Serv Serv Serv
  • 22. From Static Partitioning to Elastic Sharing Static Partitioning Elastic Sharing WEB HADOOPCACHE WASTED FREEFREE HADOOP WEB CACHE WASTED WASTED 100% — 100% —
  • 25. Mesos Facts Scales to 10,000s of nodes Top-level Apache project Twitter and Airbnb are major users and contributors APIs for C++, Python, JVM, Go Pluggable CPU, memory, IO isolation Packages and commercial support through Mesosphere Highly available, scalable, elastic
  • 26. Apache Mesos Features Multi-tenancy Improved resource utilization Resource Isolation Fault-tolerance, HA Scalability + Elasticity 26
  • 27. Mesos Slave Hadoop task-tracker Mesos Executor Task #1 Task #2 ./ruby XYZ Mesos Slave Docker Executor Docker Executor java -jar XYZ.jar ./xyz Mesos Master Mesos Master Mesos Master Hadoop scheduler Marathon scheduler Zookeeper quorum
  • 28. Mesos Slave Hadoop task-tracker Mesos Executor Task #1 Task #2 ./ruby XYZ Mesos Slave Docker Executor Docker Executor java -jar XYZ.jar ./xyz Mesos Master Mesos Master Mesos Master Hadoop scheduler Marathon scheduler Zookeeper quorum 28
  • 29. Mesos Slave Hadoop task-tracker Mesos Executor Task #1 Task #2 ./ruby XYZ Mesos Slave Docker Executor Docker Executor java -jar XYZ.jar ./xyz Mesos Master Mesos Master Mesos Master Hadoop scheduler Marathon scheduler Zookeeper quorum 29
  • 30. Mesos Slave Hadoop task-tracker Mesos Executor Task #1 Task #2 ./ruby XYZ Mesos Slave Docker Executor Docker Executor java -jar XYZ.jar ./xyz Mesos Master Mesos Master Mesos Master Hadoop scheduler Marathon scheduler Zookeeper quorum 30
  • 31. Resource Offers and Launching a Task 1. resourceOffers() 2. launchTasks() 3. launchTask() 4. statusUpdate() 5. statusUpdate() Mesos Slave Executor Mesos Master Scheduler Task Task
  • 32. Response times and overhead are significantly Timetoprovision(seconds) 1 100 10000 Bare metal VM Container Inspired by Tomas Barton’s Mesos talk at InstallFest in Prague
  • 33. Mesos provides fine-grained resource isolation Compute Node Mesos Slave Process Hadoop task-tracker Mesos Executor Task #1 Task #2 ruby XYZ Container (Cgroups) Executor
  • 34. Mesos provides fine-grained resource isolation Compute Node Mesos Slave Process Hadoop task-tracker Task #1 Task #2 Container (Cgroups) Task #3
  • 35. Mesos provides componentized resource Mesos Slave Process Mesos Containerizer CGroups CPU isolator CGroups Memory isolator Launcher Container foo Task baz Containerizer API Executor bar
  • 36. Mesos provides pluggable resource isolation External Containerizer External Containerizer API Mesos Slave Process External Containerizer Program Container foo MySQL Containerizer API Ubuntu 13.10 Container bar Ruby Centos 6.4 github.com/mesosphere/deimos
  • 37. From Static Partitioning to Elastic Sharing Static Partitioning Elastic Sharing WEB HADOOPCACHE WASTED FREEFREE HADOOP WEB CACHE WASTED WASTED 100% — 100% —
  • 38. Mesos has no single point of failure Tasks keep running! Framework Masters
  • 39. Master node can fail-over Tasks keep running! Framework Masters
  • 40. Slave processes can fail-over Tasks keep running! Compute Node Mesos Slave Process Mesos Executor Mesos Executor
  • 41. The UNIX Operating System Stack SSHd Linux, BSD MySQLApache Kernel ApplicationsMemcached Init, Upstart, Systemd Init System
  • 45. What is Marathon? “Init Daemon” for the data center •Runs any Linux binary without modification (e.g. Rails, Tomcat, …) •Cluster-wide process supervisor ! Private PaaS •Service discovery •Automated software and hardware failure handling •Deployment and scaling
  • 46. Marathon Design Goals Simplify
 Fewer things to manage = fewer sources of error
 All machines have the same configuration Automate
 Nobody likes to get paged at night
 Automatically respond to hardware & software failures Improve efficiency
 Humans are bad at estimating resource requirements
 Use software-controlled elastic resource sharing Self-serve API
 Developers want to have control over their apps
 Give them direct access to cluster resources
  • 47. Marathon Key Features REST/JSON API Easy to use web interface Authentication & SSL Highly available - no single point of failure Placement constraints (nodes, racks, etc.) Service discovery & load balancing via HAProxy Event system for integration with 3rd party components, like load balancers !
  • 48. Marathon Workflow 3.foo.com1.foo.com 2.foo.com Rails Rails Mesos Marathon POST /v2/apps
  • 49. Marathon Workflow 1.foo.com 3.foo.com2.foo.com Rails Rails Mesos Marathon POST /v2/apps Play Play
  • 50. Marathon API - Launching Self-Contained Apps •Command to start the app •URL(s) to the app archive/configuration •Environment variables
 ! POST /v2/apps { “id”: “Play”, “uris”: [“http://downloads.mesosphere.io/tutorials/ PlayHello.zip”] “cmd”: “./Hello-*/bin/hello -Dhttp.port=$PORT”, “env”: {“SECRET”: “password123”} }
  • 51. Marathon API - Launching Dockers •Starting with Mesos 0.19 containers are 1st class citizens •Deimos is the Docker containerizer ! POST /v2/apps { “id”: “Cassandra”, “container”: { “image”: “docker:///mesosphere/cassandra:2.0.6”, “options”: [“-v”, “/mnt:/mnt:rw”, “-e”, “CLUSTER_NAME=prod”] } }
  • 52. Marathon API - Integration with Deimos Mesos Master Marathon Launch 3x Redis Mesos Slave Deimos Task 1 Task 2 docker run ... Deimos docker run ... Mesos Slave Deimos docker run ... Task 3
  • 53. Marathon API - Scaling Apps •Just tell Marathon how many you want! ! ! PATCH /v2/apps/Play { “instances”: 4 }
  • 54.
  • 55.
  • 56.
  • 57. Marathon Service Discovery with HAProxy Apps available on localhost & known port HAProxy updates via Marathon REST API HAProxy runs on every cluster node Configurable policies Rails HAProxy Rails HAProxy SQL HAProxy Search Search HAProxy Cache Marathon
  • 58. Mesosphere Products & Services Infrastructure • Try out Mesos on Amazon Web Services
 https://elastic.mesosphere.io • On premise and cloud provisioning tools • Linux Packages • Mesosphere Plugins and Tools • Professional Services • Training, Installation, Support Contracts