Operating apps at web scale has become the new normal, but has been out of reach for most companies. Join us as we show you how to deploy and manage your Docker containers at scale. See how easy it is to build highly-available, fault-tolerant web scale apps using Docker with the Mesos cluster scheduler. Docker plus Mesos is a new way to scale applications. Together they give you capabilities similar to Google’s Borg, the Googleplex’s secret weapon of scalability and fault tolerance.
3. Applications in the Cloud Era
App App App App
Client-Server Era:
Small apps, big servers
App
Serv Serv Serv Serv
Cloud Era:
Big apps, small servers
Virtualization
Aggregation
Server
4. Major Components
● Hardware
● Orchestration, deployment and isolation
● Cluster and resource management
● Scale-aware applications, service discovery, etc.
5. Major Components
● Hardware
● Orchestration, deployment and isolation
● Cluster and resource management
● Scale-aware applications, service discovery
6. Applications in the Cloud Era
App App App App
Client-Server Era:
Small apps, big servers
App
Serv Serv Serv Serv
Cloud Era:
Big apps, small servers
Virtualization
Aggregation
Server
7. “Divide et impera [et sarcina]”
“Divide and rule [and package]”
—Philip II of Macedon, paraphrased
8. Containers
● Lightweight Linux execution environment
● Static application composition
● Reliable deployment
● Unit of resource isolation
● Execution isolation
● Multi-tenancy without heavyweight VMs
9. Docker
● Open source
● Configurable layers
● Reproducible
● Version-controlled
● Plenty of other people’s containers
10. Docker
● Open source
● Configurable layers
● Reproducible
● Version-controlled
● Plenty of other people’s containers
● First-class citizen in Mesos and the Mesosphere stack
● Kubernetes employs and promotes Docker
15. Apache Mesos facts
● Created in 2009 at UC Berkeley, hardened in Twitter
● Top-level Apache project
● Mesosphere, Twitter, and Airbnb are major users /
contributors
● Scales to 10 000s of nodes, production grade
● Packages and support through Mesosphere
● Google officially endorsed Mesos for Kubernetes
● Built-in containerization, including Docker
16. Mesos as a Distributed OS kernel
● Two level resource scheduling
● Launch tasks across the cluster
● Communication between tasks (like IPC)
● APIs for building “native” applications (aka frameworks):
program against the datacenter
● APIs in C++, Python, JVM-languages, Go and counting
● Pluggable CPU, memory, IO isolation
● Multi-tenant workloads
● Failure detection
● Easy failover and HA
24. What Mesos contributes
● Multi-framework: weighted fair sharing, roles, etc.
● Run Docker containers alongside other popular frameworks
(e.g. Spark, Rails, Hadoop, …)
● Run services and batch apps in the same cluster
● Advanced scheduling: resources, constraints, global view
● High resource availability, cluster self-healing
● Proven at scale, battle-tested in production
● GUI / CLI cluster management console
25. Ways of running Dockers
● Marathon
easy to setup, reliable orchestration
● Multiple Marathons
benefit from two level scheduling, reservations, framework
roles
● Custom framework
fine-grained management
26. “L’homme est libre au moment qu’il veut l’être”
“Man is free at the instant he wants to be”
—Voltaire
27. Cluster configuration example #1
Services
Mesos
App
App
AWS, DigitalOcean, GCE
Apps
API
Kernel
Hardware
Docker
Docker
Services REST API
“Marathon”
App
Docker
App
App
Docker
App
28. Cluster configuration example #2
Mesos
App
App
AWS, DigitalOcean, GCE
Apps
API
Kernel
Hardware
Services
Services REST API
“Marathon”
App
App
Kubernetes
Docker
Docker
Docker
App
Docker
App
29. Cluster configuration example #3
Docker
Services REST API
“Marathon” (init)
Serv Serv Serv Serv
Mesos
Spark, MPI,
Hadoop, Storm
Mesos SDK
App
Java, Python, C++, Go
Serv Serv Serv Serv
Batch REST API
“Chronos” (cron)
Serv
App Recurring Jobs
(ETL, backups)
Hardware
Native Long running Batch
Apps
API
Kernel
App
Docker
30. Cluster configuration example #4
Docker
AWS, DO, GCE
Mesos
Spark, MPI,
Hadoop, Storm
Mesos SDK
App
Java, Python, C++, Go
Serv Serv
Services REST API
“Marathon” (init)
Batch REST API
“Chronos” (cron)
Serv
App Recurring Jobs
(ETL, backups)
Hardware
Native Long running Batch
Apps
API
Kernel
App
Docker
Serv