Mesos is an open source cluster management framework that provides efficient resource isolation and sharing across distributed applications or frameworks. It divides resources into CPU, memory, storage, and other compute resources and shares those resources dynamically and efficiently across applications. Mesos abstracts the underlying infrastructure to provide a unified API to applications while employing operating system-level virtualization through interfaces like Docker to maximize resource utilization. It works by having a Mesos master that negotiates resources among Mesos slaves to run applications or frameworks, which are made up of a scheduler to negotiate for resources and executors to run tasks. Common frameworks that run on Mesos include Spark, Hadoop and Docker containers.

1. Cluster Management System
mesos.apache.org

2. • to understand target problem domain.
• to know the operating system for the Datacenter.
• to gain an architectural overview of a Mesos cluster.
• to learn how to run applications in a Mesos cluster.
• to understand the Service Discovery & Load Balancing
• to get an overview on Mesos Frameworks.
http://mesos.apache.org Agenda
Your Expectations

3. Mesos is the name of the cluster management
framework and the Apache Project.
Mesosphere (Mesosphere, Inc.) is the
company which is commercializing the open
source project Apache Mesos, and provides
consulting services.
http://mesos.apache.org Mesos
Mesos vs Mesosphere

4. http://mesos.apache.org Problem Domain
Static Partitioning

5. http://mesos.apache.org Problem Domain
Static partitioning Dynamic resource sharing
Resource Management

6. http://mesos.apache.org Problem Domain
Unified API

7. • Realignment of the partitions in the clusters means un-
deploy/redeploy of the running software on the
instances and requires error-prone and tedious manual
effort.
• It is hard to predict the workload and fluctuations of the
load makes an efficient resource utilisation impossible.
• Changes in infrastructure may cause changes in the
applications so the coupling between Developers and
Devops does remain strong.
http://mesos.apache.org Problem Domain
Problem Description

8. Datacenter as a Computer

9. • Process scheduling.
• Fairly resource (on the same
CPU, Memory, Disk, etc) sharing.
• Package managers (aptitude,
yum, etc.)
• Unified API between apps and
hardware.
http://mesos.apache.org Datacenter OS
DC as a Computer
• Cluster-wide application
scheduling.
• Fairly resource sharing in a
distributed environment.
• Docker as package manager tool.
• Unified API between apps and
hardware.
Operating Systems Mesos as a DC OS

10. • orchestrates the infrastructure so that it can be
extended by adding new hardware or shrunk to
downscale, dynamically.
• abstracts the underlying infrastructure while
providing a unified API to the applications.
• employs the LXC (Linux Containers) interface to
maximize resource utilisation via operating system
level virtualization (like Docker or using Docker).
http://mesos.apache.org Problem Domain
Apache Mesos

11. http://mesos.apache.org Architecture
Architecture
Coordination over
Zookeeper
Mesos CLI
Ports are ranged
resources

12. http://mesos.apache.org Datacenter OS
How does it work ?
Mesos Framework = Scheduler + Executor

13. http://mesos.apache.org Datacenter OS
How does it work ?
Resource negotiation between components

14. • You can develop your own custom Frameworks for
Mesos.
• or you can deploy open source Mesos Frameworks
like Spark, Hadoop, etc.
• or, getting even better, you can deploy Docker
containers.
• Mesos can orchestrate Docker containers in a
Mesos cluster, as well.
http://mesos.apache.org Frameworks
Applications on Mesos

15. http://mesos.apache.org Frameworks
Mesos Frameworks
OSS Frameworks as Mesos Frameworks

16. http://mesos.apache.org Architecture
Marathon

17. http://mesos.apache.org Marathon
Marathon
• Marathon is the PaaS on
Apache Mesos.
• Marathon provides REST
API to automate your
workflows.
• Marathon UI allows Dev/
Ops to manage the
frameworks by point-n-click.

18. http://mesos.apache.org Architecture
Load Balancing thru HAProxy
User Requests

19. http://mesos.apache.org Architecture
Service Discovery
User Requests
Mesos DNS provides its
clients a
REST endpoint for
Service Discovery

20. http://mesos.apache.org Hands-on
Hands-on
• Introduction: Mesos Dashboard
• Introduction: Marathon UI and REST API.
• Case Study: Elastic Jenkins on Mesos.

21. http://mesos.apache.org End
QA
any Questions?