The document compares the cluster management systems Mesos and Kubernetes. Mesos is an Apache project that provides efficient resource sharing across distributed applications. Kubernetes, developed by Google, provides cluster management capabilities for virtual machines and containers. Both support Docker containers and scheduling of workloads but Mesos gives more control while Kubernetes has easier setup and built-in scheduling. The document also discusses their architectures, major customers, and the future of these systems.

1. HUAWEI TECHNOLOGIES CO., LTD.
www.huawei.com
Mesos vs. Kubernetes
… a quick look …
Krishna M Kumar
Lead Architect
Bangalore Cloud Foundry Meetup – 28th Mar 2015

2. HUAWEI TECHNOLOGIES CO., LTD. 2
contents - mesos vs. kubernetes – the TOP TEN
1. We are comparing – The what part!
10. Where to look further – The References
5. The bits and pieces – The Components
3. The guys who plays together – The Partners/Customers
4. Unknown internals – The Architecture
7. The comparative landscape – The Competitors
2. What you get it – The Features
9. So Finally What – The Conclusion
6. Who does the real job – The Scheduler
8. What the world is going to do – The Future

3. HUAWEI TECHNOLOGIES CO., LTD. 3
Mesos Kubernetes
 Apache Mesos is a cluster manager that provides efficient
resource isolation and sharing across distributed
applications, or frameworks.
 Apache Mesos abstracts CPU, memory, storage, and other
compute resources away from machines (physical or virtual),
enabling fault-tolerant and elastic distributed systems.
 Google Kubernetes brings 'Google style' cluster
management capabilities to the world of virtual machines, or
'on the metal' scenarios.
 For production Google uses Omega/Borg, not Kubernetes
which is in beta.
• Mesos using same principle as Linux Kernel but higher layer
of abstraction.
• It can run Hadoop, MPI, Hypertable, Spark, and other
frameworks on a dynamically shared pool of nodes.
• Manage a cluster of Linux containers as a single system.
• Kubernetes works with Mesos as Framework like Marathon
Frameworks.
 Was developed at the University of California, Berkeley year
2009 developed by Benjamin Hindman, Andy Konwinski, Matel
Zaharia as well as professor Ion Stoica
 Become top level Apache project from July 2013
 Project Started at Google
 It is still in Beta stage
Works with various Frameworks like Marathon, Cronos. You can
write your own. There is a core algorithm(DRF) to share
resources and you can define your own scheduling policy.
Frameworks does Fault Tolerance, Load balancing, Quotas,
Discovery, etc.
Kubernetes establishes robust declarative primitives for
maintaining the desired state requested by the user. Self-healing
mechanisms, such as auto-restarting, re-scheduling, and
replicating containers require active controllers, not just
imperative orchestration.
1. We are comparing – The what part!

4. HUAWEI TECHNOLOGIES CO., LTD. 4
Mesos Kubernetes
Written in C++, Create a big resource pool from all
resources.
Written in Golang, works well with CoreOS, Red Hat
Atomic. Easy to run cluster resources.
Full control of the containers scheduled. Custom schedule
can manage the containers and VMs as desired.
Kubernetes will do the containers. Pods – a group of
containers scheduled on the same host supports
composable micro services.
Since Mesos enables you to run stateless services such
as web servers or application servers on the same cluster
as Spark for analytics purposes, it increases the overall
cluster utilization and accommodates the effects of data
gravity.
You can deploy any service you like into Kubernetes.
There are also some services with built-in Kubernetes
support. Cassandra, RethinkDB and Spark are some of
them. This makes better utilization, reduce complexity and
service discovery much easier.
Project Features
• Scalability to 10,000s of nodes
• Fault-tolerant replicated master and slaves using ZooKeeper
• Support for Docker containers
• Native isolation between tasks with Linux Containers
• Multi-resource scheduling (memory, CPU, disk, and ports)
• Java, Python and C++ APIs for developing new parallel applications
• Web UI for viewing cluster state
Project Features
• Primarily targeted at applications composed of multiple containers,
• such as elastic, distributed micro-services.
• Support multiple platforms
• Easy to get up and running & portable
• Manages Docker deployment well
• Scheduling provides out of the box
• Monitoring Dashboard
2. What you get it – The Features

5. HUAWEI TECHNOLOGIES CO., LTD. 5
Mesos Kubernetes
4. The guys who plays together – The Partners/Customers
Organizations using Mesos:
http://mesos.apache.org/documentation/latest/powered-by-mesos/
Twitter
Airbnb
Apple
Cisco
Ericson
Ebay
Mesosphere
Netflix
Google
Microsoft
IBM
RedHat
Docker
Mesosphere
SaltStack
CoreOS
Tectonic
Mirantis
Openstack
VMWare

6. HUAWEI TECHNOLOGIES CO., LTD. 6
Mesos Kubernetes
5. Unknown internals – The Architecture
See Components in next slide for more details.

7. HUAWEI TECHNOLOGIES CO., LTD. 7
Mesos Kubernetes
1. Master – Enable sharing of resource
2. Slave – Execute the task in it
3. Cluster – a group of machines
4. ZooKeeper – distributed synchroniztion/configuration
5. Framework – scheduler + executor
6. Scheduler – Offer resources
7. Executor – Run the Framework task
8. Task – a job to run
9. Containerizer – run & monitor executors
1. Master – Cluster controlling unit
2. etcd – HA Key/value store
3. API Server - Observing the state of the cluster
4. Controller Manager – runs multiple controllers
5. Scheduler Server – assigns workloads to nodes
6. Minion – server that perform work
7. Kubelet - server/slave node that runs pods
8. Proxy Service – host subnetting to external parties
9. Pods – One or more containers
10.Services – load balancer for containers
11.Replication Controller – For horizontally-scaled pods
12.Labels – A tag for work unit for grouping
Related:
1. CoreOS – Chrome OS based Linux distro
2. Fleet – Orchestrate CoreOS Clusters
3. Borg – massive scale resource manager
4. Omega – massive scale cluster manager
 More detailed explanations of the components: https://github.com/mesosphere/kubernetes-mesos/issues/285
6. The bits and pieces – The Components

8. HUAWEI TECHNOLOGIES CO., LTD. 8
Mesos Kubernetes
7. Who does the real job – The Scheduler
The scheduler is configurable and at compile time which fit predicates
and priority functions you want Kubernetes to apply.
It has two types of policies, FitPredicate - The sum of the requested
resources of the container(s) already running on the machine plus the
requested resources of the new container(s) you are considering
scheduling onto the machine must not be greater than the capacity of
the machine& PriorityFunction - The scheduler may find that multiple
machines "fit" the pod and it prefers the machine whose already-
running pod consume the least resources.
Future Goals -- Scheduling is a policy-rich, topology-aware,
workload-specific function that significantly impacts availability,
performance, and capacity. The scheduler needs to take into account
individual and collective resource requirements, quality of service
requirements, hardware/software/policy constraints, affinity and anti-
affinity specifications, data locality, inter-workload interference,
deadlines, and so on. Workload-specific requirements will be exposed
through the API as necessary.
Mesos has two levels of scheduling; first one intra-Frameworks level
as in below and second at inter-Framework level application specific.

9. HUAWEI TECHNOLOGIES CO., LTD. 9
8. The comparative landscape – The Competitors

10. HUAWEI TECHNOLOGIES CO., LTD. 10
Mesos Kubernetes
 Silicon Valley startup Mesosphere is
betting big on Mesos to create its future
Data Center Operating System.
 Several top companies are using Mesos
in their production environment.
 Google’s Flagship Data Center product
for cloud hosting with lots of industry
backing.
 Mesosphere have integrated Mesos and
Kubernetes together.
Focusing more on Enterprise adoption like Security,
Encryption, Reliability and Authentication
by Ben Hindman – Mesos Creator.
Better scheduling algorithm & Spanning across
multiple location is challenge.
By Eric Brewer, VP Google & CAP Theorem Creator.
Customers can choose utilization in datacenters,
security and reliability at scale – can be brought to
container- and VM-based environments using
Mesos. Huge flexibility of integrating any kind of
framework and so continue to grow popularly.
Kubernetes to be awesome everywhere, even for
customers who will run their apps on other clouds or
in their own datacenters - goal for Kubernetes is
ubiquity. AS more and more Docker deployment
happens and Kubernetes gets more maturity, it may
act alone without Mesos to deliver similar value what
Mesos provides!
9. What the world is going to do – The Future

11. HUAWEI TECHNOLOGIES CO., LTD. 11
10. So Finally What – The Conclusion
Together Kubernetes and Mesos are a match made in heaven.
Kubernetes enables the Pod (group of co-located containers) abstraction, along with Pod labels for service discovery, load-
balancing, and replication control.
Mesos provides the fine-grained resource allocations for pods across nodes in a cluster, and can make Kubernetes play nicely
with other frameworks running on the same cluster resources
- Stackoverflow
Use Mesos for:
Heavy Duty Production
Implement a new capability
Need multiple Framework
For existing workloads
Full control of workloads
Use Kubernetes for:
Testing alpha/beta
Very easy to setup and start
Pre-built scheduler algorithm
Container only world
More features on the way

12. HUAWEI TECHNOLOGIES CO., LTD. 12
10. Where to look further – The References
1. Mesos - https://github.com/apache/mesos/
2. Kubernetes - https://github.com/googlecloudplatform/kubernetes
3. Mesos Architecture - http://mesos.apache.org/documentation/latest/mesos-architecture/
4. Kubernetes Design - https://github.com/GoogleCloudPlatform/kubernetes/tree/master/docs/design
5. Kubernetes Scheduler - http://stackoverflow.com/questions/28857993/how-does-kubernetes-scheduler-work
6. Mesos Scheduler - http://cloudarchitectmusings.com/2015/04/08/playing-traffic-cop-resource-allocation-in-apache-
mesos/
7. Kubernetes Deployment - https://news.ycombinator.com/item?id=9653118
8. Mesosphere – Kubernetes-Mesos - https://github.com/mesosphere/kubernetes-mesos
9. Mesos and Kubernetes at scale - https://www.youtube.com/watch?v=pFCJ3WiVjyA
10. Mesos vs. Kubernetes - http://stackoverflow.com/questions/26705201/whats-the-difference-between-apaches-mesos-
and-googles-kubernetes
11. Kubernetes – Marathon - https://www.quora.com/What-is-the-difference-between-Googles-Kubernetes-and-
Mesospheres-Marathon
12. Mesos - Kubernetes PaaS - http://www.slideshare.net/MiguelZuniga1/platform-as-a-service-with-kubernetes-and-mesos
13. Mesos, Docker, Kubernetes - http://www.slideshare.net/timothysc/apache-coneu
14. Future of computing - https://medium.com/s-c-a-l-e/google-systems-guru-explains-why-containers-are-the-future-of-
computing-87922af2cf95
15. Future of Resource Manager by IBM - https://www.ibm.com/developerworks/community/blogs/1ba56fe3-efad-432f-
a1ab-58ba3910b073/entry/thoughts_on_future_of_resource_managers_and_schedulers_in_the_cloud?lang=en

13. Thank you
www.huawei.com
HUAWEI TECHNOLOGIES CO., LTD. 13