MesosCon 2016: Frameworks in Harmony

© 2016 Mesosphere, Inc. All Rights Reserved. 1
FRAMEWORKS IN HARMONY
MesosCon 2016

BRENDEN MATTHEWS
Engineer
Core Mesos committer, contributor to numerous Mesos
frameworks, and various other projects in the Mesos
ecosystem. He spends his free time contemplating
existentialism, cooking steak, and craigslist furniture hacking.
@brndnmtthws
VISHNU MOHAN
Solutions Engineer
Works closely with customers in the field on Performance and
Automation. When he’s not analyzing sandbox logs or coding
against some API for fun, he may be found rock climbing,
hiking, running or scuba diving.
@vm_mesos

AGENDA
MesosCon 2016:
Frameworks in Harmony

● Problem Definition, Motivations and Applications
● Intro to Roles, Weights, Quotas, Reservations, Revocation & Oversubscription
● Theory
● Dominant Resource Fairness (DRF)
● Hierarchical Dominant Resource Fairness (H-DRF)
● Mesos Quotas
● Scenarios and putting it all together
● Demo on
AGENDA
MesosCon 2016: Frameworks in Harmony

MOTIVATIONS/
APPLICATIONS
MesosCon 2016:

MOTIVATIONS
Running multiple environments on one
cluster
Guaranteeing resource availability
APPLICATIONS
Multitenancy
Isolation
Introducing new frameworks
MOTIVATIONS/APPLICATIONS

DEFINITIONS
MesosCon 2016:

ROLES
A method of identifying frameworks for use with
reservations, auth, ACLs, and other Mesos features.
WEIGHTS
Control the relative share of resources that is
offered to different roles
INVERSE OFFERS
Mechanism to reclaim resources for maintenance,
reallocation and pre-emption
DEFINITIONS
RESERVATIONS
Mechanism to reserve resources on specific agents
(Static and Dynamic)
QUOTAS
Lay away resources for consumption via cluster-
wide dynamic reservation
OVERSUBSCRIPTION
Temporarily take advantage of unused resources

DOMINANT
RESOURCE
FAIRNESS
(DRF)
MesosCon 2016:

DRF
Core resource allocation algorithm for
Mesos
Generalization of Max-Min Fairness to
multiple resources
WHY MAX-MIN FAIRNESS?
Generality
Ability to provide performance isolation
https://www.cs.berkeley.edu/~alig/papers/drf.pdf
WEIGHTED MAX-MIN FAIRNESS
Priority
Reservation
Deadline-Based Allocation
User is guaranteed to receive her fair
share irrespective of the demands of
others
Applicable to a wide variety of resources
DOMINANT RESOURCE FAIRNESS

DRF CORE CONCEPTS
In a multi-resource environment, the
allocation of a user should be
determined by the user’s “dominant
share”, which is the maximum share that
the user has been allocated of any
resource
DRF then seeks to maximize the
minimum dominant share across all
users
DRF CORE PROPERTIES
Sharing Incentive
Strategy Proofness
Pareto Efficiency
Envy Freeness
NICE-TO-HAVE PROPERTIES
Single Resource Fairness
Bottleneck Fairness
Population Monotonicity
Resource Monotonicity
DOMINANT RESOURCE FAIRNESS contd.

EXAMPLE
Total Capacity: 9 CPUs, 18 GB RAM
ROLE A
<1 CPU, 4 GB>, Dominant Resource: Mem
<x CPU, 4x GB>, Dominant Share: 2/9 (i.e., 4/18)
ROLE B
<3 CPUs, 1 GB>, Dominant Resource: CPU
<3y CPU, y GB>, Dominant Share: 1/3 (i.e., 3/9)
DRF IN ACTION
MesosCon 2016: Frameworks In Harmony
SOLUTION
x = 3, y = 2

DRF IN ACTION contd.

DRF ALLOCATION SCHEDULE (Quantum 1)
Schedule
Role A <1 CPU, 4 GB> Role B <3 CPU, 1 GB>
CPU
Total Alloc.
RAM
Total Alloc.
res. shares dom. share res. shares dom. share
Role B <0,0> 0 <3/9, 1/18> 1/3 3/9 1/18

Schedule
CPU
Total Alloc.
RAM
Total Alloc.
Role B <0,0> 0 <3/9, 1/18> 1/3 3/9 1/18
Role A <1/9, 4/18> 2/9 <3/9, 1/18> 1/3 4/9 5/18

Schedule
CPU
Total Alloc.
RAM
Total Alloc.
Role B <0,0> 0 <3/9, 1/18> 1/3 3/9 1/18
Role A <1/9, 4/18> 2/9 <3/9, 1/18> 1/3 4/9 5/18
Role A <2/9, 8/18> 4/9 <3/9, 1/18> 1/3 5/9 9/18

Schedule
CPU
Total Alloc.
RAM
Total Alloc.
Role B <0,0> 0 <3/9, 1/18> 1/3 3/9 1/18
Role A <1/9, 4/18> 2/9 <3/9, 1/18> 1/3 4/9 5/18
Role A <2/9, 8/18> 4/9 <3/9, 1/18> 1/3 5/9 9/18
Role B <2/9, 8/18> 4/9 <6/9, 2/18> 2/3 8/9 10/18

DRF ALLOCATION SCHEDULE (Quantum 5) - CPU Maxed Out
Schedule
CPU
Total Alloc.
RAM
Total Alloc.
Role B <0,0> 0 <3/9, 1/18> 1/3 3/9 1/18
Role A <1/9, 4/18> 2/9 <3/9, 1/18> 1/3 4/9 5/18
Role A <2/9, 8/18> 4/9 <3/9, 1/18> 1/3 5/9 9/18
Role B <2/9, 8/18> 4/9 <6/9, 2/18> 2/3 8/9 10/18
Role A <3/9, 12/18> 2/3 <6/9, 2/18> 2/3 1 14/18

DRF ALLOCATION AND RESOURCE CONSUMPTION

HIERARCHICAL
DOMINANT
RESOURCE
FAIRNESS
(H-DRF)
https://www.cs.berkeley.edu/~alig/papers/h-drf.pdf
MesosCon 2016:

MESOS
ROLES
MesosCon 2016:

USE CASES
● Ensuring that resources on a particular agent are only offered to a particular
framework
● Partitioning a cluster between organizations: resources reserved for use by
Organization A will only be offered to frameworks that have registered using
Organization A’s role (also, Reservations)
● Expressing that one group of frameworks has “higher priority” (and offered more
resources) than another group of frameworks (also, Weights)
● Setting a guaranteed resource allocation for one or more frameworks belonging
to a role (also, Quotas)
● Ensuring that persistent volumes created by one framework are not offered to
frameworks registered with a different role
MESOS ROLES

MESOS
QUOTAS
MesosCon 2016:

QUOTA
Quotas are a mechanism for dynamically
reserving resources.
WHY QUOTA?
DRF may offer resources to frameworks
beyond their fair share. Operators can
use quotas to ensure that frameworks
always receive enough resources.
MESOS QUOTAS
QUOTA CORE CONCEPTS
Quotas provide a mechanism for
guaranteeing resources for roles.
Frameworks may register with that role,
and will be allocated at least the
specified amount of resources for that
role.
In other words, quotas are an
implementation of cluster-wide dynamic
resource reservations by role.

© 2016 Mesosphere, Inc. All Rights Reserved.
LIMITATIONS
Can’t specify granularity or constraints
(e.g., can’t distribute resources evenly
across fault domains).
Cannot update an existing quota (must
be removed and re-added).
The current implementation is both the
minimum and the limit. No more offers
will be sent once all reserved resources
are allocated for that role.
25
QUOTA FEATURES
Minimum Guarantee
Dynamically Allocated
Configured at Runtime
Applies Per Role
Cluster-wide
Easy Reasoning
Complementary to Dynamic Reservations
MESOS QUOTAS contd.

QUOTA USAGE
core@ip-10-0-6-151 ~ $ curl -d '{"role":"myRole","guarantee":[{"name":"cpus","
type":"SCALAR","scalar":{"value":1}},{"name":"mem","type":"SCALAR","scalar":
{"value":100}}]}' -X POST http://master.mesos:5050/quota
core@ip-10-0-6-151 ~ $ curl http://master.mesos:5050/quota
{"infos":[{"guarantee":[{"name":"cpus","role":"*","scalar":{"value":1.0},"
type":"SCALAR"},{"name":"mem","role":"*","scalar":{"value":100.0},"type":"
SCALAR"}],"role":"myRole"}]}
core@ip-10-0-6-151 ~ $ curl -X DELETE http://master.mesos:5050/quota/myRole
core@ip-10-0-6-151 ~ $ curl http://master.mesos:5050/quota
{}

DEMO!
MesosCon 2016:

CONCLUSIONS
& FUTURE
WORK
MesosCon 2016:

Q&A
MesosCon 2016:
Thursday, 11:00am: Quota: Reliable
Guarantees - Jörg Schad & Joris Van
Remoortere, Mesosphere - Grand Mesa A,
http://sched.co/6jtn
Thursday, 12:00pm: Maintenance:
Automating Collaboration Between
Frameworks and Operators - Joseph Wu,
Mesosphere - Chasm Creek (this room),
http://sched.co/6jtt
Thursday, 2:00pm: Resource Allocation in
Apache Mesos: Present and Future -
Benjamin Mahler, Mesosphere - Grand
Mesa A, http://sched.co/6jtm

VISIT OUR BOOTH
Located at D1, near the main entrance
Learn more by visiting dcos.io and mesosphere.com
Vishnu’s Office Hours
Thursday 10-12:50pm

Charlie

References
MesosCon 2016:
DRF: https://www.cs.berkeley.edu/~alig/papers/drf.pdf
H-DRF: http://www.cs.berkeley.edu/~alig/papers/h-drf.pdf
https://www.usenix.org/conference/nsdi11/dominant-resource-fairness-fair-
allocation-multiple-resource-types
Mesos Docs:
https://www.cs.berkeley.edu/~alig/papers/mesos.pdf
https://mesos.apache.org/documentation/latest/roles/
https://mesos.apache.org/documentation/latest/weights/
https://mesos.apache.org/documentation/latest/attributes-resources/
https://mesos.apache.org/documentation/latest/reservation/
https://mesos.apache.org/documentation/latest/quota/
https://mesos.apache.org/documentation/latest/oversubscription/
n-level Hierarchies: https://issues.apache.org/jira/browse/MESOS-1763

MesosCon 2016: Frameworks in Harmony

Recommended

Recommended

More Related Content

Similar to MesosCon 2016: Frameworks in Harmony

Similar to MesosCon 2016: Frameworks in Harmony (20)

Recently uploaded

Recently uploaded (20)

MesosCon 2016: Frameworks in Harmony