Centralized Scheduling made Simple
Scott Weiss

Once upon a time, there was a
meeting about task scheduling…

Mesos
• Popular, mature cluster manager
• Widely used at hyperscale

Mesos Architecture

Mesos Architecture
Applications do their
own scheduling!

Mesos Architecture
Your
application
Frameworks
Mesos Master
(not a scheduler)
Mesos Slaves

How It Works
“Hey Jenkins, Do you
want to run something
on this node?”
“Sure, that node looks
good for this task”
“OK, let me run
that for you.”

Distributed Scheduling Model
• Pros:
– Applications have more control
• Cons:
– Schedulers have only a localized view of the
cluster

Distributed Scheduling Model
• Pros:
– Applications have more control
• Cons:
– Schedulers have only a localized view of the
cluster
• What if scheduling decisions need to be
made with a global view of the cluster?
Fenzo

Problems
“Hey Jenkins, Do you
want to run something
on this node?”
“I don’t like this node”
“It’s up to you to
wait for a better
one”
Schedulers only see the nodes Mesos shows them

Problems
“Hey Jenkins, Do you
want to run something
on this node?”
“I want on the same node
as Marathon”
Schedulers only know where their own tasks are running

Problems
“Hey Jenkins, Do you
want to run something
on this node?”
“My tasks are more
important than Jenkins’, I
need that node more”
Schedulers cannot overrule each other
“Then you should have
reserved it ahead of
time”

Problems
“Hey Jenkins, Do you
want to run something
on these nodes?”
Continuous placement is not possible
“Only if it belongs to
you.”
“Those nodes are too
small for me. Can I cancel
one of the tasks and move
it to another node?”

Adding Centralized Scheduling to Mesos
Your
application

Adding Centralized Scheduling to Mesos
Your
application
Layer of Abstraction

Layer-X Was born
Your
application

How it works
• Layer-X represents itself to Mesos Applications (called
“Frameworks”) as an actual Mesos cluster
• Mesos Frameworks register to Layer-X the same way they
register to Mesos itself
marathon/bin/start --master=LAYERX_URL

How it works
• Layer-X registers to Mesos like a normal
framework
config := scheduler.DriverConfig{
Scheduler: layerxScheduler,
Framework: layerxFramework,
Master: *master,
Credential: (*mesosproto.Credential)(nil),
}
driver, _ := scheduler.NewMesosSchedulerDriver(config)
status, _ := driver.Run()

How it works
• Layer-X collects tasks from frameworks with
fictitious resource offers
• Layer-X then collects resources from Mesos
and sits on them

How it works
• By collecting all tasks and all resources,
Layer-X has a global view of the cluster
• Layer-X can then be controlled through a REST
API by an external application to make better
scheduling decisions

How it works
• External Schedulers read the state of the
cluster through Layer-X API
• Using this information, External Schedulers
make scheduling decisions with Layer-X API

What’s possible with Layer-X?
Your
application

What’s possible with Layer-X?
Your
application
Plug in any scheduler
Fenzo

What’s possible with Layer-X?
Your
application
Plug in any scheduler
Fenzo
Intercept, Resize Tasks

What’s possible with Layer-X?
Your
application
Plug in any scheduler
Fenzo
Intercept, Resize Tasks
Live migration,
continuous placement

What’s possible with Layer-X?
Your
application
Plug in any scheduler
Fenzo
Intercept, Resize Tasks
Live migration,
continuous placementMulti-cluser management
us-east-1us-west-1

Demo!

What else is possible with Layer-X?
Your
application

What else is possible with Layer-X?
Your
application
We all speak
Docker!

What else is possible with Layer-X?
Your
application

Next Steps for Layer-X
• Open Source
• Plugins for more cluster managers
– Kubernetes
– Docker Swarm

Follow us on Twitter & Github
• For news about the release of Layer-X and other
interesting OSS projects at EMC, follow
• Our Github page:
https://github.com/emc-advanced-dev
• Twitter
@ilackarms
@Idit_Levine
• And please check out my Unikernels workshop,
tomorrow at 1:00 PM
• Questions?