https://www-01.ibm.com/marketing/iwm/iwmdocs/web/cc/earlyprograms/zsoftware/mesos/index.shtml https://youtu.be/3z7DUojwwuk

1. © 2012 IBM Corporation
IBM Systems & Technology Group
IBM Confidential
Mesos EGO Demo Cases

2. © 2014 IBM Corporation2
Agenda
 Architecture
 Environments
 Management Console
 Resource Sharing
 QoS

3. © 2014 IBM Corporation3
Architecture
VEMKD
Mesos/EGO Agent Mesos/EGO Agent Mesos/EGO Agent
Spark
LSF Symphony
Request with 2 containers
(2 cpu, 2G mem)
Supply: Resource Group Hierarchy
Rack
Group
Rack
Group
Rack
Group
DC
/
Rack Rack Rack Rack Rack Rack
DC
Demand: Application Topology
DB
App
Server
App
Server
App
Server
Web
Server
Web
Server
root
grp1 grp2 gp3
usr1 usr2 user3 user4
•Sharing
•Ownership
•Lending/borrow
Sharing Plan: Application Topology
app1 app2 app3 app4 app5
Mesos Master EGO
Allocator
Marathon Kubernetes
Offer with host1
(4 cpu, 64G mem)
Enterprise
Resource
Scheduler
Chorons
LSF
Symphony
ASC

4. © 2014 IBM Corporation4
Environments
Mesos/EGO Agent
(4 CPU, 14.7 G Memory)
Mesos Master Host
EGO Master
Platform Application Service Controller
(Meta Framework)
Marathon Kubernetes Spark
Mesos/EGO Agent
(4 CPU, 14.7 G Memory)
Mesos/EGO Agent
(4 CPU, 14.7 G Memory)
Mesos/EGO Agent
(4 CPU, 14.7 G Memory)

5. © 2014 IBM Corporation5
Value to Mesos User
• 100% API Compatibility: No change required in Mesos Framework and
Executor
• Simplifies Management: By providing a central management console to
dynamically manage tenant and configure resource sharing policy
• Provides Advance Resource Sharing Policies: To Apache Mesos
frameworks, these include multiple-dimensional based lending-borrow policy,
ownership policy, priority policy and time-window policy
• Provides guaranteed QoS: To Apache Mesos frameworks by enforcing task
pre-emption
• Allows Apache Mesos Frameworks: To better optimize tasks scheduling by
providing an enhanced view of resource status in the system
• Improves Apache Mesos frameworks Performance: By providing
frameworks with the ability to permanently cache resource offering
• IBM Platform Applications: Share resources with IBM Platform Applications
such as Symphony, Application Service Controller, Map Reduce, Spark
Connector
·.

6. © 2014 IBM Corporation6
Resource Plan Configuration (Ownership)
 All roles (consumers) under Root consumer
 Marathon using role (consumer) marathon1
 Kubernetes using role (consumer) k8s1
 Spark using role (consumer) spark
 Resource Plan Configuration
o Marathon(marathon1) own 2 hosts, reserve 1 host, share ratio 1, priority 2
o Kubernetes(k8s1) own 0 host, reserve 0 host, share ratio 1, priority 5
o Spark(spark) own 0 host, reserve 0 host, share ratio 1, priority 5
Root
Marathon Kubernetes
Total = 16 CPU, 58.8 G Memory
Spark

7. © 2014 IBM Corporation7
Management Console
• 1. Resource Information

8. © 2014 IBM Corporation8
Management Console
• 1. Create Resource Group

9. © 2014 IBM Corporation9
Management Console
• 1. Create Spark, Marathon, Kubernetes consumers on ASC portal

10. © 2014 IBM Corporation10
Management Console
• 1. Configure resource plan(Share policy configure)

11. © 2014 IBM Corporation11
Management Console
• 1. Configure resource plan(Owner ship policy)

12. © 2014 IBM Corporation12
Resource Sharing and QoS
 Demo Steps
o Marathon launch task to occupy 1 host.
o Kubernetes launch task occupy another 2 hosts
 Borrow 1 from Marathon
 Get 1 from share pool
o ASC launch a Spark to reclaim 1 host back from Kubernetes
o Marathon launch as many as tasks
o All resources will be reclaimed back to Marathon as Marathon has
high priority

13. © 2014 IBM Corporation13
Resource Sharing and QoS
Launch a WebSphere service on Marathon:

14. © 2014 IBM Corporation14
Resource Sharing and QoS
We can see, the websphere service was running

15. © 2014 IBM Corporation15
Resource Sharing and QoS
Then create the 2 k8s tasks:
And the 2 tasks were on the different hosts

16. © 2014 IBM Corporation16
Resource Sharing and QoS
Then, Create an Spark service on ASC:

17. © 2014 IBM Corporation17
Resource Sharing and QoS
Start the Spark service

18. © 2014 IBM Corporation18
Resource Sharing and QoS
From the URL below, we can logon Spark portal to view your Spark cluster, and
also you can use this environment to submit your jobs as well now.

19. © 2014 IBM Corporation19
Resource Sharing and QoS
Then, let’s take a look at current resource usage of the cluster:

20. © 2014 IBM Corporation20
Resource Sharing and QoS
From the page, we can see:
1) Spark allocate 4 resources
2) Kubenetes allocate 2 resources
3) Marathon allocate 1 resources and lend out 2 resources to k8s and spark
In order to make sure QoS, When Marathon request more resources, it can ask
resource back from Spark and kubenetes.

21. © 2014 IBM Corporation21
Resource Sharing and QoS
Then I ask more websphere service on Marathon and scale up to 12 instances

22. © 2014 IBM Corporation22
Resource Sharing and QoS
We can see the WebSphere services were all running

23. © 2014 IBM Corporation23
Resource Sharing and QoS
And from the resource allocate page, we can see marathon have ask all lend out
resources back

24. © 2012 IBM Corporation
IBM Systems & Technology Group
IBM Confidential