Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Execution of deadline-
constrained workloads using
jQueuer and MiCADO
Amjad Ullah
University of Westminster
September 26th...
Webinar on MiCADO [26/09/2019]
Agenda
• Introduction
1. Jozsef Kovacs (SZTAKI): What is MiCADO (good for)? [15 mins]
2. Ja...
Multi-job experiments
www.project-cola.eu 3
• Large number of jobs results in significant overall execution time
• Restric...
JQueuer
www.project-cola.eu 4
• Asynchronous Distributed Task Queue
• Master/Server Component
• Runs externally
• Queue & ...
Demo: Agent based simulation framework
September 26th 2019 www.project-cola.eu 5
• Experiment: Determining the impact of c...
Execution using jQueuer + MiCADO
September 26th 2019 www.project-cola.eu 6
experiment.json JQueuer
Manager
MiCADO Worker 1...
jQueuer deployment using ADT
September 26th 2019 www.project-cola.eu 7
Deadline-based scaling policy
September 26th 2019 www.project-cola.eu 8
Results
September 26th 2019 www.project-cola.eu 9
• Dynamic allocation of
variable length jobs
• Results in better use of
...
Thanks and Questions!
September 26th 2019 www.project-cola.eu 10
• github.com/micado-scale
• micado-scale.eu/
This project...
You’ve finished this document.
Download and read it offline.
Upcoming SlideShare
What to Upload to SlideShare
Next
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

Share

Deadline-based scaling - JQueuer Demo - MiCADO webinar No.4/4 - 09/2019

Download to read offline

4/4 Webinar: How to Automate Deployment and Orchestration of Application (MiCADO introduction)

This part of the webinar covers the software JQueuer, with JQueuer the computing a large number of jobs results in significant overall execution time. It was presented by Amjad Ullah
(University of Westminster). The webinar took place on the 26th of September 2019. If you would like to have more information visit: https://micado-scale.eu

MiCADO is open-source and a highly customisable multi-cloud orchestration and auto-scaling framework for Docker containers, orchestrated by Kubernetes.

Developed by Project COLA funded by the European Commission (grant agreement no: 731574). https://project-cola.eu

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

Deadline-based scaling - JQueuer Demo - MiCADO webinar No.4/4 - 09/2019

  1. 1. Execution of deadline- constrained workloads using jQueuer and MiCADO Amjad Ullah University of Westminster September 26th 2019
  2. 2. Webinar on MiCADO [26/09/2019] Agenda • Introduction 1. Jozsef Kovacs (SZTAKI): What is MiCADO (good for)? [15 mins] 2. James Deslauriers (WMIN): How to develop an application in MiCADO? [15 mins] • Questions [5 mins] • Use cases 3. Matt Kendon (Outlandish): Wordpress [10 mins] 4. Amjad Ullah (WMIN): JQueuer [10 mins] • Questions [5 mins]
  3. 3. Multi-job experiments www.project-cola.eu 3 • Large number of jobs results in significant overall execution time • Restricted to complete all jobs by a deadline • Where to put the jobs? • How to distribute? • How to execute (in containers) • How to liaise with deadline? MICADO MASTER ADT: infrastructure and scaling rules End user MiCADO Submitter Policy Keeper (Scaling logic) cqueue worker MICADO WORKER cqueue workerJobs Scale up/ down jQUEUER MASTER jQueuer Agent Jobs experiment .json Container and Cloud Orchestrator MICADO Queuing system jobs September 26th 2019
  4. 4. JQueuer www.project-cola.eu 4 • Asynchronous Distributed Task Queue • Master/Server Component • Runs externally • Queue & monitoring • Agent Component • Runs on worker VMs • Fetch & execute jobs MICADO MASTER ADT: infrastructure and scaling rules End user MiCADO Submitter Policy Keeper (Scaling logic) cqueue worker MICADO WORKER cqueue workerJobs Scale up/ down jQUEUER MASTER jQueuer Agent Jobs experiment .json Container and Cloud Orchestrator MICADO MICADO MASTER ADT: infrastructure and scaling rules End user MiCADO Submitter Policy Keeper (Scaling logic) cqueue worker MICADO WORKER cqueue workerJobs Scale up/ down jQUEUER MASTER jQueuer Agent Jobs experiment .json Container and Cloud Orchestrator MICADO September 26th 2019
  5. 5. Demo: Agent based simulation framework September 26th 2019 www.project-cola.eu 5 • Experiment: Determining the impact of changes in behaviour on the spread of a disease across a population • Tool: Repast Simphony • Total jobs: 200 • Deadline: 1 hour • Max VMs to use: 10
  6. 6. Execution using jQueuer + MiCADO September 26th 2019 www.project-cola.eu 6 experiment.json JQueuer Manager MiCADO Worker 1 JQueuer Agent Repast 1 MiCADO Master 200 jobs MiCADO Worker 2 JQueuer Agent Repast 2 MiCADO Worker n JQueuer Agent Repast n 1-hour deadline Calculated by scaling policy
  7. 7. jQueuer deployment using ADT September 26th 2019 www.project-cola.eu 7
  8. 8. Deadline-based scaling policy September 26th 2019 www.project-cola.eu 8
  9. 9. Results September 26th 2019 www.project-cola.eu 9 • Dynamic allocation of variable length jobs • Results in better use of cloud resources Manually: 5 VMs MiCADO: 3.86 VMs
  10. 10. Thanks and Questions! September 26th 2019 www.project-cola.eu 10 • github.com/micado-scale • micado-scale.eu/ This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 731574

4/4 Webinar: How to Automate Deployment and Orchestration of Application (MiCADO introduction) This part of the webinar covers the software JQueuer, with JQueuer the computing a large number of jobs results in significant overall execution time. It was presented by Amjad Ullah (University of Westminster). The webinar took place on the 26th of September 2019. If you would like to have more information visit: https://micado-scale.eu MiCADO is open-source and a highly customisable multi-cloud orchestration and auto-scaling framework for Docker containers, orchestrated by Kubernetes. Developed by Project COLA funded by the European Commission (grant agreement no: 731574). https://project-cola.eu

Views

Total views

106

On Slideshare

0

From embeds

0

Number of embeds

5

Actions

Downloads

0

Shares

0

Comments

0

Likes

0

×