• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Simulating the usage of SLAs for job scheduling in an HPC environment
 

Simulating the usage of SLAs for job scheduling in an HPC environment

on

  • 822 views

How to simulate job scheduling using SLAs in a high-performance computing environment by extending the Alea Grid Scheduling Simulator.

How to simulate job scheduling using SLAs in a high-performance computing environment by extending the Alea Grid Scheduling Simulator.

Statistics

Views

Total Views
822
Views on SlideShare
797
Embed Views
25

Actions

Likes
0
Downloads
13
Comments
0

1 Embed 25

http://www.bimberstube.de 25

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Simulating the usage of SLAs for job scheduling in an HPC environment Simulating the usage of SLAs for job scheduling in an HPC environment Presentation Transcript

    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionSimulating the usage of SLAs for job scheduling in an HPC environment Roland K¨bert u H¨chstleistungsrechenzentrum Stuttgart o January 31, 2010 Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro
    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps Discussion1 Introduction2 Job Scheduling - with and without SLAs3 Simulating SLAs-based scheduling4 Conclusions and next steps5 Discussion Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro
    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps Discussion1 Introduction2 Job Scheduling - with and without SLAs3 Simulating SLAs-based scheduling4 Conclusions and next steps5 Discussion Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro
    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionMotivation HPC services are only offered on best-effort basis Scheduling parameters are few and only trivial Work about SLAs has been performed at HLRS. . . . . . but is on a higher level Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro
    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionJob scheduling scheduling: “to plan (something) at a certain time” Scheduling is used in many fields Job scheduling assigns computational jobs to processing units Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro
    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionService Level Agreements in one sentence “The purpose of [a] Service Level Agreement (SLA) is to define the services and responsibilities of the [service provider] and its clients.” (Michigan State University High Performance Computing Center Service Level Agreement) Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro
    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps Discussion1 Introduction2 Job Scheduling - with and without SLAs3 Simulating SLAs-based scheduling4 Conclusions and next steps5 Discussion Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro
    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionClassical job scheduling Objective is mostly to maximize utilization or minimize waiting time Various algorithms with different advantages Either schedule-based or queue-based Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro
    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionJob scheduling - with SLAs A quite popular field Two main streams SLAs per job Trivial QoS parameters (Timing and resource requirements) Relies on precise specification of job execution times Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro
    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps Discussion1 Introduction2 Job Scheduling - with and without SLAs3 Simulating SLAs-based scheduling4 Conclusions and next steps5 Discussion Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro
    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionSimulating SLA-based job scheduling Just implementing some scheduling won’t work Production use cannot be done without previous investigations Therefore, use a simulation tool: Alea Needs to be extended in order to investigate SLAs Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro
    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionAlea’s features Supports different workload formats Various scheduling algorithms already implemented Visualization features Free software (LGPL) Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro
    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionAlea’s graphs Figure: Screenshot of Alea Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro
    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionAlea’s shortcomings Many hard-coded settings (magic numbers) No extensibility foreseen Not really user-friendly No further developments Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro
    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionAlea’s architecture Figure: High-level architecture of Alea 2.1 Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro
    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionSimulation of service levels Simulation of three different service levels: gold, silver, bronze Different service level distribution were generated and simulated against a workload format (San Diego Supercomputer Center’s Blue Horizon (144 nodes x 8 CPUs)) Investigated changes of waiting time with different distributions of service levels Example: Gold-Silver-Bronze 0-0-100, 0-5-95, 1-4-95, 2-3-95, etc.) Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro
    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionSimulation results Machine usage did not change Introducing service level increases average wait time Increasing number of prioritized jobs increases wait time for lower-prioritized classes Ensuring that not too many high-priority jobs exist enables the service provider to give “soft” guarantees on wait time Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro
    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps Discussion1 Introduction2 Job Scheduling - with and without SLAs3 Simulating SLAs-based scheduling4 Conclusions and next steps5 Discussion Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro
    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionConclusions Using SLAs for scheduling is possible (duh) Can range from trivial to complex Simulation is a good way to examine different parameters, combinations, workloads, objective functions, ... Publication has been accepted at PARENG 2011 Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro
    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionNext steps Improvements on Alea Conceptual implementation Queue-based against schedule-based algorithms Additional, more complex service levels Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro
    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps Discussion1 Introduction2 Job Scheduling - with and without SLAs3 Simulating SLAs-based scheduling4 Conclusions and next steps5 Discussion Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro
    • Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionQuestions Figure: Flammarions Holzstich Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro