Simulating the usage of SLAs for job scheduling in an HPC environment

Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionSimulating the usage of SLAs for job scheduling in an HPC environment Roland K¨bert u H¨chstleistungsrechenzentrum Stuttgart o January 31, 2010 Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro

Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps Discussion1 Introduction2 Job Scheduling - with and without SLAs3 Simulating SLAs-based scheduling4 Conclusions and next steps5 Discussion Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro

Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionMotivation HPC services are only oﬀered on best-eﬀort basis Scheduling parameters are few and only trivial Work about SLAs has been performed at HLRS. . . . . . but is on a higher level Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro

Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionJob scheduling scheduling: “to plan (something) at a certain time” Scheduling is used in many ﬁelds Job scheduling assigns computational jobs to processing units Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro

Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionService Level Agreements in one sentence “The purpose of [a] Service Level Agreement (SLA) is to deﬁne the services and responsibilities of the [service provider] and its clients.” (Michigan State University High Performance Computing Center Service Level Agreement) Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro

Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionClassical job scheduling Objective is mostly to maximize utilization or minimize waiting time Various algorithms with diﬀerent advantages Either schedule-based or queue-based Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro

Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionJob scheduling - with SLAs A quite popular ﬁeld Two main streams SLAs per job Trivial QoS parameters (Timing and resource requirements) Relies on precise speciﬁcation of job execution times Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro

Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionSimulating SLA-based job scheduling Just implementing some scheduling won’t work Production use cannot be done without previous investigations Therefore, use a simulation tool: Alea Needs to be extended in order to investigate SLAs Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro

Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionAlea’s features Supports diﬀerent workload formats Various scheduling algorithms already implemented Visualization features Free software (LGPL) Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro

Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionAlea’s graphs Figure: Screenshot of Alea Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro

Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionAlea’s shortcomings Many hard-coded settings (magic numbers) No extensibility foreseen Not really user-friendly No further developments Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro

Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionAlea’s architecture Figure: High-level architecture of Alea 2.1 Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro

Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionSimulation of service levels Simulation of three different service levels: gold, silver, bronze Different service level distribution were generated and simulated against a workload format (San Diego Supercomputer Center’s Blue Horizon (144 nodes x 8 CPUs)) Investigated changes of waiting time with different distributions of service levels Example: Gold-Silver-Bronze 0-0-100, 0-5-95, 1-4-95, 2-3-95, etc.) Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro

Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionSimulation results Machine usage did not change Introducing service level increases average wait time Increasing number of prioritized jobs increases wait time for lower-prioritized classes Ensuring that not too many high-priority jobs exist enables the service provider to give “soft” guarantees on wait time Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro

Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionConclusions Using SLAs for scheduling is possible (duh) Can range from trivial to complex Simulation is a good way to examine diﬀerent parameters, combinations, workloads, objective functions, ... Publication has been accepted at PARENG 2011 Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro

Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionNext steps Improvements on Alea Conceptual implementation Queue-based against schedule-based algorithms Additional, more complex service levels Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro

Outline Introduction Job Scheduling - with and without SLAs Simulating SLAs-based scheduling Conclusions and next steps DiscussionQuestions Figure: Flammarions Holzstich Roland K¨bert u Simulating the usage of SLAs for job scheduling in an HPC enviro

Simulating the usage of SLAs for job scheduling in an HPC environment

by Roland Kübert on Jan 31, 2011

Accessibility

Tags

Upload Details

Usage Rights

1 Embed 22

Statistics

Simulating the usage of SLAs for job scheduling in an HPC environment — Presentation Transcript