Black Box Methods for Inferring Parallel Applications' Properties in Virtual Environments Ashish Gupta March 2008 PhD Fina...
Introduction
Background <ul><li>Virtual Machine Distributed Computing </li></ul><ul><li>Virtuoso </li></ul><ul><ul><ul><li>Middleware f...
Problem Introduction <ul><ul><li>Adaptation  </li></ul></ul><ul><ul><ul><li>Resources can be heterogeneous (CPU, memory et...
Adaptation in Virtuoso Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Input for...
Thesis Statement <ul><li>My thesis is that it is feasible to  infer various useful demands and behavior of a parallel appl...
Black box assumption and impact <ul><li>Black Box  – Can make no assumptions about the implementation, behavior or interna...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Components of my dissertation Vi...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Topics I cover Virtual Topology ...
BSP application model <ul><li>Multiple processes executing a common kernel </li></ul><ul><li>Execution alternates between ...
Patterns <ul><li>Synthetic workload generator developed before. Models a BSP application </li></ul><ul><li>Can execute man...
Patterns application’s capabilities Black Box Methods for Inferring Parallel Applications' properties in Virtual Environme...
NAS parallel benchmarks <ul><li>Developed by NASA  [17, 172] </li></ul><ul><li>Set of programs to evaluate performance of ...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Different contributions of my di...
Collective Inference for Topology: VTTIF
Goal of VTTIF Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Low Level Traffic ...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Traffic Analyzer Rate based  Cha...
Inferred topology Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Parallel Integ...
Black Box Measures of performance
The problem <ul><li>Performance of BSP applications – an important goal </li></ul><ul><li>Lot of work dedicated to improvi...
Cost model for BSP applications <ul><li>Popular strategy: break super-step into its components: </li></ul><ul><ul><ul><li>...
Super-step approach <ul><li>Super-step structure an invariant </li></ul><ul><ul><ul><li>No. of steps depend on parameters ...
A new black box metric: Round Trip Iteration Rate (RIR) <ul><li>Based solely on communication behavior of the application ...
Inter send-packet delay Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments 176  Rec...
Inter send-packet delay Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Traffic ...
Patterns: clusters without load Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments ...
Patterns: clusters with external CPU load Black Box Methods for Inferring Parallel Applications' properties in Virtual Env...
Why Does Circled Bin Correspond To Iteration Rate? <ul><li>Each iteration consists of send, receive and compute phases </l...
Plotting inter send-packet delay for MG Black Box Methods for Inferring Parallel Applications' properties in Virtual Envir...
Computing the RIR metric <ul><li>Previous examples were for Patterns –  </li></ul><ul><ul><li>static performance case, eas...
RIR time series Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments For dynamic appl...
Outputting RIR time series - Workflow Black Box Methods for Inferring Parallel Applications' properties in Virtual Environ...
Representing Dynamic Performance  <ul><li>Define a spectrum of metrics for dynamic performance </li></ul>Black Box Methods...
Computing the stationary Average – sampling issues <ul><li>Sliding window resolution (sampling rate) </li></ul><ul><ul><ul...
Effectiveness of RIR avg Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments For MG ...
RIR time series graph Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments A super-step
Can we predict slowdown of application if we put it under load ? For the IS and MG applications,  which application may be...
Role of CDF <ul><li>RIR-CDF can be used to predict which application will be more affected by external load (for dynamic a...
Other metrics  <ul><li>Power spectrum of RIR </li></ul><ul><ul><ul><li>Gives idea about the super-step structure of the ap...
A sample power spectrum for 4 processes Black Box Methods for Inferring Parallel Applications' properties in Virtual Envir...
Example for MG Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Significant frequ...
Recap <ul><li>We can deduce performance for a BSP application using black box means. </li></ul><ul><li>We can predict perf...
Ball In the Court Methods
The problem <ul><li>Last chapter: Predicting performance under load (Slowdown CDF) </li></ul><ul><li>This chapter: Can we ...
Ball in the Court <ul><li>BSP super-step    computation, communication, sync </li></ul><ul><li>Communication according to...
Developing a strategy <ul><li>Focus on one process </li></ul><ul><li>If just one process is loaded, entire application slo...
Why BIC delay? <ul><li>The traffic trace captures the behavior of the process for the entire duration </li></ul><ul><li>If...
Approach to computing the BIC delay Black Box Methods for Inferring Parallel Applications' properties in Virtual Environme...
With Load, Some BIC Delays Get Larger Black Box Methods for Inferring Parallel Applications' properties in Virtual Environ...
An Algorithm for BIC Delay? <ul><li>Can we estimate the  total BIC delay  from the traffic trace alone ? </li></ul><ul><li...
BIC events <ul><li>* - S* event pairs    can be classified as BIC events </li></ul><ul><li>Intuitively,  </li></ul><ul><u...
Computing the BIC delay <ul><li>1. Count all BIC event pairs and sum up their  time differentials     total BIC delay </l...
Using BIC Delays to Measure Application Imbalance <ul><li>Get a measure of the local computing/processing time for which i...
Using BIC Delays to Measure Application Imbalance Black Box Methods for Inferring Parallel Applications' properties in Vir...
Computing the no-load runtime using BIC delays <ul><li>Global BIC imbalance algorithm:  </li></ul><ul><ul><ul><li>Assumes ...
Global BIC algorithm Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
Evaluation with the IS application Black Box Methods for Inferring Parallel Applications' properties in Virtual Environmen...
More sophisticated Imbalance Algorithms <ul><li>Process-level BIC imbalance algorithm  </li></ul><ul><ul><li>A better fit ...
Other contributions in the Chapter <ul><ul><li>Metrics for computing imbalance amongst processes  </li></ul></ul><ul><ul><...
Conclusions <ul><li>How slow is the application running because of externally imposed load ? </li></ul><ul><li>Ball in the...
Global bottlenecks and Time Decomposition
Global Bottleneck <ul><li>In the BIC discussion, we could identify the slow processes using their BIC delays </li></ul><ul...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments BIC analysis for  local imbalanc...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Different contributions of disse...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Monitoring and inference Applica...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Effect on BSP Application Throug...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Free Network Measurement For Ada...
Closest Related Work <ul><li>Wood et. al. [178] -  Black box and gray box strategies for Virtual Machine Migration, NSDI 2...
Closest Related Work <ul><li>Aguilera et. al. [14] –  Performance debugging for distributed systems of blackboxes, OSP 200...
Related Publications <ul><li>Gupta, A., Dinda, P. A.  Inferring the topology and traffic load of parallel programs running...
Other Work <ul><li>Robert Schweller et al.,  Reversible Sketches: Enabling Monitoring and Analysis over High-speed Data St...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Components of my dissertation Vi...
Thank you !
Backup slides
Infrastructure <ul><li>Virtuoso cluster, which is an IBM e1350 with 32 compute nodes, each of which is a dual 2.2 GHz Inte...
Increasing Application Performance In Virtual Environments Through Run-time Inference and Adaptation <ul><li>Dynamically a...
<ul><li>Informally stated: </li></ul><ul><li>Input </li></ul><ul><ul><li>Network traffic load matrix of application </li><...
Evaluation <ul><li>Applications </li></ul><ul><ul><li>Patterns: A synthetic BSP benchmark </li></ul></ul><ul><ul><li>TPC-W...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Effect on BSP Application Throug...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments TPCW Throughput (WIPS) With Imag...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Free Network Measurement for Ada...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments What is WREN ? Developed by my c...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments An important Contribution: Probl...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Two approaches to adaptation
VTTIF in a dynamic topology environment Black Box Methods for Inferring Parallel Applications' properties in Virtual Envir...
Questions <ul><li>Are these techniques applicable to other applications ? </li></ul><ul><li>Why do you think they are gene...
<ul><li>How about applying the techniques beyond BSP applications ? </li></ul><ul><ul><ul><li>Distributed apps </li></ul><...
Bin
Evidence of Adaptation Driven by Inference
Evidence of Adaptation <ul><li>How can automated inference assist automated adaptation without user intervention and appli...
<ul><li>Informally stated: </li></ul><ul><li>Input </li></ul><ul><ul><li>Network traffic load matrix of application </li><...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Effect on BSP Application Throug...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Evaluation Scenario 2  : Large 2...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments The New Adaptation Process
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Two approaches to adaptation
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Evaluation Scenario 2  : Large 2...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Summary of contributions Virtual...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Summary of contributions Virtual...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Summary of contributions Virtual...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Summary of contributions Virtual...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Summary of contributions Virtual...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Summary of contributions Virtual...
Overview of the Talk <ul><li>Introduction and Background </li></ul><ul><li>Overview of contributions </li></ul><ul><li>Dyn...
Overview of the Talk <ul><li>Runtime Adaptation using black box inference </li></ul><ul><li>Contributions Recap </li></ul>...
Virtual Machine Distributed Computing <ul><li>Advantages </li></ul><ul><ul><ul><li>Helps overcome heterogeneity of resourc...
Impact <ul><li>Further the goal of autonomic computing </li></ul><ul><li>Black box approach    impacts huge set of applic...
Impact Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments <ul><li>Adaptation is jus...
Application Model <ul><li>BSP Parallel applications </li></ul><ul><li>Two benchmarks being considered </li></ul><ul><ul><u...
Citations [167] VALIANT, L. A bridging model for parallel computation.  Communications of the ACM 33 , 8 (Aug. 1990), 103–...
Two threads to my dissertation <ul><li>Application inference </li></ul><ul><ul><li>Inference has to be fully automated – z...
Two threads to my dissertation <ul><li>2. Applying inference: Autonomic adaptation </li></ul><ul><ul><ul><li>Virtuoso impl...
Other inference properties <ul><li>Traffic topology for BSP-style distributed applications – work done in my initial VTTIF...
Impact <ul><li>Was the first paper to  propose and demonstrate collective inference for a virtual distributed environment ...
A VNET virtual layer <ul><li>VNET </li></ul><ul><ul><li>Abstraction:  A set of VMs on same Layer 2 network </li></ul></ul>...
Overall Design <ul><li>VNET </li></ul><ul><ul><li>Abstraction:  A set of VMs on same Layer 2 network </li></ul></ul><ul><l...
Dynamic Topology Inference by VTTIF Black Box Methods for Inferring Parallel Applications' properties in Virtual Environme...
Contributions <ul><li>Understanding the correlation between network traffic and inter-process interactions </li></ul><ul><...
Solving the issues <ul><li>Sampling rate: </li></ul><ul><ul><ul><li>Compute at a “high enough” sample rate to capture enou...
Solving the issues Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
Capturing the right time duration <ul><li>Again a  Power Spectrum based approach </li></ul><ul><li>Helps us determine if o...
Effect of load on the RIR time series Black Box Methods for Inferring Parallel Applications' properties in Virtual Environ...
The RIR-CDF metric <ul><li>More exhaustive than the RIR avg </li></ul><ul><li>Statistical scheduling </li></ul><ul><li>App...
The RIR-CDF metric Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments CDF indicates...
Taking guidance from RIR-CDF : a proof of concept <ul><li>CDF indicates the RIR region application spends most of its time...
Effect of load on different RIR areas Black Box Methods for Inferring Parallel Applications' properties in Virtual Environ...
Scheduling using the CDF Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments <ul><ul...
Scheduling using the CDF Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments For the...
Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Slowdown CDF For MG Slowdown CDF...
Actual slowdown <ul><li>MG    2.4 times </li></ul><ul><li>IS    15.52 times </li></ul><ul><li>IS is affected much more f...
Power Spectrum <ul><ul><li>Other applications of the Power Spectrum </li></ul></ul><ul><ul><ul><li>Statistical Scheduling ...
Why do I want to do this? <ul><li>Close synchronization in BSP: a single slow process can slowdown entire application dras...
The Benefits <ul><li>If we can know how badly the application is affected from the load, </li></ul><ul><li>Appropriate ste...
Process Level BIC imbalance Algorithm <ul><li>A better fit for more dynamic applications like the NAS benchmarks </li></ul...
Upcoming SlideShare
Loading in …5
×

Black Box Methods for Inferring Parallel Applications' Properties in Virtual Environments

1,043
-1

Published on

My PhD defense talk.
Please see www.ashishgupta.org for more details.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,043
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
12
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Adopt the “we” attitude. We will be looking at two main part in the presentation. The first part is about low overhead statistics gathering.
  • Black Box Methods for Inferring Parallel Applications' Properties in Virtual Environments

    1. 1. Black Box Methods for Inferring Parallel Applications' Properties in Virtual Environments Ashish Gupta March 2008 PhD Final Talk Committee: Prof. Peter Dinda Prof. Fabian Bustamante Prof. Yan Chen Prof. Dongyan Xu (Purdue University)
    2. 2. Introduction
    3. 3. Background <ul><li>Virtual Machine Distributed Computing </li></ul><ul><li>Virtuoso </li></ul><ul><ul><ul><li>Middleware for autonomic Virtual Machine distributed computing </li></ul></ul></ul><ul><ul><ul><li>Presents a simple abstraction for distributed computing, insulating from underlying computational, networking and middleware complexities </li></ul></ul></ul><ul><ul><ul><li>VMM abstracts computational resources </li></ul></ul></ul><ul><ul><ul><li>VNET abstracts different networking domains into one – also ideal point for monitoring </li></ul></ul></ul><ul><ul><ul><li>Autonomic Resource Management </li></ul></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    4. 4. Problem Introduction <ul><ul><li>Adaptation </li></ul></ul><ul><ul><ul><li>Resources can be heterogeneous (CPU, memory etc) </li></ul></ul></ul><ul><ul><ul><li>If shared, then resources availability can also highly dynamic </li></ul></ul></ul><ul><ul><ul><li>Application demands also change ! </li></ul></ul></ul><ul><ul><li>Autonomic computing: </li></ul></ul><ul><ul><ul><li>What is available ? </li></ul></ul></ul><ul><ul><ul><li>What is required ? </li></ul></ul></ul><ul><ul><ul><li>How can we effectively match the two ? </li></ul></ul></ul><ul><ul><ul><li>One of the major components  What does the distributed application want ? </li></ul></ul></ul><ul><ul><ul><li>Should work with existing unmodified applications and OS </li></ul></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    5. 5. Adaptation in Virtuoso Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Input for adaptation problem formalization in my colleague Ananth Sundararaj’s thesis Different input components Many more inferable properties VM/application demands Resource availability User constraints
    6. 6. Thesis Statement <ul><li>My thesis is that it is feasible to infer various useful demands and behavior of a parallel application running inside a collection of VMs to a significant degree using a black box model . To evaluate this thesis, I enumerate and define various demands and types of behavior that can be inferred, and also design, implement and evaluate ideas and approaches towards inferring these. </li></ul><ul><li>One of the demands I infer is the communication behavior and the runtime topology of a parallel application. I also show how to infer some very useful runtime properties of a parallel application like its runtime performance , its slowdown under external load and its global bottlenecks . </li></ul><ul><li>Significantly all of this is done using black-box assumptions and without specific assumptions about the application or the operating system . I also give evidence of how automatic black box inference can assist in adapting the application and its resource usage resulting in improved performance. </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Chapter 2 Chapter 3 Chapter 4 Appendices A, B
    7. 7. Black box assumption and impact <ul><li>Black Box – Can make no assumptions about the implementation, behavior or internal state of the Guest OS/module beyond its external interface </li></ul><ul><li>Lowers barrier to adoption of the new inference techniques </li></ul><ul><li>helps deploy my work to legacy applications </li></ul><ul><li>Mainly accomplished by looking at external signals: </li></ul><ul><ul><li>traffic, host load etc. </li></ul></ul><ul><li>Not tied to Virtuoso </li></ul><ul><ul><li>Other systems like softUDC [91], XenoServer [52], SODA [87], Violin [88], In-VIGO [13], VioCluster [136] can also benefit from black box application inference </li></ul></ul><ul><ul><li>Potentially any virtual distributed system </li></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    8. 8. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Components of my dissertation Virtual Topology and Traffic Inference Framework Black box metrics for Absolute Performance Ball in The Court principles to compute application slowdown Global Bottlenecks using time decomposition Increasing Application Performance In Virtual Environments through Run-time Inference and Adaptation Free Network Measurement For Adaptive Virtualized Distributed Computing Inference Adaptation 1 2 3 4 5 6
    9. 9. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Topics I cover Virtual Topology and Traffic Inference Framework Black box metrics for Absolute Performance Ball in The Court principles to compute application slowdown Global Bottlenecks using time decomposition Increasing Application Performance In Virtual Environments through Run-time Inference and Adaptation Free Network Measurement For Adaptive Virtualized Distributed Computing Inference Adaptation 2 3 4 5 A B
    10. 10. BSP application model <ul><li>Multiple processes executing a common kernel </li></ul><ul><li>Execution alternates between one or more computing phases and one or more communication phases </li></ul><ul><li>Combination of a schedule of computation and communication phases  Super-step </li></ul><ul><li>A very popular model for implementing a large variety of scientific applications and parallel algorithms </li></ul><ul><li>Original paper: Valiant [167] </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    11. 11. Patterns <ul><li>Synthetic workload generator developed before. Models a BSP application </li></ul><ul><li>Can execute many different types of topologies common in BSP programs </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Some parameters Topology Number of processors Message size # of iterations Flops per element Memory Reads/Writes per element Topologies N-dimensional mesh N-dimensional torus N-dimensional hypercube Binary reduction tree All to All
    12. 12. Patterns application’s capabilities Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments 2-D Mesh 3-D Toroid 3-D Hypercube Reduction Tree All-to-All
    13. 13. NAS parallel benchmarks <ul><li>Developed by NASA [17, 172] </li></ul><ul><li>Set of programs to evaluate performance of parallel supercomputers </li></ul><ul><li>Representative of CFD applications </li></ul><ul><li>To generate realistic parallel application workloads </li></ul><ul><li>5 kernel benchmarks: EP, MG, FT, IS, CG </li></ul><ul><ul><ul><li>Five very frequently used numeric methods </li></ul></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    14. 14. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Different contributions of my dissertation Virtual Topology and Traffic Inference Framework Black box metrics for Absolute Performance Ball in The Court principles to compute application slowdown Global Bottlenecks using time decomposition Increasing Application Performance In Virtual Environments through Run-time Inference and Adaptation Free Network Measurement For Adaptive Virtualized Distributed Computing Inference Adaptation 2 3 4 5 A B
    15. 15. Collective Inference for Topology: VTTIF
    16. 16. Goal of VTTIF Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Low Level Traffic Monitoring ? An online topology inference framework for a VM environment Application Topology
    17. 17. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Traffic Analyzer Rate based Change detection Traffic Matrix Query Agent VM Network Scheduling Agent VNET daemon VM VNET overlay network To other VNET daemons Physical Host VTTIF Architecture
    18. 18. Inferred topology Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Parallel Integer Sort
    19. 19. Black Box Measures of performance
    20. 20. The problem <ul><li>Performance of BSP applications – an important goal </li></ul><ul><li>Lot of work dedicated to improving performance of parallel applications, e.g. </li></ul><ul><ul><ul><li>Virtuoso </li></ul></ul></ul><ul><ul><ul><li>VioCluster </li></ul></ul></ul><ul><ul><li>How do we measure performance in a black box fashion ? </li></ul></ul><ul><ul><ul><li>The current way in Virtuoso is manual (e.g. Lin et.al. [106]) </li></ul></ul></ul><ul><ul><li>Impact </li></ul></ul><ul><ul><ul><li>Would enable superior adaptation algorithms </li></ul></ul></ul><ul><ul><ul><li>Automated evaluation and adaptation cycle </li></ul></ul></ul><ul><ul><ul><li>Generate reports on effectiveness of different adaptation/scheduling methods </li></ul></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    21. 21. Cost model for BSP applications <ul><li>Popular strategy: break super-step into its components: </li></ul><ul><ul><ul><li>computational cost </li></ul></ul></ul><ul><ul><ul><li>Communication cost of the global exchange of the data </li></ul></ul></ul><ul><ul><ul><li>Synchronization cost </li></ul></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Computation cost Communication cost Number of super steps Sync latency cost Speed of computation in FLOPS Static model of performance + requires detailed application profiling and access to source code
    22. 22. Super-step approach <ul><li>Super-step structure an invariant </li></ul><ul><ul><ul><li>No. of steps depend on parameters and data </li></ul></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Another possible measure of performance: number of super-steps executed per second, or the iteration rate  dynamic metric Multiple super-steps for dynamic applications  iteration rate is not constant
    23. 23. A new black box metric: Round Trip Iteration Rate (RIR) <ul><li>Based solely on communication behavior of the application </li></ul><ul><li>Correlated to the iteration rate </li></ul><ul><li>Indirectly measures number of process interactions  this indicates progress as synchronization happens at end of a super-step </li></ul><ul><li>Approach: Examined various properties of the traffic trace </li></ul><ul><li>Inter send-packet delay exhibits interesting properties </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    24. 24. Inter send-packet delay Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments 176 Receive from 175 176 Send to 175 176 Send to 175 176 Send to 175 176 Receive from 175 176 Send to 175
    25. 25. Inter send-packet delay Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Traffic trace for Patterns Message size = 4000 bytes Computation per iteration: 100 MFlops Clustering based on inter-send delays Count in cluster matches actual iteration rate output by application (325)
    26. 26. Patterns: clusters without load Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Actual: 568 Reported: 569
    27. 27. Patterns: clusters with external CPU load Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Actual: 142 Reported: 153 Actual execution time ratio: 3.922 Ratio from reported iteration rate: 3.72 Within 5%
    28. 28. Why Does Circled Bin Correspond To Iteration Rate? <ul><li>Each iteration consists of send, receive and compute phases </li></ul><ul><li>Number of items in large inter send-packet delay cluster represent the group that represent an inter-process interaction </li></ul><ul><li>Each inter-process interaction represents progress in the super-step </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments time Send packets
    29. 29. Plotting inter send-packet delay for MG Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments <ul><li>No clean clusters for a more complex application </li></ul><ul><li>Delays shift towards right on greater load </li></ul>
    30. 30. Computing the RIR metric <ul><li>Previous examples were for Patterns – </li></ul><ul><ul><li>static performance case, easy clustering </li></ul></ul><ul><li>Applications like MultiGrid from NAS benchmarks  changing iteration rate </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments For a given packet time series, 1. Count send pairs whose inter-packet delay exceeds by c * RTT 2. Send pair must be interleaved by one receive Based on BSP principles
    31. 31. RIR time series Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments For dynamic applications, RIR changes with time Need a time series for RIR over the trace
    32. 32. Outputting RIR time series - Workflow Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Sniff Packets Send packets that obey the conditions Slide a 1 second window over these send packets Sliding interval =  t Get a new time series denoting RIR for each 1 second sliding window Sampling duration = T Derivative Metrics from the above time series Average RIR CDF Power Spectrum Super-phase period Using tcpdump/libpcap for the VM traffic Send packets satisfying the two conditions 1 sec Slide by  t i 1 , i 2, i 3, i 4, i 5, …. i n Each number represents number of iterations for a particular 1 sec window instance
    33. 33. Representing Dynamic Performance <ul><li>Define a spectrum of metrics for dynamic performance </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments RIR avg RIR-CDF RIR-PS RIR-PSE Long term stationary average of RIR time series CDF of RIR time series indicating spread of iteration rates Phase structure, periodicities, application fingerprinting, statistical scheduling Summary of the periodic behavior for multiple supersteps
    34. 34. Computing the stationary Average – sampling issues <ul><li>Sliding window resolution (sampling rate) </li></ul><ul><ul><ul><li>Needs to be high enough to capture the any important high frequency behavior </li></ul></ul></ul><ul><li>Capture duration </li></ul><ul><ul><ul><li>Enough to capture the stationarity of the signal (repetition of all super-steps) </li></ul></ul></ul><ul><li>Assumption: </li></ul><ul><ul><ul><li>iteration dynamics of the application are indeed empirically stationary for the long run. </li></ul></ul></ul><ul><ul><ul><li>For a dynamic application that consist of repetitive phases, it means capturing enough of its performance behavior to capture this repetitive element. </li></ul></ul></ul><ul><li>Capturing stationarity </li></ul><ul><ul><li>Power Spectrum based techniques help us determine the right sampling rate and duration </li></ul></ul><ul><ul><li>Details in dissertation </li></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    35. 35. Effectiveness of RIR avg Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments For MG application, running under different load conditions (100% and 60% CPU load), predicted execution time error rates were 13% and 7% respectively (completely black box) Value of c = 1.1 here (c*RTT factor)
    36. 36. RIR time series graph Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments A super-step
    37. 37. Can we predict slowdown of application if we put it under load ? For the IS and MG applications, which application may be hurt more if one of the processes from each application shares the physical host with an external computational load? The impact: we can now determine in advance , the impact of external load if we must choose one of these applications to be influenced by the load. <ul><li>Scheduling fact: Depending the scheduling algorithm, affect of load on different RIR regions can be different ( Govindan et.al. [60]) </li></ul><ul><li>Reason: Scheduling handled differently for CPU bound processes vs I/O bound processes </li></ul><ul><li>“ Providing enough CPU is not enough, an equally important consideration is to provide the CPU at the right time” </li></ul>
    38. 38. Role of CDF <ul><li>RIR-CDF can be used to predict which application will be more affected by external load (for dynamic applications) </li></ul><ul><li>Very useful when extra load needs to be introduced over existing applications due to demand </li></ul><ul><li>Scheduling using the CDF: </li></ul><ul><ul><ul><li>From the normal CDF, we predict a slowdown CDF based on a slowdown mapping </li></ul></ul></ul><ul><ul><ul><li>Slowdown CDF  How will the RIR-CDF of the application look like after load ? </li></ul></ul></ul><ul><ul><ul><li>Slowdown mapping  What is the slowdown for a particular RIR under a particular load ? </li></ul></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Slowdown mapping
    39. 39. Other metrics <ul><li>Power spectrum of RIR </li></ul><ul><ul><ul><li>Gives idea about the super-step structure of the application </li></ul></ul></ul><ul><ul><ul><li>Length of Super-step </li></ul></ul></ul><ul><ul><ul><li>A summary of the power spectrum and the significant frequencies serves as a fingerprint/super-step snapshot </li></ul></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    40. 40. A sample power spectrum for 4 processes Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Consistency across processes
    41. 41. Example for MG Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Significant frequency separation Exec time = 19.44 seconds Super-phase period = 4.1 seconds (1/0.244) Number of super-phases ~4
    42. 42. Recap <ul><li>We can deduce performance for a BSP application using black box means. </li></ul><ul><li>We can predict performance for an application, when imposed with load </li></ul><ul><li>Entirely based on Based on packet analysis </li></ul><ul><li>New metric called RIR : Round trip Iteration Rate </li></ul><ul><li>More complex metrics for dynamic applications </li></ul><ul><ul><ul><li>RIR avg </li></ul></ul></ul><ul><ul><ul><li>RIR-CDF </li></ul></ul></ul><ul><ul><ul><li>RIR Power Spectrum </li></ul></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    43. 43. Ball In the Court Methods
    44. 44. The problem <ul><li>Last chapter: Predicting performance under load (Slowdown CDF) </li></ul><ul><li>This chapter: Can we predict performance of application if an existing external load was removed ? </li></ul><ul><li>I.e. What is the slowdown of the application under the current load conditions ? </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    45. 45. Ball in the Court <ul><li>BSP super-step  computation, communication, sync </li></ul><ul><li>Communication according to a certain schedule and then computation in between </li></ul><ul><li>Each process acts (computes) on a message before sending out the next one </li></ul><ul><li>This acting on a message is called “Ball in the Court” (BIC) </li></ul><ul><li>Ball is the responsibility of the process to do some local processing and then interact with other processes , court is the local host. </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments BIC delay
    46. 46. Developing a strategy <ul><li>Focus on one process </li></ul><ul><li>If just one process is loaded, entire application slowed down because of one process </li></ul><ul><li>All processes operate in sync, and iterations can only proceed if the loaded process does its duties </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments stretched Under load
    47. 47. Why BIC delay? <ul><li>The traffic trace captures the behavior of the process for the entire duration </li></ul><ul><li>If the process slows down, the trace time length will increase. </li></ul><ul><li>There will be corresponding changes in the trace as well. What are those changes ?  Seed of the idea </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    48. 48. Approach to computing the BIC delay Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Let’s compute the time differential for event pairs, for *.176 Some computation (6 ms) Some computation (6ms)
    49. 49. With Load, Some BIC Delays Get Larger Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Loaded process Unloaded process 1. Sending an ack  process’s responsibility 2. This responsibility was hugely inflated in the loaded case ( 64 us to 23822 us) 3. BIC delays for other receives are similar (receive  BIC for other processes)
    50. 50. An Algorithm for BIC Delay? <ul><li>Can we estimate the total BIC delay from the traffic trace alone ? </li></ul><ul><li>Investigated this question with different approaches. </li></ul><ul><li>Each packet can be of the following types: </li></ul><ul><ul><ul><li>1. Send Packet (SP) </li></ul></ul></ul><ul><ul><ul><li>2. Send Ack (SA) </li></ul></ul></ul><ul><ul><ul><li>3. Receive Packet (SP) </li></ul></ul></ul><ul><ul><ul><li>4. Receive Ack (RA) </li></ul></ul></ul><ul><ul><li>We can pair up consecutive packets to form event pairs  SA followed by SP  SA-SP </li></ul></ul><ul><ul><li>For 4 event possibilities  we have 16 event pairs </li></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    51. 51. BIC events <ul><li>* - S* event pairs  can be classified as BIC events </li></ul><ul><li>Intuitively, </li></ul><ul><ul><ul><li>process has either received a packet and is responsible to send the next one </li></ul></ul></ul><ul><ul><ul><li>Just sent a packet and is responsible to send the next one as well </li></ul></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    52. 52. Computing the BIC delay <ul><li>1. Count all BIC event pairs and sum up their time differentials  total BIC delay </li></ul><ul><li>2. BIC delays partitioned by the recipient IP address </li></ul><ul><ul><ul><li>E.g.: RP (175) – SP (173)  delay goes into the bucket of 173 </li></ul></ul></ul><ul><ul><li>Some special cases and concerns for counting BIC delays are explained in the dissertation (Page 140) </li></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    53. 53. Using BIC Delays to Measure Application Imbalance <ul><li>Get a measure of the local computing/processing time for which its responsible </li></ul><ul><li>Comparing this local time with another unloaded process  gives an idea of imbalance for the loaded process </li></ul><ul><li>Gives an idea of how much extra time the loaded process is taking </li></ul><ul><li>The way we compare  produces different strategies for computing slowdown </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    54. 54. Using BIC Delays to Measure Application Imbalance Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Compute BIC delay Imbalance Algorithm Slowdown
    55. 55. Computing the no-load runtime using BIC delays <ul><li>Global BIC imbalance algorithm: </li></ul><ul><ul><ul><li>Assumes all processes are executing the same BSP kernel </li></ul></ul></ul><ul><ul><ul><li>1. Compute the BIC delay for the loaded process </li></ul></ul></ul><ul><ul><ul><li>2. Select a partner process for comparison </li></ul></ul></ul><ul><ul><ul><ul><li>What if the loaded process was put in the shoes of the partner process ? </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Choice of partner process can affect results. Select least loaded for most optimistic results. Or select one that reflects possible migration conditions </li></ul></ul></ul></ul><ul><ul><ul><li>3. Compute the “imbalance” by finding the difference in the BIC delays of these two processes </li></ul></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    56. 56. Global BIC algorithm Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    57. 57. Evaluation with the IS application Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Balanced case Total runtime = 14.36 seconds Total runtime = 152.67 seconds Loaded case Imbalance between 176 and 175 = 124.49 seconds Actual difference = 138.41 seconds Using completely black box means, we could determine, how slow is the application running with load, without knowing about the unloaded case
    58. 58. More sophisticated Imbalance Algorithms <ul><li>Process-level BIC imbalance algorithm </li></ul><ul><ul><li>A better fit for more dynamic applications like the NAS benchmarks </li></ul></ul><ul><ul><li>Sometimes work done by all processes is not the same </li></ul></ul><ul><ul><li>Algorithm takers into account inter-process level interactions and BIC delays </li></ul></ul><ul><li>Multi-iteration BIC-delay Bias-Aware Imbalance Algorithm </li></ul><ul><ul><li>BIC algorithm for multi-load situations </li></ul></ul><ul><li>Covered in detail in the dissertation </li></ul><ul><li>Gives a range of slowdown (optimistic and pessimistic values) </li></ul><ul><li>Evaluated with MG and IS benchmarks </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    59. 59. Other contributions in the Chapter <ul><ul><li>Metrics for computing imbalance amongst processes </li></ul></ul><ul><ul><ul><li>Gives idea of heterogeneity experienced by different processes </li></ul></ul></ul><ul><ul><ul><li>Draws attention to applications that can use lot of help </li></ul></ul></ul><ul><ul><ul><li>Three imbalance metrics </li></ul></ul></ul><ul><ul><ul><ul><li>Two global: Standard Deviation from average, Squared distance </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Process Level imbalance metric </li></ul></ul></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    60. 60. Conclusions <ul><li>How slow is the application running because of externally imposed load ? </li></ul><ul><li>Ball in the Court concept and ways to measure it </li></ul><ul><li>Various algorithms to compute the imbalance </li></ul><ul><ul><ul><li>Global Imbalance Algorithm </li></ul></ul></ul><ul><ul><ul><li>Process-level Bias Aware BIC Imbalance Algorithm </li></ul></ul></ul><ul><ul><ul><li>Multi-load BIC Imbalance Algorithm </li></ul></ul></ul><ul><ul><li>Application Imbalance metrics </li></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    61. 61. Global bottlenecks and Time Decomposition
    62. 62. Global Bottleneck <ul><li>In the BIC discussion, we could identify the slow processes using their BIC delays </li></ul><ul><li>This chapter extends the argument to network and I/O resources </li></ul><ul><li>Looks at the problem of Global Time Decomposition </li></ul><ul><ul><ul><li>Where does the application spend its time ? </li></ul></ul></ul><ul><ul><ul><li>At network resource and process level </li></ul></ul></ul><ul><ul><li>Time decomposition can point out potential sources of imbalance </li></ul></ul><ul><ul><li>Some metrics output are: </li></ul></ul><ul><ul><ul><li>Cumulative Message Latency </li></ul></ul></ul><ul><ul><ul><li>Average Latencies observed </li></ul></ul></ul><ul><ul><ul><li>Cumulative message transfer time </li></ul></ul></ul><ul><ul><ul><li>Average Bandwidth observed </li></ul></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    63. 63. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments BIC analysis for local imbalance Global Time Decomposition Measure current performance <ul><li>Indicates if application is highly imbalanced, </li></ul><ul><li>Indicates amount of slowdown, and </li></ul><ul><li>Which host to focus on? </li></ul><ul><li>Shows time allocation for network components, and </li></ul><ul><li>Which non-host resources to focus on ? </li></ul><ul><li>Is application running at expected speed compared to previous known instances ? </li></ul><ul><li>Multiple application case: which application will be slowed down more from load ? </li></ul>Eliminate a bottleneck using adaptation mechanisms NO <ul><li>Traffic based iteration rate metrics </li></ul><ul><li>RIR time series </li></ul><ul><li>Dynamic Performance Metrics </li></ul><ul><li>Global BIC imbalance algorithm </li></ul><ul><li>Process-level Bias Aware BIC imbalance algorithm </li></ul><ul><li>Multi-load BIC imbalance algorithm </li></ul><ul><li>Imbalance metrics </li></ul><ul><li>Global Time decomposition techniques for various network metrics </li></ul><ul><li>Unix tools to find root case </li></ul><ul><li>Adaptation Mechanisms </li></ul><ul><ul><li>Migration </li></ul></ul><ul><ul><li>Network overlay links </li></ul></ul><ul><ul><li>Routing rules </li></ul></ul><ul><ul><li>Network reservation </li></ul></ul>Flowchart for diagnosis Chap 3 Chap 4 Chap 5 Appendices A,B Tools Questions/problems
    64. 64. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Different contributions of dissertation Virtual Topology and Traffic Inference Framework Black box metrics for Absolute Performance Ball in The Court principles to compute application slowdown Global Bottlenecks using time decomposition Increasing Application Performance In Virtual Environments through Run-time Inference and Adaptation Free Network Measurement For Adaptive Virtualized Distributed Computing Inference Adaptation 2 3 4 5 A B * An API also defined in the dissertation
    65. 65. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Monitoring and inference Application performance measure Adaptation algorithm Adaptation mechanisms Adaptation Applications Optimization metric <ul><li>Overlay topology </li></ul><ul><li>Forwarding rules </li></ul><ul><li>VM migration </li></ul><ul><li>Application Throughput </li></ul><ul><li>BSP </li></ul><ul><li>Transactional ecommerce </li></ul><ul><li>Application throughput </li></ul><ul><li>VTTIF </li></ul><ul><li>Network monitoring </li></ul><ul><li>Single metric </li></ul><ul><li>Combined metric </li></ul>
    66. 66. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Effect on BSP Application Throughput For high Compute/Communicate Ratio, Migration + Topology dramatically improves performance Adapting to External Load Imbalance External load removed, but I/O still dominates, so topology helps External load removed, can drive higher I/O
    67. 67. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Free Network Measurement For Adaptive Virtualized Distributed Computing
    68. 68. Closest Related Work <ul><li>Wood et. al. [178] - Black box and gray box strategies for Virtual Machine Migration, NSDI 2007 </li></ul><ul><li>Goal: To improve performance of Stand-alone applications running inside VMs </li></ul><ul><li>Black box and gray box strategies </li></ul><ul><li>Black box  externally visible parameters like CPU load, disk swapping activity and network usage </li></ul><ul><li>Focuses on stand-alone applications </li></ul><ul><li>I focus on distributed parallel applications  collective properties </li></ul>
    69. 69. Closest Related Work <ul><li>Aguilera et. al. [14] – Performance debugging for distributed systems of blackboxes, OSP 2003 </li></ul><ul><li>Goal: To find components that contribute to high latency amongst critical message paths in a distributed application </li></ul><ul><li>Somewhat analagous to BIC chapter </li></ul><ul><li>Assume constant delay per component </li></ul><ul><li>Unclear how their techniques translate to parallel applications (cyclic computation) </li></ul><ul><li>No estimate of actual slowdown </li></ul><ul><li>Reynolds et. al. [133] – WAP5: black-box performance debugging for wide-area systems, WWW 2006 </li></ul><ul><li>Focus on wide area systems now, more focus on the network delay aspects </li></ul><ul><li>Major work focuses on identifying causal relationships between messages </li></ul><ul><li>Assume a one to one message casualty  not so in Parallel applications </li></ul><ul><li>Acknowledge that will not work for barrier type communications </li></ul>
    70. 70. Related Publications <ul><li>Gupta, A., Dinda, P. A. Inferring the topology and traffic load of parallel programs running in a virtual machine environment . In Proceedings of the 10th Workshop on Job Scheduling Strategies for Parallel Processing (JSPPS 2004 (June 2004). </li></ul><ul><li>A. Sundararaj, A. Gupta, P. Dinda, Dynamic Topology Adaptation of Virtual Networks of Virtual Machines , Proceedings of the Seventh Workshop on Languages, Compilers and Run-time Support for Scalable Systems (LCR 2004) </li></ul><ul><li>Sundararaj, A. Gupta, P. Dinda, Increasing Distributed Application Performance in Virtual Environments through Run-time Inference and Adaptation , In Proceedings of the 14th IEEE International Symposium on High Performance Distributed Computing (HPDC 2005) </li></ul><ul><li>Ashish Gupta, Ananth Sundararaj, Marcia Zangrilli, Peter Dinda, Bruce B. Lowekamp, Free Network Measurement For Adaptive Virtualized Distributed Computing , In Proceedings of 20th IEEE International Parallel & Distributed Processing Symposium, 2006. </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    71. 71. Other Work <ul><li>Robert Schweller et al., Reversible Sketches: Enabling Monitoring and Analysis over High-speed Data Streams , Journal paper in IEEE/ACM Transactions on Networking, 2006. </li></ul><ul><li>Robert Schweller et al., Monitoring Flow-level High-speed Data Streams with Reversible Sketches , In Proceedings of IEEE INFOCOM 2006. </li></ul><ul><li>Robert Schweller, Ashish Gupta, Elliot Parsons, Yan Chen, Reverse Hashing Algorithms for Sketch-based Change Detection on Highspeed Networks: , In Proceedings of ACM SIGCOMM Internet Measurement Conference, October 2004, Taormina, Sicily </li></ul><ul><li>P. Dinda, G. Memik, R. Dick, B. Lin, A. Mallik, A. Gupta, S. Rossoff, The User In Experimental Computer Systems Research , Proceedings of the Workshop on Experimental Computer Science (ExpCS 2007) </li></ul><ul><li>A. Gupta, B. Lin, P. Dinda, Measuring And Understanding User Comfort With Resource Borrowing , In Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing (HPDC 2004), Honolulu, Hawaii </li></ul><ul><li>Bin Lin, A. Gupta, Peter Dinda, Measuring, Understanding, and Exploiting Direct User Input In Resource Scheduling ,  Journal paper in submission </li></ul><ul><li>Accepted Posters </li></ul><ul><li>Ashish Gupta, Peter Dinda, Fabian Bustamante , Distributed Popularity Indices , Poster Presentation at ACM SIGCOMM 2005, Philadelphia </li></ul><ul><li>Ashish Gupta, Manan Sanghi, Peter Dinda, Fabian Bustamante, Magnolia: a novel DHT architecture for Keyword based search , Poster Presentation at Network System Design and Implementation (NSDI 2005), Boston </li></ul><ul><li>Ashish Gupta et al., Free Network Measurement For Adaptive Virtualized Distributed Computing , Poster Presentation at Supercomputing 2005, Seattle </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Sketch-based reverse hashing work User feedback work
    72. 72. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Components of my dissertation Virtual Topology and Traffic Inference Framework Black box metrics for Absolute Performance Ball in The Court principles to compute application slowdown Global Bottlenecks using time decomposition Increasing Application Performance In Virtual Environments through Run-time Inference and Adaptation Free Network Measurement For Adaptive Virtualized Distributed Computing Inference Adaptation 1 2 3 4 5 6
    73. 73. Thank you !
    74. 74. Backup slides
    75. 75. Infrastructure <ul><li>Virtuoso cluster, which is an IBM e1350 with 32 compute nodes, each of which is a dual 2.2 GHz Intel HT Xeon Processors (except the management node which is faster), 1.5 GB RAM, and 40 GB of disk. </li></ul><ul><li>The machines also have 1 Gbit interfaces and in some evaluation scenarios in later chapters(Chapters 3, 4, 5), I use 4 of these machines which form a mini-cluster of their own. These machines are then connected via a 100 Mbit switch. </li></ul><ul><li>Virtual Machine Monitor Used: </li></ul><ul><ul><li>I use both VMWare and the popular open source Xen VMM in my evaluations. </li></ul></ul><ul><ul><li>VMWare GSX Server 2.5 (VTTIF evaluation) </li></ul></ul><ul><ul><li>Xen version 3.0.3-rc3-1.2798.f in my evaluations in Linux Kernel release 2.6.18-1.2798.fc6xen. (other chapters) </li></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    76. 76. Increasing Application Performance In Virtual Environments Through Run-time Inference and Adaptation <ul><li>Dynamically adapt unmodified applications on unmodified operating systems in virtual environments to available resources </li></ul><ul><li>The adaptation mechanisms are application independent and controlled automatically without user or developer help </li></ul><ul><li>Demonstrate the feasibility of adaptation at the level of collection of VMs connected by Virtual Networks </li></ul><ul><li>Show that its benefits can be significant for two classes of applications </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Published in LCR 2004, HPDC 2005
    77. 77. <ul><li>Informally stated: </li></ul><ul><li>Input </li></ul><ul><ul><li>Network traffic load matrix of application </li></ul></ul><ul><ul><li>Topology of the network </li></ul></ul><ul><li>Output </li></ul><ul><ul><li>Mapping of VMs to hosts </li></ul></ul><ul><ul><li>Overlay topology connecting hosts </li></ul></ul><ul><ul><li>Forwarding rules on the topology </li></ul></ul><ul><ul><li>Such that the application throughput is maximized </li></ul></ul>Optimization Problem (2/2) Topology + Migration Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments The algorithm is described in detail in the paper
    78. 78. Evaluation <ul><li>Applications </li></ul><ul><ul><li>Patterns: A synthetic BSP benchmark </li></ul></ul><ul><ul><li>TPC-W: Transactional web ecommerce benchmark </li></ul></ul><ul><li>Benefits of adaptation (performance speedup) </li></ul><ul><ul><li>Adapting to compute/communicate ratio </li></ul></ul><ul><ul><li>Adapting to external load imbalance </li></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    79. 79. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Effect on BSP Application Throughput Adapting to Compute/Communicate Ratio For high Compute/Communicate Ratio, Migration + Topology dramatically improves performance Less time spent in I/O, so migration alone is enough Since I/O dominates, drop in latency improves performance Even for small amount of I/O, it takes up significant time
    80. 80. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments TPCW Throughput (WIPS) With Image Server Facing External Load No Topology Topology No Migration 1.216 1.76 Migration 1.4 2.52
    81. 81. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Free Network Measurement for Adaptive Virtualized Distributed Computing <ul><li>ADAPTATION : A FOUR STEP PROCESS </li></ul><ul><li>Automatically infer application demands (network/CPU) </li></ul><ul><li>Monitor resource availability (bw/latency/CPU) </li></ul><ul><li>Adapt distributed application for better performance/cost effectiveness </li></ul><ul><li>Reserve Resources when possible </li></ul>WREN PAPER, Published in IPDPS 2006
    82. 82. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments What is WREN ? Developed by my colleague Marcia Zangrilli from WM College 1. Observes incoming/outgoing packets 2. Online analysis to derive latency/bandwidth information for all host pair connections 3. Answers network queries for any pair of hosts What does it do ?
    83. 83. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments An important Contribution: Problem Formalization
    84. 84. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Two approaches to adaptation
    85. 85. VTTIF in a dynamic topology environment Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Parameters: Smoothing Window  sliding window duration over which updates are aggregated Update Rate Detection Threshold
    86. 86. Questions <ul><li>Are these techniques applicable to other applications ? </li></ul><ul><li>Why do you think they are generic enough even for BSP applications ? How did you test their generality ? </li></ul><ul><ul><ul><li>Because the reasoning for their derivation was based on the BSP model not on any particular application </li></ul></ul></ul><ul><ul><ul><li>Tested on Patterns and then tried on other more complex applications. </li></ul></ul></ul><ul><ul><ul><li>VTTIF was used by other components like adaptation </li></ul></ul></ul><ul><ul><li>What modifications would you suggest to the </li></ul></ul><ul><ul><ul><li>Network hardware </li></ul></ul></ul><ul><ul><ul><li>VMM </li></ul></ul></ul><ul><ul><ul><li>Router level ? </li></ul></ul></ul><ul><ul><li>How could you improve time synchronization issues ? </li></ul></ul><ul><ul><li>What is the closed work to you and how is it different from whats out there </li></ul></ul><ul><ul><ul><li>Black box inference NSDI 2007 </li></ul></ul></ul><ul><ul><ul><li>Black box debugging from HP labs </li></ul></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    87. 87. <ul><li>How about applying the techniques beyond BSP applications ? </li></ul><ul><ul><ul><li>Distributed apps </li></ul></ul></ul><ul><ul><ul><li>P2P </li></ul></ul></ul><ul><ul><ul><li>Web apps </li></ul></ul></ul><ul><ul><li>What are the limitations of these techniques ? When will they not work ? What are the assumptions ? </li></ul></ul><ul><ul><li>What is the main research contribution to your work..? </li></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    88. 88. Bin
    89. 89. Evidence of Adaptation Driven by Inference
    90. 90. Evidence of Adaptation <ul><li>How can automated inference assist automated adaptation without user intervention and application knowledge ? </li></ul><ul><li>Significant work already done in this area by me </li></ul><ul><li>Using multiple mechanisms for adaptation </li></ul><ul><li>Combining resource availability with application demands to match needs </li></ul><ul><li>Non-scientific apps also investigated (multi-tier web sites) </li></ul><ul><li>More in following slides… </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    91. 91. <ul><li>Informally stated: </li></ul><ul><li>Input </li></ul><ul><ul><li>Network traffic load matrix of application </li></ul></ul><ul><ul><li>Topology of the network </li></ul></ul><ul><li>Output </li></ul><ul><ul><li>Mapping of VMs to hosts </li></ul></ul><ul><ul><li>Overlay topology connecting hosts </li></ul></ul><ul><ul><li>Forwarding rules on the topology </li></ul></ul><ul><ul><li>Such that the application throughput is maximized </li></ul></ul>Optimization Problem (2/2) Topology + Migration Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments The algorithm is described in detail in the paper
    92. 92. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Effect on BSP Application Throughput For high Compute/Communicate Ratio, Migration + Topology dramatically improves performance Adapting to External Load Imbalance External load removed, but I/O still dominates, so topology helps External load removed, can drive higher I/O
    93. 93. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Evaluation Scenario 2 : Large 256 host topology. 32 potential hosts, 8 Virtual Machines Results for Multi Constraint Cost Function : Bandwidth and Latency Annealing easy to adapt and finds good mappings compared to heuristic
    94. 94. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments The New Adaptation Process
    95. 95. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Two approaches to adaptation
    96. 96. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Evaluation Scenario 2 : Large 256 host topology. 32 potential hosts, 8 Virtual Machines Results for Multi Constraint Cost Function : Bandwidth and Latency Annealing easy to adapt and finds good mappings compared to heuristic
    97. 97. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Summary of contributions Virtual Topology and Traffic Inference Framework Black box metrics for Absolute Performance Ball in The Court principles to compute application slowdown Global Bottlenecks using time decomposition Increasing Application Performance In Virtual Environments through Run-time Inference and Adaptation Free Network Measurement For Adaptive Virtualized Distributed Computing <ul><li>Infer the traffic matrix for a BSP application </li></ul><ul><li>Spatial Communication Topology of the application </li></ul><ul><li>Online implementation and low performance overhead </li></ul><ul><li>Handling dynamic situations and avoiding oscillations </li></ul><ul><li>Integration with adaptation system </li></ul>
    98. 98. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Summary of contributions Virtual Topology and Traffic Inference Framework Black box metrics for Absolute Performance Ball in The Court principles to compute application slowdown Global Bottlenecks using time decomposition Increasing Application Performance In Virtual Environments through Run-time Inference and Adaptation Free Network Measurement For Adaptive Virtualized Distributed Computing <ul><li>How to evaluate BSP application performance under black box conditions ? </li></ul><ul><li>New set of metrics based on traffic </li></ul><ul><li>RIR (Round trip Iteration Rate)  correlated to the iteration rate </li></ul><ul><li>Complex metrics for dynamic applications </li></ul><ul><ul><li>RIR avg </li></ul></ul><ul><ul><li>RIR-CDF </li></ul></ul><ul><ul><li>RIR-PS </li></ul></ul><ul><li>Scheduling using these advanced metrics (Slowdown CDF) </li></ul><ul><li>Other applications beyond performance </li></ul><ul><ul><li>Fingerprinting </li></ul></ul><ul><ul><li>Statistical Scheduling </li></ul></ul>
    99. 99. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Summary of contributions Virtual Topology and Traffic Inference Framework Black box metrics for Absolute Performance Ball in The Court principles to compute application slowdown Global Bottlenecks using time decomposition Increasing Application Performance In Virtual Environments through Run-time Inference and Adaptation Free Network Measurement For Adaptive Virtualized Distributed Computing <ul><li>Quantitative estimate of the application slowdown under external load </li></ul><ul><li>Amount of speedup achievable </li></ul><ul><li>Can greatly help in adaptation/scheduling decisions  predicts the benefit of migration in advance </li></ul><ul><li>Concept of Ball In the Court delays </li></ul><ul><li>Using BIC delays to compute imbalance in application </li></ul><ul><li>Algorithms to compute slowdown </li></ul><ul><ul><li>Global Imbalance Algorithm </li></ul></ul><ul><ul><li>Process Level BIC imbalance algo </li></ul></ul><ul><ul><li>Multi-load BIC imbalance algo </li></ul></ul><ul><li>Metrics to indicate amount of imbalance in application as a whole </li></ul>
    100. 100. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Summary of contributions Virtual Topology and Traffic Inference Framework Black box metrics for Absolute Performance Ball in The Court principles to compute application slowdown Global Bottlenecks using time decomposition Increasing Application Performance In Virtual Environments through Run-time Inference and Adaptation Free Network Measurement For Adaptive Virtualized Distributed Computing <ul><li>How can we find the global bottleneck for the application ? </li></ul><ul><li>Time Decomposition of execution time  Helps identify regions of great imbalance </li></ul><ul><li>Includes the network aspects </li></ul><ul><li>Many new metrics for latency and bandwidth time components </li></ul><ul><li>Evaluation with various load scenarios </li></ul>
    101. 101. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Summary of contributions Virtual Topology and Traffic Inference Framework Black box metrics for Absolute Performance Ball in The Court principles to compute application slowdown Global Bottlenecks using time decomposition Increasing Application Performance In Virtual Environments through Run-time Inference and Adaptation Free Network Measurement For Adaptive Virtualized Distributed Computing
    102. 102. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Summary of contributions Virtual Topology and Traffic Inference Framework Black box metrics for Absolute Performance Ball in The Court principles to compute application slowdown Global Bottlenecks using time decomposition Increasing Application Performance In Virtual Environments through Run-time Inference and Adaptation Free Network Measurement For Adaptive Virtualized Distributed Computing
    103. 103. Overview of the Talk <ul><li>Introduction and Background </li></ul><ul><li>Overview of contributions </li></ul><ul><li>Dynamic Topology Inference </li></ul><ul><li>Measures of Absolute Performance </li></ul><ul><li>Ball in the Court Delays and predicting unloaded performance </li></ul><ul><li>Global Bottleneck Detection </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    104. 104. Overview of the Talk <ul><li>Runtime Adaptation using black box inference </li></ul><ul><li>Contributions Recap </li></ul><ul><li>Conclusion </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    105. 105. Virtual Machine Distributed Computing <ul><li>Advantages </li></ul><ul><ul><ul><li>Helps overcome heterogeneity of resources, middleware, operating systems </li></ul></ul></ul><ul><ul><ul><li>Easier to administer (Virtual administration) </li></ul></ul></ul><ul><ul><ul><li>Autonomic Resource Management </li></ul></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    106. 106. Impact <ul><li>Further the goal of autonomic computing </li></ul><ul><li>Black box approach  impacts huge set of applications </li></ul><ul><li>Help in lower the entry barriers for distributed and parallel autonomic computing </li></ul><ul><li>Shared resources  no new tweaks needed to existing applications </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    107. 107. Impact Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments <ul><li>Adaptation is just one application for inference </li></ul><ul><ul><li>Application/Resource management and accounting </li></ul></ul><ul><ul><li>Dynamic problem detection and debugging </li></ul></ul><ul><ul><ul><li>Bottleneck detection </li></ul></ul></ul><ul><ul><li>Intrusion detection </li></ul></ul><ul><ul><ul><li>Abnormal behavior of part of application </li></ul></ul></ul><ul><ul><ul><li>Poor correlation with others </li></ul></ul></ul>
    108. 108. Application Model <ul><li>BSP Parallel applications </li></ul><ul><li>Two benchmarks being considered </li></ul><ul><ul><ul><li>Patterns </li></ul></ul></ul><ul><ul><ul><li>NAS Benchmarks </li></ul></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    109. 109. Citations [167] VALIANT, L. A bridging model for parallel computation. Communications of the ACM 33 , 8 (Aug. 1990), 103–111.
    110. 110. Two threads to my dissertation <ul><li>Application inference </li></ul><ul><ul><li>Inference has to be fully automated – zero intervention </li></ul></ul><ul><ul><ul><li>And no application or OS changes </li></ul></ul></ul><ul><ul><li>Research question: Whether and to what extent can we infer the application’s demands and behavior using only passive observations i.e. black box model ? </li></ul></ul><ul><ul><ul><li>E.g. Computational load, communication behavior, application topology, performance behavior, prediction and current bottlenecks </li></ul></ul></ul><ul><ul><li>System inference not part of my thesis… </li></ul></ul><ul><ul><ul><li>But I’ve contributed there too </li></ul></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    111. 111. Two threads to my dissertation <ul><li>2. Applying inference: Autonomic adaptation </li></ul><ul><ul><ul><li>Virtuoso implements dynamic adaptation according to changing application demands </li></ul></ul></ul><ul><ul><ul><li>Inference provides updated view </li></ul></ul></ul><ul><ul><ul><li>Different adaptation mechanisms like VM migration, overlay networking etc. </li></ul></ul></ul><ul><ul><ul><li>Adapt application if performance degrades or opportunity exists to boost performance </li></ul></ul></ul><ul><ul><li>Limited evidence of how inference-adaptation combination can boost performance </li></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    112. 112. Other inference properties <ul><li>Traffic topology for BSP-style distributed applications – work done in my initial VTTIF paper </li></ul><ul><li>Measuring performance of a BSP application </li></ul><ul><li>Computing affect of load on a BSP application using passive measurements </li></ul><ul><li>Finding global bottlenecks in a BSP application </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    113. 113. Impact <ul><li>Was the first paper to propose and demonstrate collective inference for a virtual distributed environment </li></ul><ul><li>Was my first Virtuoso project </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    114. 114. A VNET virtual layer <ul><li>VNET </li></ul><ul><ul><li>Abstraction: A set of VMs on same Layer 2 network </li></ul></ul><ul><ul><li>Virtual Ethernet LAN </li></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments VNET Layer Physical Layer A Virtual LAN over wide area
    115. 115. Overall Design <ul><li>VNET </li></ul><ul><ul><li>Abstraction: A set of VMs on same Layer 2 network </li></ul></ul><ul><li>Extend VNET to include the required features </li></ul><ul><ul><li>Monitoring at Ethernet packet level </li></ul></ul><ul><li>The Challenge here </li></ul><ul><ul><li>Lacks manual control </li></ul></ul><ul><ul><li>How to detect interesting parallel program communication ? </li></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    116. 116. Dynamic Topology Inference by VTTIF Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments 1. Fast updates Smoothed Traffic Matrix 2. Low Pass Filter Aggregation 3. Threshold change detection Topology change output VNET Daemons on Hosts VNET Daemon at Proxy Aggregated Traffic Matrix
    117. 117. Contributions <ul><li>Understanding the correlation between network traffic and inter-process interactions </li></ul><ul><li>Defining a black box measure of application performance (RIR) </li></ul><ul><ul><ul><li>Techniques to derive it </li></ul></ul></ul><ul><ul><li>Advanced metrics for Dynamic Applications and their performance </li></ul></ul><ul><ul><li>Frequency Domain based metrics for greater insight about application </li></ul></ul><ul><ul><li>Applications to advanced scheduling and application fingerprinting </li></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    118. 118. Solving the issues <ul><li>Sampling rate: </li></ul><ul><ul><ul><li>Compute at a “high enough” sample rate to capture enough high frequency dynamics </li></ul></ul></ul><ul><ul><ul><li>Compute power spectrum to calculate total energy in the signal (power spectrum is magnitude of Fourier Transform )  Integrate the power spectrum </li></ul></ul></ul><ul><ul><ul><li>Sample at a slightly lower rate and see reduction in energy. </li></ul></ul></ul><ul><ul><ul><li>Repeat until we start losing energy beyond a certain threshold (5%)  this indicates at this rate we start losing significant part of signal </li></ul></ul></ul><ul><ul><ul><li>If actual sampling rate is much higher (5 to 10 times) than this cutoff point, we are doing alright </li></ul></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Gives us 95% cutoff sampling rate
    119. 119. Solving the issues Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    120. 120. Capturing the right time duration <ul><li>Again a Power Spectrum based approach </li></ul><ul><li>Helps us determine if our sample size captures most of the energy in the signal </li></ul><ul><li>Ideally we will capture as much as resources allow us </li></ul><ul><li>The process: </li></ul><ul><ul><ul><li>After capturing, we test the amount of energy loss, if we shorten the sample size. </li></ul></ul></ul><ul><ul><ul><li>Find the cut-off point as last time </li></ul></ul></ul><ul><ul><ul><li>Compare the cut-off sample size and our measured sample size </li></ul></ul></ul><ul><ul><li>After sampling rate and sampling duration is satisfactory, </li></ul></ul><ul><ul><li>RIR avg computed by averaging the time series </li></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    121. 121. Effect of load on the RIR time series Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments 1. Expansion effect 2. Basic structure of time series is same 3. Each phase more irregular (more spikes) 4. Captures the time dynamics of the application
    122. 122. The RIR-CDF metric <ul><li>More exhaustive than the RIR avg </li></ul><ul><li>Statistical scheduling </li></ul><ul><li>Application fingerprint </li></ul><ul><li>Performance comparison of runs at more detailed level than RIR avg </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Global metric
    123. 123. The RIR-CDF metric Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments CDF indicates the RIR region application spends most of its time in RIR-CDF can be used to predict which application will be more affected by external load (for dynamic applications) I show how we can predict slowdown for MG vs IS Slowdown mapping
    124. 124. Taking guidance from RIR-CDF : a proof of concept <ul><li>CDF indicates the RIR region application spends most of its time in </li></ul><ul><li>Scheduling fact: Depending the scheduling algorithm, affect of load on different RIR regions is different (Govindan et.al.) </li></ul><ul><li>Reason: Scheduling handled differently for CPU bound processes vs I/O bound processes </li></ul><ul><li>“ Providing enough CPU is not enough, an equally important consideration is to provide the CPU at the right time” </li></ul><ul><li>Higher RIR  more inter-process interaction  can use more efficient communication </li></ul><ul><li>Lower RIR  more compute intensive or large messages </li></ul><ul><li>How do applications with different RIRs suffer from external load ? </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    125. 125. Effect of load on different RIR areas Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Difference of performance over 12.46 times at the extremes under full load Applications with higher iteration rates always seem to be more drastically affected by external load then those with lower iteration rates.
    126. 126. Scheduling using the CDF Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments <ul><ul><li>The slowdown mapping can be complicated and built over a long history. We use results from Patterns benchmark to provide a simpler mapping </li></ul></ul>Input: RIR value Output: Fractional slowdown Assumed load = 100% More realistic mappings may take bandwidth and CPU utilization as input
    127. 127. Scheduling using the CDF Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments For the IS and MG applications we have seen till now, which application may be hurt more if one of the processes from each application shares the physical host with an external computational load? The impact: we can now determine in advance , the impact of external load if we must choose one of these applications to be influenced by the load. Map each RIR value to its slowdown RIR value from the mapping Slowdown CDF
    128. 128. Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments Slowdown CDF For MG Slowdown CDF For IS More time spent in low regions Avg = 0.103 Avg = 0.065
    129. 129. Actual slowdown <ul><li>MG  2.4 times </li></ul><ul><li>IS  15.52 times </li></ul><ul><li>IS is affected much more from the load as predicted. </li></ul><ul><li>A better mapping function can yield more powerful slowdown estimates for complex applications. </li></ul><ul><li>Shows how we can make complex decisions for dynamic applications using RIR – CDF. </li></ul><ul><li>Other applications  Statistical scheduling ideas </li></ul><ul><ul><ul><li>examples </li></ul></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    130. 130. Power Spectrum <ul><ul><li>Other applications of the Power Spectrum </li></ul></ul><ul><ul><ul><li>Statistical Scheduling decisions </li></ul></ul></ul><ul><ul><ul><li>Application categorization </li></ul></ul></ul><ul><ul><ul><li>Component separation </li></ul></ul></ul><ul><ul><li>More discussion in dissertation </li></ul></ul><ul><ul><li>Steps in computing the Power Spectrum – numerous measures taken to get a representative power spectrum (Windowing functions, zero padding, eliminating the DC bias) </li></ul></ul><ul><li>Evaluation with another application: IS (Integer Sort) from the NAS benchmarks  similar results </li></ul><ul><li>Implementation details and making it work online </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    131. 131. Why do I want to do this? <ul><li>Close synchronization in BSP: a single slow process can slowdown entire application drastically </li></ul><ul><li>IS application  One process loaded  15 fold increase in execution time </li></ul><ul><li>Same external load affects different applications differently </li></ul><ul><li>No uniform decision making method for improving performance for loaded applications </li></ul><ul><li>Therefore, very useful to know the impact of any decisions beforehand before actually executing the actual steps of adaptation </li></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    132. 132. The Benefits <ul><li>If we can know how badly the application is affected from the load, </li></ul><ul><li>Appropriate steps can be taken to alleviate the load, especially in multi-application scenarios </li></ul><ul><ul><ul><li>Migrating the process VM </li></ul></ul></ul><ul><ul><ul><li>Moving the external load </li></ul></ul></ul><ul><li>Constraints : </li></ul><ul><ul><ul><li>Black box, no intrusion into the application </li></ul></ul></ul><ul><ul><ul><li>No extra loading or removal of load (no perturbation) </li></ul></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments
    133. 133. Process Level BIC imbalance Algorithm <ul><li>A better fit for more dynamic applications like the NAS benchmarks </li></ul><ul><li>Sometimes work done by all processes is not the same </li></ul><ul><li>Algorithm takers into account inter-process level interactions and BIC delays </li></ul><ul><li>Phenomenon of Load Bias also observed  A heavily loaded process affects the BIC delay of other unloaded processes too (shifting of work) </li></ul><ul><li>Covered in detail in the dissertation </li></ul><ul><li>Gives a range of slowdown (optimistic and pessimistic values) </li></ul><ul><li>Evaluation with MG </li></ul><ul><ul><ul><li>Range of slowdown = [28.67s, 50.78s ] </li></ul></ul></ul><ul><ul><ul><li>Actual slowdown = 42.67s </li></ul></ul></ul>Black Box Methods for Inferring Parallel Applications' properties in Virtual Environments

    ×